Method and system for time spectrum radiation homogenization processing of sequence data under reference data constraint

Abstract

The application discloses a sequence data timespectrum radiation unification processing method and system under reference data constraints, and the method comprises the following steps: acquiring a remote sensing image sequence and generating a multi-source radiation normalization image database to form a multi-source reference sequence data pair; a reconstruction self-similarity normalization pre-training model is constructed, the reconstruction self-similarity normalization pre-training model comprises a decomposition and reconstruction module, a self-similarity weight matrix module and a variational normalization module, the decomposition and reconstruction module obtains a final timespectrum feature sequence, and the self-similarity weight matrix module obtains a timespectrum self-similarity weight matrix; the variational normalization module processes to form timespectrum radiation consistent sequence data; and corresponding timespectrum radiation consistent sequence data is obtained based on the reconstruction self-similarity normalization model. The application constructs a variational model framework of timespectrum normalization, cooperates self-similarity weight, timespectrum self-regression and similarity, realizes timespectrum radiation normalization of a timespectrum sequence cube, and solves timeserial remote sensing data which is accurate in trend and faithful in spectrum.

Description

Technical Field

[0001] This invention relates to the field of remote sensing data processing technology, and in particular to a method and system for processing temporal spectrum radiometric uniformity of sequence data under reference data constraints. Background Technology

[0002] Massive amounts of satellite remote sensing Earth observation data provide abundant data resources for various applications, possessing powerful service capabilities. Remote sensing classification is essential for ensuring the accuracy of data in large-scale geographical surveys and natural resource investigations. However, current practical applications face certain limitations. For example, despite the abundance of data, issues such as data gaps, insufficient spatiotemporal integrity, and inadequate comparability often hinder in-depth technological development and the widespread application of methods. The crux of the problem lies in the incompatibility and inconsistency between multi-source data, limiting data integration and collaborative applications. This issue also affects the generalization ability of classification methods. Deep learning technology, due to its advantages in feature mining and solving nonlinear problems, performs well in classification tasks. However, consistent distribution between test and training data is a prerequisite for model generalization. Inconsistent radiation between multi-source data exacerbates dataset bias, reducing the transfer and generalization capabilities of deep learning models across time, space, and sensors, which is particularly pronounced in large-area classification tasks.

[0003] Even after spatial radiometric normalization, optical remote sensing time-series data still suffer from problems such as temporal offset and disordered temporal-spectral relationships. These issues arise primarily because the temporal-spectral radiometric normalization process for the sequence data is not precise enough under the constraint of reference data. Temporal-spectral radiometric normalization is an effective means to address these problems. However, existing normalization methods lack absolute reference information constraints and have insufficient depth in mining and utilizing temporal-spectral information, resulting in spatial normalization failing to provide accurate and reliable temporal-spectral information for time-series classification. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for processing temporal spectral radiation uniformity of sequence data under reference data constraints.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A method for unifying the temporal spectral radiation of sequence data under reference data constraints includes the following steps:

[0007] Acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Randomly select two sets of remote sensing image sequences to form a multi-source reference sequence data pair.

[0008] A reconstructed self-similar normalized pre-trained model is constructed, comprising a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation and inter-spectral constraints of multi-source data, and optimizes the time-spectral self-similar weight matrix based on the optimization model to form time-spectral radiation consistent sequence data.

[0009] The reconstructed self-similar normalization pre-trained model was trained using a multi-causal radiation normalization database to obtain the reconstructed self-similar normalization model.

[0010] The reconstructed self-similar normalization model is used to process the multi-source reference sequence data pairs to be normalized, and the corresponding time-spectral radiometric consistent sequence data is obtained.

[0011] As one possible implementation, the decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectral analysis to obtain the final time-spectral feature sequence, including the following steps:

[0012] Obtain the temporal spectrum sequence of the multi-source reference sequence data pair, wherein the temporal spectrum sequence is formed based on the pixel temporal spectrum vector, and the pixel temporal spectrum vector is formed by the change of each pixel over time;

[0013] Based on the pixel temporal spectrum vector and the preset window length, the temporal spectrum sequence is arranged with lag to obtain the trajectory matrix;

[0014] The trajectory matrix is ​​decomposed using the singular value decomposition algorithm, yielding multiple submatrices and their corresponding singular values, left singular vector, and right singular vector, as follows: , ;

[0015] Based on the numerical value of the singular values, all submatrices are grouped. The submatrix corresponding to the singular value with the largest value is reconstructed. Then, the reconstructed submatrices are transformed into the final temporal spectral eigenvector through a diagonal averaging process, as follows: ;

[0016] in, Submatrices representing the trajectory matrix, expressed by singular values Left singular vector and right singular vectors Calculations show that Represents the final temporal spectral eigenvector. Represents pixel values ​​in different spectral bands. Represents the data in the submatrix. , , Represents the time spectrum vector. Indicates the preset window length. Represents the trajectory matrix.

[0017] As one possible implementation, the feature encoding includes the following steps:

[0018] The final temporal feature sequence is used as the input image. Based on each pixel position in the input image, a first feature map of the corresponding size is obtained after 1×1 convolution.

[0019] The first feature maps from different channels are fused while maintaining the size of the first feature maps to obtain a fused feature map.

[0020] Spatial coding is added to the fused feature map to recover the original sequence arrangement, resulting in a coded feature map with coding information. In this coded feature map, the feature vector of each pixel is represented as... .

[0021] As one possible implementation, the self-attention processing, which yields a temporal spectral self-similarity weight matrix, includes the following steps:

[0022] For each pixel vector in the encoded feature map, a projection matrix is ​​used... , and Convert to query embedding, key embedding, and value embedding; calculate query embedding based on dot product. With key embedding The similarity between the values ​​is scaled according to a preset ratio to obtain the scaled result, which is then transformed by the softmax function to obtain the normalized attention score. ;

[0023] Based on the spatial location information encoded by location, the attention scores of each pixel feature vector are recombined to make them consistent with the spatial dimension of the input image, thereby constructing a temporal self-similarity weight matrix.

[0024] in, , , , , , and These represent the projection matrices, Indicates query embedding, Indicates key embedding, Indicates value embedding, This represents the attention score. Indicates the preset ratio. The input sequence represents pixel-by-pixel features.

[0025] As one possible implementation, the input image is represented as M×N×(C×T), and the first feature map is represented as M×N×k;

[0026] Then the first A convolutional kernel, represented as: ;

[0027] A 1×1 convolution is represented as: ;

[0028] in, Indicates the location of the first feature map The The value of each channel, Indicates the position of the input image. The The value of each channel, Indicates the first The first convolutional kernel Each weight, Indicates the first The bias term of each convolutional kernel, where M represents height, N represents width, C represents the number of channels, and T represents the time step. This represents the position of each pixel in the first feature map.

[0029] As one possible implementation, the variational normalization module forms an optimization model based on the temporal-spectral correlation and inter-spectral constraints of multi-source data, including the following steps:

[0030] Based on multi-source reference sequence data pairs, the temporal spectral correlation and inter-spectral constraint relationships of multi-source data are obtained;

[0031] An optimization model is constructed by using the temporal spectrum self-similarity weight matrix, the temporal spectrum correlation relationship of multi-source data, and the inter-spectral constraint relationship, so that the spatiotemporal weight of the kernel function of the overall data space in the optimization result reaches the minimum value.

[0032] The temporal-spectral correlation and inter-spectral constraints of the multi-source data are expressed as follows:

[0033] ;

[0034] ;

[0035] The optimization model is expressed as follows:

[0036] ;

[0037] in, Represents the overall data space. This represents the spatial downsampling relationship between the time-series multispectral image after spectral radiometric normalization and the external reference image. This represents the denoised, time-series multispectral image with normalized spectral radiation. Indicates the first Each pixel in The temporal spectrum self-similarity weight matrix is ​​as follows. , and These represent combinations of different temporal, spatial, and spectral resolutions of multi-source reference temporal multispectral image data, respectively. The subscripts indicate... , They represent the first , No. One pixel, , , , , These represent the data points of the time-spectrum curves for different reference data. The spatiotemporal weights of the kernel function are represented. This represents the temporal spectrum self-similarity weight matrix. and These represent the relationship between the image to be solved and the input multi-source reference time-series image data, respectively. This indicates that a time-spectral autoregressive constraint is being considered. This represents the similarity constraint representation of time-spectrum curves. This represents the time-spectral autoregressive weight coefficient.

[0038] As one possible implementation, the multi-source radiation normalized image database includes the following steps:

[0039] The atmospheric radiative transfer process was simulated based on the atmospheric radiative transfer model, and remote sensing observation data under different atmospheric conditions were obtained.

[0040] The data from different sensors are converted based on the spectral response function to obtain a converted remote sensing image dataset. Specifically, the spectral response function is obtained, and the response value of one sensor is converted to the band of another sensor based on the spectral response function. By comparing the spectral response functions of the two sensors, the spectral matching conversion factor between the two sensors is calculated. The conversion factor is then applied to the original data to obtain the converted remote sensing image dataset.

[0041] The transformed remote sensing image dataset was resampled using area weighting to generate a multi-causal radiation normalized database.

[0042] Among them, image resampling by area weighting specifically involves calculating the actual ground area of ​​each pixel in the transformed remote sensing image;

[0043] The weight of each pixel is obtained by dividing the actual ground area of ​​each pixel by the sum of the areas of all pixels. Then, a weighted average is obtained through the weight of each pixel to achieve image resampling.

[0044] A system for unifying the temporal spectrum radiation of sequence data under reference data constraints includes a database construction module, a model construction module, a model training module, and a model inference module.

[0045] The database construction module is used to acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Two sets of remote sensing image sequences are arbitrarily selected to form a multi-source reference sequence data pair, where one set of remote sensing image sequences is used as the first reference data and the other set of remote sensing image sequences is used as the second reference data.

[0046] The model building module is used to construct a reconstructed self-similar normalized pre-trained model. The reconstructed self-similar normalized pre-trained model includes a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation and inter-spectral constraint relationship of multi-source data. Based on the optimization model, the time-spectral self-similar weight matrix is ​​optimized to form time-spectral radiation consistent sequence data.

[0047] The model training module is used to train the reconstructed self-similar normalized pre-trained model through a multi-causal radiation normalization database to obtain the reconstructed self-similar normalized model.

[0048] The model inference module processes the multi-source reference sequence data pairs to be normalized based on the reconstructed self-similar normalization model to obtain the corresponding time-spectral radiation consistent sequence data.

[0049] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the following method:

[0050] Acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Randomly select two sets of remote sensing image sequences to form a multi-source reference sequence data pair.

[0051] A reconstructed self-similar normalized pre-trained model is constructed, comprising a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation and inter-spectral constraints of multi-source data, and optimizes the time-spectral self-similar weight matrix based on the optimization model to form time-spectral radiation consistent sequence data.

[0052] The reconstructed self-similar normalization pre-trained model was trained using a multi-causal radiation normalization database to obtain the reconstructed self-similar normalization model.

[0053] The reconstructed self-similar normalization model is used to process the multi-source reference sequence data pairs to be normalized, and the corresponding time-spectral radiometric consistent sequence data is obtained.

[0054] A temporal spectral radiometric homogenization processing apparatus for sequence data under reference data constraints includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the following method:

[0055] Acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Randomly select two sets of remote sensing image sequences to form a multi-source reference sequence data pair.

[0056] A reconstructed self-similar normalized pre-trained model is constructed, comprising a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation and inter-spectral constraints of multi-source data, and optimizes the time-spectral self-similar weight matrix based on the optimization model to form time-spectral radiation consistent sequence data.

[0057] The reconstructed self-similar normalization pre-trained model was trained using a multi-causal radiation normalization database to obtain the reconstructed self-similar normalization model.

[0058] The reconstructed self-similar normalization model is used to process the multi-source reference sequence data pairs to be normalized, and the corresponding time-spectral radiometric consistent sequence data is obtained.

[0059] This invention, by adopting the above technical solutions, has significant technical effects:

[0060] The invention provides a time-spectral radiation normalization method under reference data constraints. The radiation normalization is classification-oriented and adopts a combination of variational models and deep learning to mine the spatiotemporal spectral correlation of data, integrate prior knowledge, and deal with the problems of complex nonlinear radiation inconsistency and global optimization. The causes of radiation inconsistency between multi-source data are numerous and complex. In order to achieve high-quality spatiotemporal radiation normalization, different scenarios need to be considered during model training, and a normalization database needs to be constructed. In addition to collecting radiometrically normalized real image pairs as the database data source, the system will also simulate sensor data under different imaging conditions based on the radiative transfer equation, simulate multi-sensor data with different spectral settings based on the spectral response function, and conduct multi-scale data simulation considering scale effects based on the mixed pixel theory. With the reference of the temporal spectral evolution information of the external data, the system will obtain time trend and fluctuation information through signal decomposition and reconstruction, mine the self-similarity weights of the temporal spectral relationship of the reference data through network learning, and mine the temporal spectral information of the temporal spectral sequence data. By learning the self-similarity weights, stable features will be screened and enhanced, and a variational model framework for temporal spectral normalization will be constructed. By coordinating the self-similarity weights, temporal spectral autoregression, and similarity, the temporal spectral radiometric normalization of the temporal spectral sequence cube will be achieved, and the solution will yield time-series remote sensing data with accurate trends and spectral fidelity. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0063] Figure 2 This is a schematic diagram of the overall structure of the system of the present invention;

[0064] Figure 3 This is a schematic diagram of the overall process of a specific embodiment of the present invention. Detailed Implementation

[0065] The present invention will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0066] A method for unifying the temporal spectral radiation of sequence data under reference data constraints, such as... Figure 1 and Figure 3 As shown, it includes the following steps:

[0067] S100. Acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Randomly select two sets of remote sensing image sequences to form a multi-source reference sequence data pair, with one set of remote sensing image sequences serving as the first reference data and the other set of remote sensing image sequences serving as the second reference data.

[0068] S200. Construct a reconstructed self-similar normalized pre-trained model, which includes a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation relationship and inter-spectral constraint relationship of multi-source data, and optimizes the time-spectral self-similar weight matrix based on the optimization model to form time-spectral radiation consistent sequence data.

[0069] S300. The reconstructed self-similar normalization pre-trained model is trained using a multi-causal radiation normalization database to obtain the reconstructed self-similar normalization model;

[0070] S400. Based on the reconstructed self-similar normalization model, the normalized multi-source reference sequence data pairs are processed to obtain the corresponding time-spectral radiometric consistent sequence data.

[0071] The invention provides a time-spectral radiation normalization method under reference data constraints. The radiation normalization is classification-oriented and adopts a combination of variational models and deep learning to mine the spatiotemporal spectrum correlation of data, integrate prior knowledge, and deal with the problems of complex nonlinear radiation inconsistency and global optimization. The causes of radiation inconsistency between multi-source data are numerous and complex. In order to achieve high-quality spatiotemporal radiation normalization, different scenarios need to be considered during model training, and a normalization database needs to be constructed. In addition to collecting radiometrically normalized real image pairs as the database data source, the system will also simulate sensor data under different imaging conditions based on the radiative transfer equation, simulate multi-sensor data with different spectral settings based on the spectral response function, and conduct multi-scale data simulation considering scale effects based on the mixed pixel theory. With the reference of the temporal spectral evolution information of the external data, the system will obtain time trend and fluctuation information through signal decomposition and reconstruction, mine the self-similarity weights of the temporal spectral relationship of the reference data through network learning, and mine the temporal spectral information of the temporal spectral sequence data. By learning the self-similarity weights, stable features will be screened and enhanced, and a variational model framework for temporal spectral normalization will be constructed. By coordinating the self-similarity weights, temporal spectral autoregression, and similarity, the temporal spectral radiometric normalization of the temporal spectral sequence cube will be achieved, and the solution will yield time-series remote sensing data with accurate trends and spectral fidelity.

[0072] In one embodiment, processing multi-source reference sequence data pairs typically requires noise removal to obtain regular and reliable time-series data trends. Singular spectral analysis (SSA) is a time-series analysis and prediction technique that decomposes a raw sequence into several independent components or subsequences. These components can be interpreted as trends, oscillations, or noise. Using spectral pixels or band images as input, it effectively extracts spectral trends through four steps: embedding, decomposition, grouping, and reconstruction. The decomposition and reconstruction module, based on SSA, embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs to obtain the final time-spectral feature sequence, including the following steps:

[0073] Obtain the temporal spectrum sequence of the multi-source reference sequence data pair, wherein the temporal spectrum sequence is formed based on the pixel temporal spectrum vector, and the pixel temporal spectrum vector is formed by the change of each pixel over time;

[0074] Based on the pixel temporal spectrum vector and the preset window length, the temporal spectrum sequence is arranged with lag to obtain the trajectory matrix;

[0075] The trajectory matrix is ​​decomposed using the singular value decomposition algorithm, yielding multiple submatrices and their corresponding singular values, left singular vector, and right singular vector, as follows: , ;

[0076] Based on the numerical value of the singular values, all submatrices are grouped. The submatrix corresponding to the singular value with the largest value is reconstructed. Then, the reconstructed submatrices are transformed into the final temporal spectral eigenvector through a diagonal averaging process, as follows: ;

[0077] in, Submatrices representing the trajectory matrix, expressed by singular values Left singular vector and right singular vectors Calculations show that Represents the final temporal spectral eigenvector. Represents pixel values ​​in different spectral bands. Represents the data in the submatrix. , , Represents the time spectrum vector. Indicates the preset window length. This represents the trajectory matrix. Compared to the original spectral pixels, the data in the submatrix produces a smoother trend in the spectral feature vector. The pixel-by-pixel temporal feature vectors are then stitched together to generate a time-series remote sensing image with temporal noise eliminated.

[0078] In a specific embodiment, to avoid the loss of ground feature change information during temporal-spectral normalization, a convolutional neural network with a self-attention mechanism is proposed to learn a self-similar attention weight matrix for the temporal-spectral sequence data. Each pixel will learn a weight. These weights represent the importance of spatial location information. This spatial attention matrix is ​​appended to the original image to be normalized, increasing the utilization of stable features and weakening the influence of changing features, thereby achieving feature selection and enhancement. Feature encoding and self-attention processing to obtain the temporal-spectral self-similar weight matrix include the following steps:

[0079] The final temporal feature sequence is used as the input image. Based on each pixel position in the input image, a first feature map of the corresponding size is obtained after 1×1 convolution.

[0080] The first feature maps from different channels are fused while maintaining the size of the first feature maps to obtain a fused feature map.

[0081] Spatial coding is added to the fused feature map to recover the original sequence arrangement, resulting in a coded feature map with coding information. In this coded feature map, the feature vector of each pixel is represented as... ;

[0082] For each pixel vector in the encoded feature map, a projection matrix is ​​used... , and Convert to query embedding, key embedding, and value embedding; calculate query embedding based on dot product. With key embedding The similarity between the values ​​is scaled according to a preset ratio to obtain the scaled result, which is then transformed by the softmax function to obtain the normalized attention score. ;

[0083] Based on the spatial location information encoded by the location, the attention score of each pixel feature vector is recombined and kept consistent with the spatial dimension of the input image, thereby constructing a temporal self-similarity weight matrix.

[0084] in, , , , , , and These represent the projection matrices, Indicates query embedding, Indicates key embedding, Indicates value embedding, This represents the attention score. Indicates the preset ratio. The input sequence represents pixel-by-pixel features.

[0085] In other words, after obtaining the attention scores, spatial location encoding is injected, and each pixel receives a feature vector with location information. Subsequently, the attention scores of these vectors are rearranged and weighted according to the row and column coordinates of the original image, ultimately generating a "time-spectral self-similarity weight matrix" with the exact same spatial dimensions as the input image. This explicitly reflects the self-similarity between different time / spectral dimensions at the same spatial location, resulting in the final time-spectral self-similarity weight matrix.

[0086] In this embodiment, the input image is represented as M×N×(C×T), and the first feature map is represented as M×N×k;

[0087] Then the first A convolutional kernel, represented as: ;

[0088] A 1×1 convolution is represented as: ;

[0089] in, Indicates the location of the first feature map The The value of each channel, Indicates the position of the input image. The The value of each channel, Indicates the first The first convolutional kernel Each weight, Indicates the first The bias term of each convolutional kernel, where M represents height, N represents width, C represents the number of channels, and T represents the time step. This represents the position of each pixel in the first feature map.

[0090] In other words, the size is M N (C The image of T) is obtained through 1 The first convolutional layer obtains pixel-wise feature vectors, fuses features from different channels without changing the size of the feature map, and adds positional or spatial encoding to restore the original sequence arrangement. The pixel-wise feature representation vectors are then used. The output features are obtained by adding the feature vector and the positional encoding. The output features learn a self-similar attention weight matrix through self-attention blocks. In other words: the input embedding... First, it passes through a learnable projection matrix. , and Convert to query ,key Sum Embedded. Utilizing and Calculating attention scores using embedding dot product ,according to After scaling the scale, the similarity is normalized by the softmax function. The attention scores are restored and combined pixel by pixel using the spatial location information provided by the positional encoding to maintain the same spatial dimension as the input image, thus generating a normalized weight matrix.

[0091] In one embodiment, the variational normalization module forms an optimization model based on the temporal-spectral correlation and inter-spectral constraints of multi-source data, including the following steps:

[0092] Based on multi-source reference sequence data pairs, the temporal spectral correlation and inter-spectral constraint relationships of multi-source data are obtained;

[0093] An optimization model is constructed by using the temporal spectrum self-similarity weight matrix, the temporal spectrum correlation relationship of multi-source data, and the inter-spectral constraint relationship, so that the spatiotemporal weight of the kernel function of the overall data space in the optimization result reaches the minimum value.

[0094] The temporal-spectral correlation and inter-spectral constraints of the multi-source data are expressed as follows:

[0095] ;

[0096] ;

[0097] The optimization model is expressed as follows:

[0098] ;

[0099] in, Represents the overall data space. This represents the spatial downsampling relationship between the time-series multispectral image after spectral radiometric normalization and the external reference image. This represents the denoised, time-series multispectral image with normalized spectral radiation. Indicates the first Each pixel in The temporal spectrum self-similarity weight matrix is ​​as follows. , and These represent combinations of different temporal, spatial, and spectral resolutions of multi-source reference temporal multispectral image data, respectively. The subscripts indicate... , They represent the first , No. One pixel, , , , , These represent the data points of the time-spectrum curves for different reference data. The spatiotemporal weights of the kernel function are used to explore the optimal kernel function expression in ice and snow parameter fusion. This represents the temporal spectrum self-similarity weight matrix. and These represent the relationship between the image to be solved and the input multi-source reference time-series image data, respectively. This indicates that a time-spectral autoregressive constraint is being considered. This represents the similarity constraint representation of time-spectrum curves. This represents the time-spectral autoregressive weight coefficient.

[0100] In one embodiment, generating a multi-gene radiation normalization database includes the following steps:

[0101] The atmospheric radiative transfer process was simulated based on the atmospheric radiative transfer model, and remote sensing observation data under different atmospheric conditions were obtained.

[0102] The data from different sensors are converted based on the spectral response function to obtain a converted remote sensing image dataset. Specifically, the spectral response function is obtained, and the response value of one sensor is converted to the band of another sensor based on the spectral response function. By comparing the spectral response functions of the two sensors, the spectral matching conversion factor between the two sensors is calculated. The conversion factor is then applied to the original data to obtain the converted remote sensing image dataset.

[0103] The transformed remote sensing image dataset was resampled using area weighting to generate a multi-causal radiation normalized database.

[0104] Among them, image resampling by area weighting specifically involves calculating the actual ground area of ​​each pixel in the transformed remote sensing image;

[0105] The weight of each pixel is obtained by dividing the actual ground area of ​​each pixel by the sum of the areas of all pixels. Then, a weighted average is obtained through the weight of each pixel to achieve image resampling.

[0106] In this embodiment, the 6S model of atmospheric radiative transfer is used to simulate the atmospheric radiative transfer process and simulate remote sensing observation data under different atmospheric conditions. The 6S model is mainly used to simulate the transmission process of solar radiation in the Earth-atmosphere system and calculate the radiance at the satellite entrance pupil. It can consider the absorption of water vapor, CO2, O3 and O2, molecular and aerosol scattering, and the effects of non-uniform ground and bidirectional reflectance distribution under the assumption of a cloudless atmosphere. The input parameters of the 6S model include: satellite geometry, including solar zenith angle, solar azimuth angle, observation zenith angle, observation azimuth angle, etc.; atmospheric model, which defines the basic composition of the atmosphere and temperature and humidity profiles, and can select preset models or input measured data; aerosol model, which defines the main global aerosol parameters, such as aerosol phase function, asymmetry factor and single scattering albedo, etc.; sensor spectral characteristics, which defines the spectral response function of the sensor channel; and surface reflectance, which defines the surface reflectance model, including both uniform and non-uniform surfaces.

[0107] Example 2:

[0108] A system for processing temporal spectrum radiation uniformity of sequence data under reference data constraints includes a database construction module 100, a model construction module 200, a model training module 300, and a model inference module 400.

[0109] The database construction module 100 is used to acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Two sets of remote sensing image sequences are arbitrarily selected to form a multi-source reference sequence data pair, where one set of remote sensing image sequences is used as the first reference data and the other set of remote sensing image sequences is used as the second reference data.

[0110] The model building module 200 is used to construct a reconstructed self-similar normalized pre-trained model. The reconstructed self-similar normalized pre-trained model includes a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation relationship and inter-spectral constraint relationship of multi-source data. Based on the optimization model, the time-spectral self-similar weight matrix is ​​optimized to form time-spectral radiation consistent sequence data.

[0111] The model training module 300 is used to train the reconstructed self-similar normalized pre-trained model through a multi-causal radiation normalization database to obtain the reconstructed self-similar normalized model.

[0112] The model inference module 400 processes the multi-source reference sequence data pairs to be normalized based on the reconstructed self-similar normalization model to obtain the corresponding time-spectral radiation consistent sequence data.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0114] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] It should be noted that:

[0119] The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0120] Furthermore, it should be noted that the shapes and names of the parts and components described in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.

Claims

1. A method for unifying the temporal spectrum radiation of sequence data under reference data constraints, characterized in that, Includes the following steps: Acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Randomly select two sets of remote sensing image sequences to form a multi-source reference sequence data pair. A reconstructed self-similar normalized pre-trained model is constructed, comprising a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation and inter-spectral constraints of multi-source data, and optimizes the time-spectral self-similar weight matrix based on the optimization model to form time-spectral radiation consistent sequence data. The reconstructed self-similar normalization pre-trained model was trained using a multi-causal radiation normalization database to obtain the reconstructed self-similar normalization model. Based on the reconstructed self-similar normalization model, the normalized multi-source reference sequence data pairs are processed to obtain the corresponding temporal-spectral radiometric consistent sequence data. The variational normalization module forms an optimization model based on the temporal-spectral correlation and inter-spectral constraints of multi-source data, including the following steps: Based on multi-source reference sequence data pairs, the temporal spectral correlation and inter-spectral constraint relationships of multi-source data are obtained; An optimization model is constructed by using the temporal spectrum self-similarity weight matrix, the temporal spectrum correlation relationship of multi-source data, and the inter-spectral constraint relationship, so that the spatiotemporal weight of the kernel function of the overall data space in the optimization result reaches the minimum value. The temporal-spectral correlation and inter-spectral constraints of the multi-source data are expressed as follows: ； ； The optimization model is expressed as follows: ； in, Represents the overall data space. This represents the spatial downsampling relationship between the time-series multispectral image after spectral radiometric normalization and the external reference image. This represents the denoised, time-series multispectral image with normalized spectral radiation. Indicates the first Each pixel in The temporal spectrum self-similarity weight matrix is ​​as follows. , and These represent combinations of different temporal, spatial, and spectral resolutions of multi-source reference temporal multispectral image data, respectively. The subscripts indicate... , They represent the first , No. One pixel, , , , , These represent the data points of the time-spectrum curves for different reference data. The spatiotemporal weights of the kernel function are represented. This represents the temporal spectrum self-similarity weight matrix. and These represent the relationship between the image to be solved and the input multi-source reference time-series image data, respectively. This indicates that a time-spectral autoregressive constraint is being considered. This represents the similarity constraint representation of time-spectrum curves. This represents the time-spectral autoregressive weight coefficient.

2. The method for unifying the temporal spectrum and radiation of sequence data under reference data constraints according to claim 1, characterized in that, The decomposition and reconstruction module, based on singular spectral analysis, embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs to obtain the final temporal spectral feature sequence, including the following steps: Obtain the temporal spectrum sequence of the multi-source reference sequence data pair, wherein the temporal spectrum sequence is formed based on the pixel temporal spectrum vector, and the pixel temporal spectrum vector is formed by the change of each pixel over time; Based on the pixel temporal spectrum vector and the preset window length, the temporal spectrum sequence is arranged with lag to obtain the trajectory matrix; The trajectory matrix is ​​decomposed using the singular value decomposition algorithm, yielding multiple submatrices and their corresponding singular values, left singular vector, and right singular vector, as follows: , ; Based on the numerical value of the singular values, all submatrices are grouped. The submatrix corresponding to the singular value with the largest value is reconstructed. Then, the reconstructed submatrices are transformed into the final temporal spectral eigenvector through a diagonal averaging process, as follows: ； in, Submatrices representing the trajectory matrix, expressed by singular values Left singular vector and right singular vector Calculations show that Represents the final temporal spectral eigenvector. Represents pixel values ​​in different bands. Represents the data in the submatrix. , , Represents the time spectrum vector. Indicates the preset window length. Represents the trajectory matrix.

3. The method for unifying the temporal spectrum and radiation of sequence data under reference data constraints according to claim 1, characterized in that, The feature encoding includes the following steps: The final temporal feature sequence is used as the input image. Based on each pixel position in the input image, a first feature map of the corresponding size is obtained after 1×1 convolution. The first feature maps from different channels are fused while maintaining the size of the first feature maps to obtain a fused feature map. Spatial coding is added to the fused feature map to recover the original sequence arrangement, resulting in a coded feature map with coding information. In this coded feature map, the feature vector of each pixel is represented as... .

4. The method for unifying the temporal spectrum and radiation of sequence data under reference data constraints according to claim 3, characterized in that, The self-attention processing, which yields the temporal-spectral self-similarity weight matrix, includes the following steps: For each pixel vector in the encoded feature map, a projection matrix is ​​used... , and Convert to query embedding, key embedding, and value embedding; calculate query embedding based on dot product. With key embedding The similarity between the values ​​is scaled according to a preset ratio to obtain the scaled result, which is then transformed by the softmax function to obtain the normalized attention score. ; Based on the spatial location information encoded by location, the attention scores of each pixel feature vector are recombined to make them consistent with the spatial dimension of the input image, thereby constructing a temporal self-similarity weight matrix. in, , , , , , and These represent the projection matrices, Indicates query embedding, Indicates key embedding, Indicates value embedding, This represents the attention score. Indicates the preset ratio. The input sequence represents pixel-by-pixel features.

5. The method for unifying the temporal spectrum and radiation of sequence data under reference data constraints according to claim 3, characterized in that, The input image is represented as M×N×(C×T), and the first feature map is represented as M×N×k; Then the first A convolutional kernel, represented as: ; A 1×1 convolution is represented as: ; in, Indicates the location of the first feature map The The value of each channel, Indicates the position of the input image. The The value of each channel, Indicates the first The first convolutional kernel Each weight, Indicates the first The bias term of each convolutional kernel, where M represents height, N represents width, C represents the number of channels, and T represents the time step. This represents the position of each pixel in the first feature map.

6. The method for unifying the temporal spectrum and radiation of sequence data under reference data constraints according to claim 1, characterized in that, The multi-source radiation normalized image database includes the following steps: The atmospheric radiative transfer process was simulated based on the atmospheric radiative transfer model, and remote sensing observation data under different atmospheric conditions were obtained. The data from different sensors are converted based on the spectral response function to obtain a converted remote sensing image dataset. Specifically, the spectral response function is obtained, and the response value of one sensor is converted to the band of another sensor based on the spectral response function. By comparing the spectral response functions of the two sensors, the spectral matching conversion factor between the two sensors is calculated. The conversion factor is then applied to the original data to obtain the converted remote sensing image dataset. The transformed remote sensing image dataset was resampled using area weighting to generate a multi-causal radiation normalized database. Among them, image resampling by area weighting specifically involves calculating the actual ground area of ​​each pixel in the transformed remote sensing image; The weight of each pixel is obtained by dividing the actual ground area of ​​each pixel by the sum of the areas of all pixels. Then, a weighted average is obtained through the weight of each pixel to achieve image resampling.

7. A system for processing temporal-spectral-radiative uniformity of sequence data under reference data constraints, characterized in that, It includes a database construction module, a model construction module, a model training module, and a model inference module; The database construction module is used to acquire remote sensing image sequences and generate a multi-source radiometric normalized image database. Two sets of remote sensing image sequences are arbitrarily selected to form a multi-source reference sequence data pair, where one set of remote sensing image sequences is used as the first reference data and the other set of remote sensing image sequences is used as the second reference data. The model building module is used to construct a reconstructed self-similar normalized pre-trained model. The reconstructed self-similar normalized pre-trained model includes a decomposition and reconstruction module, a self-similar weight matrix module, and a variational normalization module. The decomposition and reconstruction module embeds, decomposes, groups, and reconstructs multi-source reference sequence data pairs based on singular spectrum analysis to obtain the final time-spectral feature sequence. The self-similar weight matrix module performs feature encoding and self-attention processing on the final time-spectral feature sequence to obtain the time-spectral self-similar weight matrix. The variational normalization module forms an optimization model based on the time-spectral correlation and inter-spectral constraint relationship of multi-source data. Based on the optimization model, the time-spectral self-similar weight matrix is ​​optimized to form time-spectral radiation consistent sequence data. The model training module is used to train the reconstructed self-similar normalized pre-trained model through a multi-causal radiation normalization database to obtain the reconstructed self-similar normalized model. The model inference module processes the multi-source reference sequence data pairs to be normalized based on the reconstructed self-similar normalization model to obtain the corresponding time-spectral radiometric consistent sequence data. The variational normalization module forms an optimization model based on the temporal-spectral correlation and inter-spectral constraints of multi-source data, including the following steps: Based on multi-source reference sequence data pairs, the temporal spectral correlation and inter-spectral constraint relationships of multi-source data are obtained; An optimization model is constructed by using the temporal spectrum self-similarity weight matrix, the temporal spectrum correlation relationship of multi-source data, and the inter-spectral constraint relationship, so that the spatiotemporal weight of the kernel function of the overall data space in the optimization result reaches the minimum value. The temporal-spectral correlation and inter-spectral constraints of the multi-source data are expressed as follows: ； ； The optimization model is expressed as follows: ； in, Represents the overall data space. This represents the spatial downsampling relationship between the time-series multispectral image after spectral radiometric normalization and the external reference image. This represents the denoised, time-series multispectral image with normalized spectral radiation. Indicates the first Each pixel in The temporal spectrum self-similarity weight matrix is ​​as follows. , and These represent combinations of different temporal, spatial, and spectral resolutions of multi-source reference temporal multispectral image data, respectively. The subscripts indicate... , They represent the first , No. One pixel, , , , , These represent the data points of the time-spectrum curves for different reference data. The spatiotemporal weights of the kernel function are represented. This represents the temporal spectrum self-similarity weight matrix. and These represent the relationship between the image to be solved and the input multi-source reference time-series image data, respectively. This indicates that a time-spectral autoregressive constraint is being considered. This represents the similarity constraint representation of time-spectrum curves. This represents the time-spectral autoregressive weight coefficient.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

9. A processing apparatus for temporal-spectral-radiative homogenization of sequence data under reference data constraints, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

Images