A faint light data inversion method and device based on multi-source data and deep learning

Abstract

The application discloses a kind of micro-light data inversion method and device based on multi-source data and deep learning, belong to remote sensing image processing technical field.The method includes: obtaining the same area SDGSAT-1 micro-light data, Sentinel-2 multispectral data and VIIRS night light data, and carries out geographic registration and resolution matching;The data after registration is normalized, and the original feature and derived feature are spliced to construct comprehensive feature;K-means clustering is used to obtain double threshold, and light source positive sample and non-light source negative sample are divided;Res2-Unet model is constructed, channel attention mechanism is introduced, classification head and regression head double output structure are set, and light source classification result and micro-light brightness value are output respectively;Multi-loss fusion strategy is used to construct total loss function for model training.The application realizes the high-precision, high-frequency reconstruction of night light data.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to a method and apparatus for inverting low-light data based on multi-source data and deep learning. Background Technology

[0002] Nighttime light remote sensing data is an important data source for monitoring the intensity of human activities, assessing energy consumption, analyzing urbanization processes, and studying the distribution of light pollution. High-resolution nighttime low-light imagery can clearly present the fine-scale light distribution characteristics of urban internal structures, transportation networks, and rural settlements, and has broad application prospects in social sciences, environmental monitoring, and urban planning.

[0003] However, due to limitations in satellite platform and sensor design, it is difficult to directly acquire nighttime low-light data that simultaneously possesses high spatial and temporal resolution. Existing mainstream nighttime light remote sensing data sources have significant performance bottlenecks: the SDGSAT-1 low-light imager can acquire nighttime light data with a resolution of 10 meters, providing an unprecedented high spatial resolution data source for applications such as refined urban management, light pollution monitoring, and energy consumption assessment. However, due to limitations in satellite platform and orbit design, the revisit cycle of SDGSAT-1 is approximately 11 days, which is insufficient to meet the high-frequency operational monitoring needs. On the other hand, while nighttime light data such as VIIRS have the advantages of global coverage and daily updates, their original resolution is only 500 meters, making it difficult to depict the fine-scale nighttime light distribution characteristics of urban internal structures, road networks, and rural settlements, resulting in serious pixel mixing problems in applications.

[0004] To address the aforementioned issues, existing research has attempted to improve the quality of nighttime light data through technological means. Some studies utilize low-resolution nighttime light data to spatially downscale high-resolution nighttime light data; however, the spectral fidelity of the downscaling results is limited due to the limited intrinsic information content of low-resolution data. Other studies attempt to use daytime multispectral remote sensing data to perform statistical regression modeling of nighttime light intensity; however, a complex nonlinear relationship exists between nighttime light distribution and daytime land cover type, making it difficult for simple statistical models to accurately capture this mapping relationship. Furthermore, the fundamental differences between daytime and nighttime illumination conditions limit the generalization ability of these models.

[0005] It is worth noting that research on intelligent inversion and reconstruction of nighttime low-light data is still in its early stages. In particular, there is a lack of systematic and effective technical solutions for achieving high-precision, high-frequency reconstruction of high-resolution low-light data by utilizing complementary information between multi-source heterogeneous remote sensing data. How to fully explore the synergistic information between high-timeliness, low-resolution nighttime light data and high-resolution multispectral data, and construct an intelligent model that can accurately learn the deep mapping relationship between the spectral characteristics of ground objects and the distribution of nighttime lights, is a technical challenge that urgently needs to be solved in the field of remote sensing image processing. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method and apparatus for inverting low-light data based on multi-source data and deep learning. By utilizing low-resolution, high-time-efficiency night light data and high-resolution multispectral data, high-resolution low-light data is generated through inversion using a deep learning model. This improves the interpretability of nighttime light features in multi-source remote sensing data, effectively mines the complementary features between high-time-efficiency, low-resolution night light data and high-resolution multispectral data, and achieves high-precision, high-frequency reconstruction of SDGSAT-1 low-light data.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for inverting low-light data based on multi-source data and deep learning, the method comprising:

[0009] Step 1: Obtain high-resolution low-light tag data, high-resolution multispectral data, and low-resolution night light data from the same region. Perform resolution matching on the low-resolution night light data to make it spatially correspond to the high-resolution multispectral data, and perform normalization processing on each band of the three types of data respectively.

[0010] Step 2: Select the red, green, blue and near-infrared bands of the normalized high-resolution multispectral data, and the panchromatic band of the normalized low-resolution night light data as the original features. At the same time, construct the normalized difference index and fusion variables as the derived features, and splice all the original features and derived features to form a comprehensive feature.

[0011] Step 3: Based on the normalized high-resolution low-light tag data and the brightness values ​​of the panchromatic band of the low-resolution night light data, define the positive samples of the light source and the negative samples of the non-light source.

[0012] Step 4: Construct a Res2-Unet model with the Res2Net module as the core multi-scale feature extraction unit. In the model input stage, a channel attention mechanism is introduced, and the comprehensive features are used as the model input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer and decoder. The fused features output by the decoder are input into the classification head and the regression head respectively. The classification head outputs the light source classification result, and the regression head outputs the normalized low light brightness value. According to the light source classification result of the classification head, the brightness value of non-light source pixels is set to zero, and then inverse normalization is performed to obtain the inverted high-resolution low light data.

[0013] The model is trained using a weighted fusion strategy of classification loss and regression loss, whereby the classification loss is calculated based on the division results of positive samples from the light source and negative samples from the non-light source.

[0014] Furthermore, in step 2: the normalized difference index includes the normalized vegetation index and the normalized water index. The normalized vegetation index is calculated from the near-infrared band and red band of the normalized high-resolution multispectral data, and the normalized water index is calculated from the near-infrared band and green band of the normalized high-resolution multispectral data. The fusion variable is formed by fusing the near-infrared band of the normalized high-resolution multispectral data with the panchromatic band of the low-resolution night light data.

[0015] Furthermore, in step 3: the panchromatic brightness values ​​of the normalized high-resolution low-light tag data and the low-resolution night light data are respectively subjected to K-means clustering to obtain a first threshold and a second threshold, and a dual-threshold joint judgment rule is constructed. Only when the brightness value of the same pixel in the high-resolution low-light tag data is greater than the first threshold and the brightness value in the low-resolution night light data is greater than the second threshold, the pixel is judged as a positive sample of the light source; otherwise, it is judged as a negative sample of the non-light source.

[0016] Furthermore, after defining the positive light source samples and the negative non-light source samples, the method further includes: expanding the positive light source samples by using geometric transformation and adding Gaussian noise. The geometric transformation includes horizontal mirroring, vertical mirroring, rotation, random translation, random scaling, and Gaussian shift. After the sample expansion is completed, the training set and the test set are divided by a region-based hierarchical random sampling strategy, and random Gaussian noise, color jitter, and random erasure are applied to the training set samples to improve the model's generalization ability.

[0017] Furthermore, in step 4: global average pooling is performed on the input comprehensive features to compress each channel into a one-dimensional numerical value to obtain a channel description vector; the channel description vector is passed through two convolutional layers to learn the correlation between channels, and then the attention weights of each channel are generated by the Sigmoid activation function; the attention weights are multiplied with the original input comprehensive features channel by channel to achieve weighted enhancement of the feature channels.

[0018] Furthermore, in step 4: the encoder comprises two encoding units, each encoding unit comprising two convolutional layers, a Res2Net module, and a max-pooling downsampling layer cascaded in sequence; the bottleneck layer comprises two convolutional layers, a Res2Net module, and a random deactivation layer cascaded in sequence; the decoder comprises two decoding units cascaded in sequence, each decoding unit comprising a transposed convolutional upsampling layer, a feature concatenation layer, two convolutional layers, and a Res2Net module.

[0019] Furthermore, the specific structure of the Res2Net module is as follows: the input feature map is reduced to one-quarter of the channel number by the first convolutional layer and then divided into four branches. The first branch is directly passed, and the outputs of the subsequent branches are added to the outputs of the previous branches in turn. After each branch performs a convolution operation, the outputs of the four branches are concatenated in the channel dimension. Then, the original number of channels is restored by the second convolutional layer. Finally, the output is added to the input through the residual connection, and a random deactivation layer and Gaussian noise are added before output.

[0020] On the other hand, the present invention provides a micro-light data inversion device based on multi-source data and deep learning, comprising:

[0021] The acquisition module is used to acquire high-resolution low-light tag data, high-resolution multispectral data and low-resolution night light data in the same area, perform resolution matching on the low-resolution night light data to make it spatially correspond to the high-resolution multispectral data, and perform normalization processing on each band of the three types of data respectively.

[0022] The stitching module is used to select the red band, green band, blue band and near-infrared band of the normalized high-resolution multispectral data, as well as the panchromatic band of the normalized low-resolution night light data as the original features, and at the same time construct the normalized difference index and fusion variable as the derived features, and stitch all the original features and derived features together to form a comprehensive feature.

[0023] The delineation module is used to delineate positive light source samples and negative non-light source samples based on the brightness values ​​of the normalized high-resolution micro-light tag data and the panchromatic band of the low-resolution night light data.

[0024] The inversion module is used to construct a Res2-Unet model with the Res2Net module as the core multi-scale feature extraction unit. A channel attention mechanism is introduced in the model input stage, and the comprehensive features are used as the model input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer and decoder. The fused features output by the decoder are input into the classification head and the regression head respectively. The classification head outputs the light source classification result, and the regression head outputs the normalized low light brightness value. According to the light source classification result of the classification head, the brightness value of non-light source pixels is set to zero, and then inverse normalization is performed to obtain the inverted high-resolution low light data.

[0025] The model is trained using a weighted fusion strategy of classification loss and regression loss, whereby the classification loss is calculated based on the division results of positive samples from the light source and negative samples from the non-light source.

[0026] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for inverting low-light data based on multi-source data and deep learning.

[0027] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned method for inverting low-light data based on multi-source data and deep learning.

[0028] The beneficial effects of this invention are as follows:

[0029] This invention proposes for the first time an intelligent inversion method for high-resolution low-light data. It takes low-resolution, high-time-efficiency night light data and high-resolution multispectral data as inputs to generate high-resolution low-light data. This effectively breaks through the technical bottleneck of long revisit cycles for a single high-resolution night light data source and realizes high-frequency, large-scale operational reconstruction of high-resolution night light data.

[0030] This invention constructs a multi-source data collaborative modeling mechanism. By comprehensively utilizing the original bands and derived features, the model can fully learn the deep mapping relationship between the spectral features of ground objects and the distribution of nighttime lights, which significantly improves the accuracy and robustness of the inversion results.

[0031] This invention employs a K-means clustering dual-threshold joint judgment strategy to automatically classify light source and non-light source samples, effectively solving the problem of extreme imbalance between positive and negative samples in nighttime light data. Combined with diverse data augmentation techniques, it significantly improves the model's ability to identify sparse light source regions.

[0032] The Res2-Unet model designed in this invention integrates a channel attention mechanism and an encoder-decoder structure. The channel attention module adaptively enhances the feature channels that are correlated with low light intensity, and the multi-scale feature extraction unit accurately captures the light distribution features at different scales, thus achieving dual perception of small light source points and light source clusters.

[0033] This invention employs a dual-output structure of a classification head and a regression head to simultaneously complete the tasks of light source region identification and low-light brightness value inversion, avoiding interference from non-light source regions on the inversion results; the multi-loss fusion strategy takes into account both classification accuracy and regression accuracy, achieving synergistic optimization of the two types of tasks. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method for inverting low-light data based on multi-source data and deep learning according to the present invention.

[0035] Figure 2This is a structural diagram of the Res2-Unet model of the present invention;

[0036] Figure 3 This is a structural diagram of the Res2Net module of the present invention. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, this invention provides a method for inverting low-light data based on multi-source data and deep learning, comprising the following steps:

[0039] Step 1: Obtain high-resolution low-light tag data, high-resolution multispectral data, and low-resolution night light data from the same region. Perform resolution matching on the low-resolution night light data to make it spatially correspond to the high-resolution multispectral data, and perform normalization processing on each band of the three types of data respectively.

[0040] Step 2: Select the red, green, blue and near-infrared bands of the normalized high-resolution multispectral data, and the panchromatic band of the normalized low-resolution night light data as the original features. At the same time, construct the normalized difference index and fusion variables as the derived features, and splice all the original features and derived features to form a comprehensive feature.

[0041] Step 3: Based on the normalized high-resolution low-light tag data and the brightness values ​​of the panchromatic band of the low-resolution night light data, define the positive samples of the light source and the negative samples of the non-light source.

[0042] Step 4: Construct a Res2-Unet model with the Res2Net module as the core multi-scale feature extraction unit. In the model input stage, a channel attention mechanism is introduced, and the comprehensive features are used as the model input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer and decoder. The fused features output by the decoder are input into the classification head and the regression head respectively. The classification head outputs the light source classification result, and the regression head outputs the normalized low light brightness value. According to the light source classification result of the classification head, the brightness value of non-light source pixels is set to zero, and then inverse normalization is performed to obtain the inverted high-resolution low light data.

[0043] The model is trained using a weighted fusion strategy of classification loss and regression loss, whereby the classification loss is calculated based on the division results of positive samples from the light source and negative samples from the non-light source.

[0044] Example:

[0045] This invention provides a method for inverting low-light data based on multi-source data and deep learning. This method improves the interpretability of nighttime light features in multi-source remote sensing data by constructing a multi-source data collaborative modeling mechanism; it effectively mines complementary features between high-timeliness, low-resolution nighttime light data and high-resolution multispectral data, achieving high-precision, high-frequency reconstruction of SDGSAT-1 low-light data. The specific implementation of this method includes the following steps:

[0046] Step 1: Acquire SDGSAT-1 micro-light tag data from multiple regions, Sentinel-2 multispectral data and VIIRS night light data from the same date, achieve one-to-one spatial correspondence of each data through georegistration and resolution matching, and perform normalization processing on each band of the three types of data respectively.

[0047] Step 2: Select the red, green, blue and near-infrared bands of the normalized Sentinel-2 multispectral data, and the panchromatic band of the normalized VIIRS night light data as the original features, construct the normalized difference index and fusion variables as the derived features, and splice the original features and derived features to form a comprehensive feature.

[0048] Step 3: Based on the normalized SDGSAT-1 low-light tag data and the brightness values ​​of the panchromatic band of the VIIRS night light data, determine the criteria for judging positive samples from light sources and negative samples from non-light sources; expand the positive samples using various data augmentation methods; complete the division of the training set and test set using a hierarchical random sampling strategy; and apply data augmentation to the training set samples to improve generalization ability.

[0049] Step 4: Construct a Res2-Unet model, using the comprehensive features as input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer, and decoder. A channel attention mechanism is introduced at the input stage to adaptively adjust the weights of each feature channel. The model employs a dual-output structure with a classification head and a regression head, used to output the inverted light source classification results and the brightness values ​​of the low-light data, respectively. A multi-loss fusion strategy is adopted, designing corresponding loss functions for the classification and regression tasks, and constructing a total loss function through weighted fusion to guide model training.

[0050] Specifically, step 1 includes:

[0051] Step 11: Resample the VIIRS night light data to 10 meters using the nearest neighbor method to make it spatially correspond to the high-resolution multispectral data;

[0052] Step 12: Perform maximum and minimum value normalization on each band of the Sentinel-2 multispectral data, VIIRS night light data, and SDGSAT-1 micro-light tag data, mapping the values ​​to the [0, 1] interval:

[0053] ,

[0054] in, Representing the band, For the first The normalized values ​​for the band For the first Original values ​​of the band, For the first Band minimum, For the first Maximum value of the band.

[0055] Specifically, step 2 includes:

[0056] Step 21: The original features include the red, green, blue and near-infrared bands of the normalized Sentinel-2 multispectral data, and the panchromatic band of the normalized VIIRS night light data.

[0057] Step 22: The normalized difference index in the derived features includes the normalized vegetation index and the normalized water index calculated from Sentinel-2 multispectral data.

[0058] ,

[0059] ,

[0060] in, and These are the Normalized Difference Vegetation Index (NDVI) and the Normalized Difference Water Index (NDVI), respectively. These are the green, red, and near-infrared bands of Sentinel-2 multispectral data, respectively.

[0061] Step 23: The fusion variable in the derived features is formed by fusing the near-infrared band of the normalized Sentinel-2 multispectral data and the panchromatic band of the VIIRS night light data.

[0062] ,

[0063] in, For fusion variables, For the near-infrared band of Sentinel-2 multispectral data, This refers to the panchromatic band of VIIRS nighttime light data.

[0064] Specifically, step 3 includes:

[0065] Step 31: Apply K-means clustering to the panchromatic brightness values ​​of the SDGSAT-1 low-light tag data and VIIRS night light data after coverage area normalization to obtain the boundary thresholds between light source pixels and non-light source pixels for each of the two datasets. and A dual-threshold joint judgment rule is constructed: only when the brightness value of the same pixel in the panchromatic band data of the SDGSAT-1 low-light tag is greater than... Furthermore, the brightness value of the VIIRS night light data panchromatic band is greater than If the pixel is a light source sample (positive sample), it is marked as 1; otherwise, it is a non-light source sample (negative sample) and marked as 0.

[0066] Step 32: Design 12 geometric transformation operations, including horizontal mirroring, vertical mirroring, 90° / 180° / 270° rotation, random translation (±10 pixels), main / anti-diagonal mirroring, random rotation (±45°), random scaling (0.8~1.2 times), and Gaussian shift (standard deviation=3). The number of transformation rounds is dynamically determined according to the proportion of positive samples in each region. During the model training phase, Gaussian noise (variance=0.01) is randomly added to the expanded positive samples to improve the model's anti-interference ability.

[0067] Step 33: After completing the sample expansion, a stratified random sampling strategy by region is adopted for the positive and negative sample sets (sampling ratio = 0.005). Independent sampling is performed in each region to ensure that the ratio of positive and negative samples is consistent with the original distribution. Then, the samples from each region are merged and randomly shuffled. The training set and the test set are divided into a 7:3 ratio, and the ratio of positive and negative samples in the two parts is kept consistent.

[0068] Step 34: To improve the model's generalization ability, additional data augmentation is performed on the training set samples, including random Gaussian noise, color jitter (brightness and contrast changes by 0.8 to 1.2 times), and random erasure, to simulate local missing data and spectral diversity.

[0069] Specifically, step 4 includes:

[0070] like Figure 2As shown, the Res2-Unet model uses the Res2Net module as its core multi-scale feature extraction unit, and its overall structure consists of three parts: an encoder, a bottleneck layer, and a decoder. The encoder progressively reduces the feature map size and increases the number of channels through two cascaded encoding units. Each encoding unit contains a convolutional layer and a Res2Net module. The bottleneck layer further extracts global deep correlation features. The decoder progressively restores the feature map size and fuses the shallow detail features from the encoder through two cascaded decoding units. Each decoding unit contains upsampling, feature concatenation, a convolutional layer, and a Res2Net module. Specifically:

[0071] For the input composite feature map, a channel attention layer weighting mechanism is executed. Specifically, for each channel, the average value of all its pixel values ​​is calculated to obtain the global pooling result for that channel. The pooling results of all channels constitute the channel description vector. The channel description vector is then passed sequentially through a fully connected layer consisting of two 1×1 convolutional layers to learn the relationships between the channels. The first convolutional layer reduces the number of channels to half of the original number (dimensionality reduction ratio set to 2), and the second convolutional layer restores the number of channels to the original number, resulting in the transformed feature vector. The Sigmoid activation function is used to map the transformed feature vector to the range of 0 to 1, generating attention weights for each channel. The generated attention weights are multiplied channel-by-channel with the original input composite feature map to achieve weighted enhancement of the feature channels. The enhanced feature map has the same size as the input and is used as the input to the encoder.

[0072] The encoder consists of cascaded first and second coding units, progressively reducing the feature map size and increasing the number of channels. The structure of each coding unit is as follows: two 3×3 convolutional layers (including batch normalization and ReLU activation), a Res2Net module, and max pooling downsampling with a stride of 2.

[0073] The bottleneck layer receives the high-level features output by the encoder. Its structure consists of two 3×3 convolutional layers (including batch normalization and ReLU activation), a Res2Net module, and a random deactivation layer, which gradually outputs global deep correlation features.

[0074] The decoder consists of a cascaded first decoding unit and a second decoding unit, which progressively restore the feature map size and fuse the shallow detail features of the encoder. The structure of each decoding unit is as follows: transposed convolution upsampling (stride 2), channel concatenation with the features of the corresponding layer of the encoder, two 3×3 convolutions (including batch normalization and LeakyReLU activation), and a Res2Net module.

[0075] The encoder, bottleneck layer, and decoder all contain Res2Net modules with the same structure, which is also the core structure of the network in this invention. Figure 3As shown, the specific implementation of the Res2Net module is as follows: The feature map input to the Res2Net module is reduced to 1 / 4 of the channel count by a 1×1 convolution, and then divided into 4 branches (S1-S4). A cross-branch progressive fusion strategy is adopted (the first branch is directly passed, and subsequent branches are added to the output of the previous branch in turn). After each branch performs a 3×3 convolution, the outputs of the four branches are concatenated along the channel dimension, and then a 1×1 convolution is performed to restore the original number of channels. Finally, it is added to the input through residual connections, and a random deactivation layer (deactivation rate 0.3) and Gaussian noise (standard deviation 0.01) are added to output the feature map.

[0076] The final fused feature output by the decoder is shared by the classification head and the regression head, which respectively perform the tasks of light source region identification and low-light brightness value inversion.

[0077] The classification head is used for binary classification between light source regions and non-light source regions. The input is the fused features output from the decoder. First, the features are refined using multiple 3×3 convolutions (including batch normalization, ReLU activation, and random deactivation layers), with a random deactivation rate set to 0.3. Then, a 1×1 convolution compresses the feature channels to 1, and a Sigmoid activation function maps the output to the 0-1 range, obtaining the confidence score for each pixel as a light source region. Pixels with a confidence score greater than 0.5 are classified as light source points, and the rest as non-light source points.

[0078] The regression head is used for quantitative inversion of low-light brightness values. The input is the fused features output from the decoder. The features are mined through multiple 3×3 convolutions (including batch normalization, LeakyReLU activation, and random deactivation layers). Then, the feature channels are compressed to 1 through 1×1 convolutions, and the output is restricted to the range of 0 to 1 using the Sigmoid activation function to obtain the normalized low-light brightness value for each pixel. Its value range is consistent with the normalized low-light data.

[0079] The outputs of the classification head and the regression head are fused. Based on the classification head's judgment result, only the brightness value of the light source point is retained, while the brightness values ​​of non-light source points are set to zero. Specifically, when the confidence level of the classification head output is greater than 0.5, the brightness value of that pixel is retained; otherwise, the brightness value of that pixel is set to 0. The normalized low-light brightness value is obtained by fusing the outputs from the classification head and the back-end output. Finally, the fused brightness value is denormalized to obtain the low-light brightness value:

[0080] ,

[0081] in, The final retrieved low-light brightness values ​​have the same value range as the original SDGSAT-1 low-light data. and These represent the maximum and minimum values ​​of the original low-light data, respectively.

[0082] The classification loss employs a weighted fusion of FocalLoss and DiceLoss. FocalLoss introduces the weight of light source samples generated by the ratio of positive to negative pixels, thereby reducing the weight of easily classified negative samples and strengthening the learning of difficult-to-classify positive samples. DiceLoss focuses on the intersection-union ratio of positive and negative samples, further alleviating the sample imbalance problem. The regression loss employs a weighted fusion of smoothed L1 loss and mean squared error loss. L1 loss is the mean absolute error between the predicted and true values, which has strong robustness to outliers. Mean squared error loss is the mean squared error between the predicted and true values, making the model's prediction results smoother. Weights are constructed based on the magnitudes of the classification and regression loss values. The total loss function is constructed by weighted summation of the classification and regression losses, achieving collaborative optimization of the two tasks.

[0083] In one specific embodiment of the present invention, taking a multi-source remote sensing image containing multiple regions as an example (including SDGSAT-1 low-light data, Sentinel-2 multispectral data and VIIRS night light data), high-resolution low-light data inversion is carried out. To verify the effectiveness and superiority of this invention, the method of this invention is compared with the following four low-light data inversion methods, including: a single-modal inversion method: using only Sentinel-2 multispectral data and its derived features as input, without using VIIRS night light data, and using the same Res2-Unet model structure as this invention; a U-Net inversion method: using the same multi-source data as this invention as input, employing the U-Net network, replacing the Res2-Unet model structure and channel attention mechanism of this invention; a no-derived-feature inversion method: using the same network structure as this invention, but only using Sentinel-2 multispectral and VIIRS night light bands at the input end, removing derived features containing normalized vegetation index, normalized water index, and fusion variables; and a single regression inversion method: using the same multi-source data and Res2-Unet model structure as this invention, but removing the classification head, and only performing regression fitting of low-light data brightness values.

[0084] This embodiment uses the F1 score (reflecting the accuracy of light source classification; this indicator is not available in single regression inversion methods) and the coefficient of determination R. 2 The inversion accuracy of each method is evaluated by assessing the goodness of fit of the regression of low-light brightness values. As shown in Table 1, the method of this invention achieves the best results in all metrics.

[0085] Table 1

[0086]

[0087] In summary, by utilizing the technical solutions described above in this invention, and using a comprehensive feature constructed from Sentinel-2 multispectral data, NPP-VIIRS nighttime light data, and derived features as input, the technical bottleneck of long revisit cycles for SDGSAT-1 low-light data is effectively overcome. The light source samples constructed using a dual-threshold joint judgment strategy can accurately distinguish between light source and non-light source regions, effectively alleviating the problem of extreme imbalance in nighttime light samples. The dual-output structure of the classification head and regression head enables collaborative learning of light source region identification and low-light brightness value inversion, avoiding interference from non-light source regions on the inversion results. The introduced Res2Net module and channel attention mechanism can accurately capture multi-scale spatial features and channel correlation features of low-light regions, achieving high-precision, high-frequency reconstruction of SDGSAT-1 low-light data.

[0088] On the other hand, the present invention provides a micro-light data inversion device based on multi-source data and deep learning, the various modules of which can implement the various steps of the aforementioned method, specifically including:

[0089] The acquisition module is used to acquire high-resolution low-light tag data, high-resolution multispectral data and low-resolution night light data in the same area, perform resolution matching on the low-resolution night light data to make it spatially correspond to the high-resolution multispectral data, and perform normalization processing on each band of the three types of data respectively.

[0090] The stitching module is used to select the red band, green band, blue band and near-infrared band of the normalized high-resolution multispectral data, as well as the panchromatic band of the normalized low-resolution night light data as the original features, and at the same time construct the normalized difference index and fusion variable as the derived features, and stitch all the original features and derived features together to form a comprehensive feature.

[0091] The delineation module is used to delineate positive light source samples and negative non-light source samples based on the brightness values ​​of the normalized high-resolution micro-light tag data and the panchromatic band of the low-resolution night light data.

[0092] The inversion module is used to construct a Res2-Unet model with the Res2Net module as the core multi-scale feature extraction unit. A channel attention mechanism is introduced in the model input stage, and the comprehensive features are used as the model input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer and decoder. The fused features output by the decoder are input into the classification head and the regression head respectively. The classification head outputs the light source classification result, and the regression head outputs the normalized low light brightness value. According to the light source classification result of the classification head, the brightness value of non-light source pixels is set to zero, and then inverse normalization is performed to obtain the inverted high-resolution low light data.

[0093] The model is trained using a weighted fusion strategy of classification loss and regression loss, whereby the classification loss is calculated based on the division results of positive samples from the light source and negative samples from the non-light source.

[0094] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for inverting low-light data based on multi-source data and deep learning.

[0095] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned method for inverting low-light data based on multi-source data and deep learning.

[0096] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for inverting low-light data based on multi-source data and deep learning, characterized in that, The method includes: Step 1: Obtain high-resolution low-light tag data, high-resolution multispectral data, and low-resolution night light data from the same region. Perform resolution matching on the low-resolution night light data to make it spatially correspond to the high-resolution multispectral data, and perform normalization processing on each band of the three types of data respectively. Step 2: Select the red, green, blue, and near-infrared bands of the normalized high-resolution multispectral data, and the panchromatic band of the normalized low-resolution night light data as the original features. Simultaneously, construct a normalized difference index and a fusion variable as derived features. Combine all original and derived features to form a comprehensive feature. The normalized difference index includes a normalized vegetation index and a normalized water index. The normalized vegetation index is calculated from the near-infrared and red bands of the normalized high-resolution multispectral data, and the normalized water index is calculated from the near-infrared and green bands of the normalized high-resolution multispectral data. The fusion variable is formed by fusing the near-infrared band of the normalized high-resolution multispectral data and the panchromatic band of the low-resolution night light data. Step 3: Based on the normalized high-resolution low-light tag data and the panchromatic brightness values ​​of the low-resolution night light data, define positive light source samples and negative non-light source samples; this includes: applying K-means clustering to the panchromatic brightness values ​​of the normalized high-resolution low-light tag data and the low-resolution night light data to obtain a first threshold and a second threshold, constructing a dual-threshold joint judgment rule, determining the pixel as a positive light source sample only when the brightness value of the same pixel in the high-resolution low-light tag data is greater than the first threshold and the brightness value in the low-resolution night light data is greater than the second threshold; otherwise, it is determined as a negative non-light source sample. Step 4: Construct a Res2-Unet model with the Res2Net module as the core multi-scale feature extraction unit. In the model input stage, a channel attention mechanism is introduced, and the comprehensive features are used as the model input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer and decoder. The fused features output by the decoder are input into the classification head and the regression head respectively. The classification head outputs the light source classification result, and the regression head outputs the normalized low light brightness value. According to the light source classification result of the classification head, the brightness value of non-light source pixels is set to zero, and then inverse normalization is performed to obtain the inverted high-resolution low light data. The model is trained using a weighted fusion strategy of classification loss and regression loss, whereby the classification loss is calculated based on the division results of positive samples from the light source and negative samples from the non-light source.

2. The method for inverting low-light data based on multi-source data and deep learning according to claim 1, characterized in that, After defining the positive light source samples and the negative non-light source samples, the method further includes: expanding the positive light source samples by using geometric transformation and adding Gaussian noise. The geometric transformation includes horizontal mirroring, vertical mirroring, rotation, random translation, random scaling, and Gaussian shift. After the sample expansion is completed, the training set and the test set are divided by a region-based hierarchical random sampling strategy. Random Gaussian noise, color jitter, and random erasure are additionally applied to the training set samples to improve the model's generalization ability.

3. The method for inverting low-light data based on multi-source data and deep learning according to claim 1, characterized in that, In step 4: global average pooling is performed on the comprehensive features of the input to compress each channel into a one-dimensional numerical value to obtain a channel description vector; the channel description vector is passed through two convolutional layers to learn the correlation between channels, and then the attention weights of each channel are generated by the Sigmoid activation function; the attention weights are multiplied with the comprehensive features of the original input channel by channel to achieve weighted enhancement of the feature channels.

4. The method for inverting low-light data based on multi-source data and deep learning according to claim 1, characterized in that, In step 4: the encoder includes two encoding units, each encoding unit including two convolutional layers, a Res2Net module and a max pooling downsampling layer cascaded in sequence; the bottleneck layer includes two convolutional layers, a Res2Net module and a random deactivation layer cascaded in sequence; the decoder includes two decoding units cascaded in sequence, each decoding unit including a transposed convolutional upsampling layer, a feature concatenation layer, two convolutional layers and a Res2Net module.

5. The method for inverting low-light data based on multi-source data and deep learning according to claim 4, characterized in that, The specific structure of the Res2Net module is as follows: the input feature map is reduced to one-quarter of the number of channels by the first convolutional layer and then divided into four branches. The first branch is passed directly, and the outputs of the subsequent branches are added to the outputs of the previous branches in turn. After each branch performs a convolution operation, the outputs of the four branches are concatenated in the channel dimension. Then, the original number of channels is restored by the second convolutional layer. Finally, the output is added to the input through the residual connection and a random deactivation layer and Gaussian noise are added before output.

6. A micro-light data inversion device based on multi-source data and deep learning, characterized in that, include: The acquisition module is used to acquire high-resolution low-light tag data, high-resolution multispectral data and low-resolution night light data in the same area, perform resolution matching on the low-resolution night light data to make it spatially correspond to the high-resolution multispectral data, and perform normalization processing on each band of the three types of data respectively. The stitching module is used to select the red band, green band, blue band and near-infrared band of the normalized high-resolution multispectral data, as well as the panchromatic band of the normalized low-resolution night light data as the original features, and at the same time construct the normalized difference index and fusion variable as the derived features, and stitch all the original features and derived features together to form a comprehensive feature. The normalized difference index includes a normalized vegetation index and a normalized water index. The normalized vegetation index is calculated from the near-infrared and red bands of the normalized high-resolution multispectral data, and the normalized water index is calculated from the near-infrared and green bands of the normalized high-resolution multispectral data. The fusion variable is formed by fusing the near-infrared band of the normalized high-resolution multispectral data with the panchromatic band of the low-resolution night light data. The delineation module is used to delineate positive light source samples and negative non-light source samples based on the brightness values ​​of the panchromatic bands of the normalized high-resolution low-light tag data and the low-resolution night light data. This includes: applying K-means clustering to the panchromatic band brightness values ​​of the normalized high-resolution low-light tag data and the low-resolution night light data to obtain a first threshold and a second threshold; constructing a dual-threshold joint determination rule; determining a pixel as a positive light source sample only when the brightness value of the same pixel in the high-resolution low-light tag data is greater than the first threshold and the brightness value in the low-resolution night light data is greater than the second threshold; otherwise, determining it as a negative non-light source sample. The inversion module is used to construct a Res2-Unet model with the Res2Net module as the core multi-scale feature extraction unit. A channel attention mechanism is introduced in the model input stage, and the comprehensive features are used as the model input. Multi-scale features are extracted and fused layer by layer through the encoder, bottleneck layer and decoder. The fused features output by the decoder are input into the classification head and the regression head respectively. The classification head outputs the light source classification result, and the regression head outputs the normalized low light brightness value. According to the light source classification result of the classification head, the brightness value of non-light source pixels is set to zero, and then inverse normalization is performed to obtain the inverted high-resolution low light data. The model is trained using a weighted fusion strategy of classification loss and regression loss, whereby the classification loss is calculated based on the division results of positive samples from the light source and negative samples from the non-light source.

7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the micro-light data inversion method based on multi-source data and deep learning as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, enable the processor to implement the low-light data inversion method based on multi-source data and deep learning as described in any one of claims 1-5.

Images