Spaceborne lidar feature layer unsupervised identification method and system

Abstract

The application discloses a kind of feature layer unsupervised identification method and system of spaceborne lidar, belong to atmospheric remote sensing and lidar detection technical field.The method includes: first, extract and preprocess spaceborne lidar data and auxiliary meteorological data, construct three-dimensional physical prior lookup table including region, height layer, time period dimension;Multi-view neighborhood sampling is extracted to extract multi-view spatial context features, and signal purification is realized by using physically guided convolutional autoencoder pre-training;After multi-view representation is fused, deep clustering is carried out, and finally, multi-stage physical constraint cluster merging and fine processing output feature layer recognition result.The application can automatically realize high-precision identification of spaceborne lidar cloud, aerosol and other feature layers without manual annotation.

Description

Technical Field

[0001] This invention relates to the field of atmospheric remote sensing and lidar detection technology, specifically to an unsupervised identification method and system for feature layers of spaceborne lidar. Background Technology

[0002] Spaceborne lidar (such as CALIPSO and EarthCARE) can acquire three-dimensional distribution information of global atmospheric clouds and aerosols with high vertical resolution, making it a key detection tool for studying climate change, air quality, and cloud-aerosol-radiation interactions. However, accurately identifying feature layers (including cloud and aerosol layers) from lidar echo signals is a crucial prerequisite for subsequent optical parameter inversion and a long-standing technical challenge. Existing feature layer detection methods are mainly divided into two categories. One category consists of traditional detection algorithms represented by the slope method and the threshold method. The slope method locates the layer boundary by detecting the inflection point of the derivative of the attenuated backscattered signal, which is effective for high signal-to-noise ratio signals, but its applicability is limited under low signal-to-noise ratio conditions of spaceborne lidar. The threshold method (such as the SIBYL algorithm used by CALIPSO) identifies feature layers by comparing the signal with an adaptive threshold, and is currently the mainstream method, reliably detecting strong signal layers. However, this method is highly dependent on empirical thresholds, which can easily lead to missed detection of weak feature layers under weak signal or low signal-to-noise ratio conditions. Although existing studies have attempted to improve the detection capabilities by using methods such as two-dimensional threshold arrays, minimum cost function segmentation, and multi-view hypothesis testing, none of these methods have escaped the constraints of threshold presets or precise system parameters, and the ability to detect weak signals has not yet achieved a fundamental breakthrough.

[0003] Another category is deep learning methods introduced in recent years, currently dominated by supervised learning. These methods rely on manually labeled data or feature layer labels output by traditional algorithms for training. While this improves automation, it has significant limitations: firstly, acquiring high-quality, large-scale manually labeled data is costly; secondly, using labels from traditional algorithms only approximates their performance limits, failing to overcome inherent defects such as threshold dependence, and inheriting corresponding false negative and false positive errors. Therefore, the generalization ability of existing supervised methods is limited, making it difficult to adapt to varied observation scenarios. In recent years, the development of unsupervised learning methods has provided a new solution to these problems. These methods do not rely on labeled data or threshold settings, and can directly learn features and discover intrinsic structures from unlabeled data, exhibiting strong adaptability. However, the application of unsupervised learning in the field of LiDAR feature layer recognition is still very limited, and existing deep models generally lack physical principle guidance, resulting in insufficient interpretability. Therefore, there is an urgent need for a high-precision feature layer recognition method that requires no manual labeling, possesses physical interpretability, and can adapt to different signal-to-noise ratio conditions. Summary of the Invention

[0004] The purpose of this invention is to provide an unsupervised identification method and system for feature layers of spaceborne lidar, which solves the technical problems of traditional thresholding methods relying on empirical parameters and having weak detection capabilities for weak feature layers, as well as supervised learning relying on manual annotation, having poor generalization ability and insufficient interpretability.

[0005] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides an unsupervised identification method for feature layers of a spaceborne lidar, comprising the following steps: S1. Extract the attenuation backscattering coefficient of the spaceborne lidar and auxiliary meteorological data, and perform preprocessing operations such as removing surface signals, profile averaging and median filtering on the raw data; use the auxiliary meteorological data to simulate the atmospheric molecular attenuation backscattering signal under ideal cloudless and aerosol-free conditions, and statistically analyze its distribution characteristics. S2. Multiple two-dimensional neighborhood sampling with different spatial ranges are used to traverse the preprocessed attenuated backscattered image with a step size of 1. Each neighborhood sampling result is used as an independent view. For each pixel in the image, its neighborhood spatial data is extracted to construct a multi-view spatial context dataset. S3. Construct an independent convolutional autoencoder for each view. During the pre-training phase, based on the distribution characteristics of atmospheric molecular attenuation backscattered signals under ideal cloudless and aerosol-free conditions calculated in the previous stage, Gaussian noise conforming to these statistical characteristics is generated and added to the input signal of the corresponding view. The autoencoder is trained to reconstruct the original input signal before adding noise from the noise-added signal. By minimizing the reconstruction error, the model is guided to learn to recognize background signal components that conform to the statistical characteristics of atmospheric molecules. The core of this step is to use physical priors to guide the encoder to extract discriminative embedding features, so that the background molecular signal and the feature layer signal are well separated in the embedding space, thereby providing a high-quality initial feature representation for subsequent deep clustering. In other words, the pre-training of the autoencoder is not only a signal processing process, but also a feature learning process that serves the deep clustering task. S4. After completing the pre-training of the autoencoders for each view, the low-dimensional embedding features output by all view encoders are weighted and fused, with the weights of each view preset to be equal, to obtain a unified multi-view representation; deep clustering training is performed on this multi-view representation space, and the clustering loss is optimized by minimizing the KL divergence between the soft assignment distribution and the target distribution to obtain preliminary clustering results. S5. Based on the similarity between clusters and the probability distribution of signals within each cluster, the preliminary clustering results are merged under multiple physical constraints to obtain preliminary binary classification results of the feature layer and non-feature layer. Then, the probability of the non-feature layer is calculated based on the cumulative probability of the binomial distribution, and the non-feature layer samples are optimized at the pixel level to recover weak signals that may have been missed during the cluster merging process, and the final feature layer recognition result is output.

[0006] Furthermore, in step S1, the distribution characteristics of atmospheric molecular attenuation backscattering signals under ideal cloudless and aerosol-free conditions specifically include: statistical analysis of long-term simulated atmospheric molecular attenuation backscattering signals for at least one year for different geographical sub-regions, different altitude layers, and day and night periods, and calculation of their mean μ and standard deviation σ.

[0007] Further, step S2 specifically includes: Multiple two-dimensional neighborhood samplings with different spatial ranges are used to traverse the preprocessed attenuated backscattered image with a step size of 1. Each neighborhood sampling result is used as an independent view. At the same time, two variables, the total attenuated backscattering coefficient and the vertical channel attenuated backscattering coefficient, are extracted to form a multi-view spatial context dataset.

[0008] Furthermore, in step S3, the convolutional autoencoder constructed for each view adopts a differentiated architecture design. Views with different spatial ranges correspond to autoencoders with different structures or different depths to achieve adaptive feature extraction: For views with a large spatial range (such as 11×11 or 9×9), a deep convolutional autoencoder is used. The encoder contains three convolutional layers, which expand the receptive field by progressive downsampling to capture a wide range of spatial context information; batch normalization and two-dimensional dropout layers are added after each convolutional layer to prevent overfitting; the output is flattened and mapped to a low-dimensional embedding space through a fully connected layer.

[0009] For views with medium spatial range (such as 7×7, 5×5), a convolutional autoencoder is also used, but the network depth and number of parameters are adapted and adjusted according to the input size to control the model complexity while ensuring feature extraction capability.

[0010] For views with a smaller spatial range (such as 3×3), a fully connected autoencoder is used, which consists of stacked fully connected layers forming the encoder and decoder, avoiding information loss and overfitting risks caused by convolution operations under small input sizes.

[0011] Each view's autoencoder maps the input to a low-dimensional embedding layer of uniform dimension, facilitating subsequent multi-view feature fusion. Dropout and batch normalization techniques are employed during training to prevent overfitting. This differentiated design allows views of different spatial scales to extract features using the most suitable network structure, avoiding a one-size-fits-all approach to inputs at different scales using a single network structure.

[0012] Furthermore, the parameters of the encoder and decoder are updated by minimizing the reconstruction error: Where n is the total number of training samples. Let F be the center pixel of the two-dimensional neighborhood view of the i-th input model, and let ω and ω′ represent the learnable weight parameters of the encoder and decoder networks (including the weight matrices and bias terms of each convolutional layer), respectively. ω G represents an encoder network with weight parameter ω. ω′ F represents a decoder network with weight parameters ω′. ω (x i ) represents the low-dimensional embedding features of the encoder for the noisy input and output. This represents the center pixel reconstructed by the decoder. During training, ω and ω′ are initially initialized randomly, and then iteratively updated using backpropagation and gradient descent optimizers by minimizing the reconstruction loss function until the loss function converges. By minimizing this reconstruction error, the decoder is able to recover the center pixel of the original input from the embedded features.

[0013] Furthermore, in step S4, after the independent pre-training of each view encoder is completed, the model enters the multi-view joint training stage, and the overall objective function of the joint training is defined as: in, To reconstruct the loss, The clustering loss is γ, which is a coefficient controlling the degree of deformation of the embedding space, and its value is 0.1. The clustering loss adopts a deep embedding clustering paradigm, constructed based on KL divergence, and is defined as the KL divergence between the soft-assigned distribution Q and the target distribution P: in, It is a soft-assigned vector The j-th element represents the probability that the embedded feature i belongs to cluster j.

[0014] Furthermore, in step S5, the cluster merging of the multi-stage physical constraints specifically includes the following steps: S51. Based on the sample ratio within each cluster and the similarity measure between clusters, identify reference clusters representing non-feature layers and feature layers. The cluster with the highest sample frequency is selected as the non-feature layer reference cluster, while the cluster that exhibits strong separability in the feature space is designated as the feature layer reference cluster. S52. Construct empirical probability distributions of attenuation backscattering coefficients for reference clusters in the feature layer and non-feature layer, use the Kolmogorov-Smirnov test to quantify the difference between each unlabeled cluster and the two types of reference distributions, and assign categories to unlabeled clusters based on the test results. S53. Perform pixel-level optimization on non-feature layer samples to recover weak signals that may be missed during cluster merging. By counting the number of pixels with attenuated scattering ratio (ASR) exceeding a preset threshold in different spatial ranges, calculate the probability of each pixel belonging to the non-feature layer based on the cumulative probability of the binomial distribution, so as to recover the weak signal feature layer that may be missed during cluster merging.

[0015] Further, in step S52, if the p-value of the unlabeled cluster compared with the feature layer reference cluster distribution is greater than both the p-value of the unlabeled cluster compared with the non-feature layer reference cluster distribution and the significance threshold, then the unlabeled cluster is assigned to the feature layer category; otherwise, it is assigned to the non-feature layer category.

[0016] Furthermore, in step S53, the optimization algorithm for non-feature layer samples is specifically as follows: Calculate the attenuation scattering ratio of pixels in different spatial ranges, count the number of pixels whose attenuation scattering ratio exceeds a preset threshold, and calculate the probability that the pixel belongs to the non-feature layer based on the cumulative probability of the binomial distribution; if the pixel is identified as a feature layer in at least any small neighborhood and any large neighborhood, then it is finally marked as a feature layer.

[0017] The Physically Guided Deep Multi-View Clustering (PG-MVC) framework proposed in this invention consists of five functionally defined and logically progressive modules: Unlike traditional methods that use atmospheric molecular signals as detection thresholds, the physical prior construction module extracts the attenuation backscattering coefficient of the spaceborne lidar and auxiliary meteorological data and performs preprocessing. It uses the auxiliary meteorological data to simulate the atmospheric molecular attenuation backscattering signal under ideal cloudless and aerosol-free conditions, and uses its statistical characteristics as physical priors to guide the subsequent model training, rather than as a direct basis for judgment.

[0018] Unlike traditional one-dimensional profile or single-scale window methods, the multi-view spatial context extraction module constructs multiple two-dimensional neighborhood windows with different spatial ranges for each pixel as independent views, enabling the model to perceive the feature layer structure from complementary spatial perspectives. It is important to emphasize that the core logic of this module is multi-view collaborative feature learning. Specifically, each view is input in parallel to its corresponding encoder for independent feature extraction. Then, a learnable weighted fusion mechanism integrates the embedded features of each view into a unified representation. This representation simultaneously contains complementary information from different spatial ranges and directly participates in subsequent deep clustering and joint optimization.

[0019] Unlike conventional autoencoders that only focus on reconstruction error, the physics-guided discriminative feature learning module embeds physical priors as noise distributions into the autoencoder pre-training process. This physical prior guides the encoder to extract discriminative embedded features, ensuring good separation between background molecular signals and feature layer signals in the embedding space. This provides high-quality initial feature representations for subsequent deep clustering. In other words, the training of the autoencoder is not only a signal processing process but also a feature learning process serving the deep clustering task.

[0020] Unlike traditional methods that perform feature extraction and clustering independently, the multi-view fusion and deep clustering joint optimization module optimizes both reconstruction loss and clustering loss after fusing multi-view features, enabling the feature space to evolve in a direction more conducive to clustering.

[0021] Unlike methods that directly use statistical tests as detection methods, the physical constraint cluster merging and refinement module uses statistical tests (KS test, binomial distribution) as an interpreter for deep clustering results. It maps the soft assignment of clusters without physical meaning to binary classification of feature layers / non-feature layers with clear physical meaning, and corrects clustering errors and recalls weak signals in the process.

[0022] Based on the above technical solution, the embodiments of the present invention can produce at least the following technical effects: (1) This invention employs an unsupervised learning framework. By constructing a multi-view spatial context dataset and a deep clustering model, it directly learns the intrinsic structure of the feature layer from the original signal without requiring any manually labeled data for training. This invention constructs multiple two-dimensional neighborhood views with different spatial ranges for each pixel, enabling the model to perceive the feature layer structure from complementary spatial perspectives. Each view is input in parallel to the corresponding encoder for independent feature extraction, and then integrated into a unified representation through a learnable weighted fusion mechanism. This representation directly participates in subsequent deep clustering and joint optimization, realizing multi-view collaborative fusion at the feature level. It can more effectively capture complementary features across views, providing richer and more discriminative input representations for deep clustering.

[0023] (2) This invention addresses the problem of weak signal detection under low signal-to-noise ratio conditions in spaceborne lidar by improving detection capabilities in two ways. First, it uses the statistical characteristics of ideal atmospheric molecule attenuation backscattered signals calculated based on the Rayleigh scattering model as a physical prior. During the pre-training stage of the autoencoder, Gaussian noise conforming to these statistical characteristics is added to the input signal to guide the encoder in extracting discriminative embedding features. This ensures good separation between background molecular signals and feature layer signals in the embedding space, thereby providing high-quality initial feature representations for subsequent deep clustering and realizing physically based feature learning. Second, it uses multiple two-dimensional neighborhood views with different spatial ranges to extract multi-view spatial context features. Small-range views are sensitive to feature layer boundaries, while large-range views can effectively enhance weak signal response under low signal-to-noise ratio conditions. Complementary enhancement is achieved through a multi-view fusion strategy.

[0024] (3) This invention employs a multi-stage cluster merging strategy, using statistical tests (KS test, binomial distribution) as interpreters for deep clustering results. First, reference clusters representing the feature layer and non-feature layer are identified. Second, the KS test is used to assign the remaining physically meaningless clusters to the feature layer or non-feature layer based on evidence. Finally, pixel-level optimization of non-feature layer samples is performed based on the cumulative probability of the binomial distribution. This strategy effectively alleviates the oversegmentation problem of unsupervised clustering and can recall some potentially missed weak signals during the cluster merging process, thereby improving the detection capability of weak signal feature layers and realizing a physical interpretation of deep clustering results combined with statistical tests. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the unsupervised identification method for the feature layer of spaceborne lidar according to the present invention; Figure 2 This is a comparison of nighttime and daytime layer detection results in the Sahara region according to an embodiment of the present invention; Figure 2 'a' represents a profile of the total attenuation backscattering at 532nm in the nighttime case; Figure 2 b represents a cross-sectional view of the total attenuation backscattering at 532nm in the daytime case; Figure 2 c represents the feature layer detection results of the nighttime case CALIPSO VFM at a resolution of 1–80 km; Figure 2 d represents the feature layer detection results of the daytime case CALIPSO VFM at a resolution of 1–80 km; Figure 2 e represents the feature layer detection result of the nighttime case PG-MVC at a resolution of 5 kilometers; Figure 2 f represents the feature layer detection result of the daytime case PG-MVC at a resolution of 5 kilometers; Figure 2 g represents the comparison results of feature layer detection differences between the nighttime cases PG-MVC and SIBYL. The red box marks the area of ​​over-detection by SIBYL at coarse resolution. Figure 2 h represents the comparison results of feature layer detection differences between daytime cases PG-MVC and SIBYL; Figure 2 i represents the attenuated backscattering profile at latitude 18.8°N in the selected nighttime case. The green part represents the feature layer signal detected by both PG-MVC and SIBYL, the blue part represents the feature layer signal detected only by PG-MVC, and the orange part represents the feature layer signal detected only by SIBYL. Figure 2 j represents the attenuated backscatter profile at latitude 28.3°N in the selected nighttime case; Figure 2 k represents the attenuated backscatter profile at latitude 21.8°N in the selected daytime case; Figure 2 l represents the attenuated backscatter profile at latitude 26.08°N in the selected daytime case; Figure 3 This invention provides a statistical analysis of the percentage of samples in each of the 30 clusters output by the PG-MVC model in an embodiment of the invention. Detailed Implementation

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0028] This invention proposes an unsupervised identification method for feature layers of spaceborne lidar. Based on unsupervised clustering, this method first simulates atmospheric pure molecular signals and statistically analyzes their distribution as a physical prior, guiding the model to learn the nature of noise and perform signal purification. Second, it extracts spatial context features using two-dimensional neighborhood sampling across different spatial ranges to construct a multi-view training dataset. Finally, it constructs a deep convolutional autoencoder and a deep clustering network, integrating signal purification with unsupervised clustering, ultimately directly outputting binary classification results for feature layers (clouds / aerosols) and non-feature layers from the original signal.

[0029] Example 1 like Figure 1 As shown, this embodiment provides an unsupervised identification method for the feature layer of a spaceborne lidar, specifically including the following steps: S1, Data Preprocessing and Physical Prior Construction First, attenuated backscattering coefficients and auxiliary meteorological data are extracted from L1-level spaceborne lidar products. Preprocessing operations such as surface signal removal, profile averaging, and median filtering are then performed to improve the signal-to-noise ratio and suppress background noise. Second, the attenuated backscattering signal of atmospheric molecules under ideal cloudless conditions is simulated using the auxiliary meteorological data. Finally, statistical analysis is performed on long-term (e.g., one year) simulated molecular signals for different geographical regions, altitudes, and time periods, calculating their mean μ and standard deviation σ. This prior characterizes the physical and statistical properties of the background atmospheric molecular signal.

[0030] S2, Multi-view Spatial Context Data Extraction To avoid low-resolution averaging, multiple two-dimensional neighborhood samples of different sizes (e.g., 3×3, 5×5, 7×7, 9×9, 11×11) are used to traverse the preprocessed attenuated backscattered image with a stride of 1. Each neighborhood sample result is treated as an independent view, and both the total attenuated backscattering coefficient and the vertical attenuated backscattering coefficient are extracted. Multi-view neighborhood sampling captures neighborhood spatial context information, constructing a multi-view spatial context dataset.

[0031] S3, Physically Guided Deep Autoencoder Pre-training An independent convolutional autoencoder is constructed for each view. During the pre-training phase, Gaussian noise conforming to the statistical parameters (μ, σ) of atmospheric molecular signals corresponding to the geographical region, altitude layer, and day / night time period of the input view's center pixel is generated and added to the input signal of the corresponding view. The autoencoder is trained to reconstruct the original input signal before noise from the noise-added signal. By minimizing the reconstruction error, the model is guided to learn to recognize background signal components that conform to the statistical characteristics of atmospheric molecules. The core of this step is to use physical priors to guide the encoder to extract discriminative embedding features, so that the background molecular signals and feature layer signals are well separated in the embedding space, thereby providing high-quality initial feature representations for subsequent deep clustering. In other words, the pre-training of the autoencoder is not only a signal processing process, but also a feature learning process serving the deep clustering task.

[0032] S4, Multi-view Deep Clustering Joint Training After pre-training the autoencoders for each view, the low-dimensional embedding features output by all view encoders are weighted and fused, with the weights for each view preset to be equal, to obtain a unified multi-view representation. Deep clustering training is then performed on this representation space, and the clustering loss is optimized by minimizing the KL divergence between the soft-assigned distribution and the target distribution. The initial number of clusters is set to 30, and preliminary clustering results are output.

[0033] S5. Clustering and Refinement of Physical Constraints The preliminary clustering results are subjected to multi-stage physical constraint cluster merging: First, reference clusters representing the feature layer and non-feature layer are determined based on the sample frequency and feature space similarity within the cluster; second, the attenuation backscattering coefficient distribution of the remaining clusters is compared with that of the reference clusters using the Kolmogorov-Smirnov test to assign their categories; then, the probability of the non-feature layer is calculated based on the cumulative probability of the binomial distribution, and pixel-level optimization is performed on the non-feature layer samples to recover weak signals that may have been missed during the cluster merging process, and finally, the binary classification recognition results of the feature layer and non-feature layer are output.

[0034] This embodiment will describe in detail the specific implementation process in the Sahara region. The Sahara region has a perennial layer of dust aerosols and complex background noise, making it an ideal area to verify the feature layer recognition effect of this invention. This embodiment uses CALIPSO / CALIOP's L1B and L2 VFM products as data sources, selecting observation data from the entire year of 2019, and elaborates in detail on the specific implementation steps, parameter settings, and detection results of the method.

[0035] This embodiment uses the L1B version 4.51 product from the CALIPSO satellite CALIOP payload, specifically including: the 532nm total attenuation backscattering coefficient and the 532nm vertical attenuation backscattering coefficient, used to characterize the intensity of atmospheric echo signals; and auxiliary meteorological data, including atmospheric molecular number density, ozone number density, air pressure, and temperature, used to simulate pure atmospheric molecular signals. Simultaneously, the L2VFM product is used as the validation benchmark. The study area is set in the Sahara region, geographically covering 14°–35°N and 17°W–39°E. The time range is the entire year of 2019, with 40 days randomly selected for model training (approximately 500,000 training samples), and the remaining days used for validation and testing. The vertical range is limited to 0–20 km.

[0036] Data preprocessing: First, the original 333m horizontal resolution data was adjusted to 1km resolution by averaging every three adjacent profiles. This operation improves the signal-to-noise ratio while preserving sufficient spatial details for feature layer boundary identification. Then, using the product's built-in Digital Elevation Model (DEM) data, echo signals from the ground surface were identified and removed to avoid strong surface signals interfering with subsequent model training. Next, a median filter with 7×7 neighborhood sampling was used to process the attenuated backscattered image. Previous comparisons of the filtering effects across different spatial ranges showed that 7×7 neighborhood sampling effectively suppresses random noise while maintaining the clarity of feature layer boundaries and internal fine structures. Subsequently, five different spatial ranges of two-dimensional neighborhood sampling (3×3, 5×5, 7×7, 9×9, 11×11) were used to segment the preprocessed image with a step size of 1. Two channels were extracted from each spatial range: the total attenuated backscattering coefficient and the vertical attenuated backscattering coefficient. To balance sample size and training efficiency, profiles for 40 days were randomly selected from the data of the entire year of 2019. After multi-view neighborhood sampling and segmentation, approximately 500,000 training samples were obtained. The complementary advantages of the multi-view complementary strategy are: small-range views are sensitive to changes in feature layer boundaries, which is beneficial for accurately delineating layer contours; large-range views can effectively enhance the detection capability of weak signals under low signal-to-noise ratio conditions.

[0037] Background Atmospheric Molecular Signal Simulation: Based on auxiliary meteorological data provided by the CALIPSO L1B product, the attenuation backscattering coefficient of pure atmospheric molecules under ideal cloudless and aerosol-free conditions is simulated using lidar equations. The calculation formula is as follows: Where r is the detection height; The backscattering coefficient of atmospheric molecules; The atmospheric molecule attenuation backscattering coefficient; The bidirectional transmittance of ozone; The bidirectional transmittance of molecules is given. Based on meteorological auxiliary data from the CALIPSO L1b product, the ideal atmospheric molecule attenuation backscattering coefficient can be theoretically calculated. The Kolmogorov-Smirnov test further verifies that the simulated molecular signal approximately follows a Gaussian distribution at all altitude levels, providing a theoretical basis for the normality assumption. Finally, we organize the above statistical analysis results into a three-dimensional lookup table: region (3 sub-regions); altitude level; time period (day / night). Each entry contains two statistical parameters: mean μ and standard deviation σ. This lookup table constitutes the physical prior library of this invention.

[0038] Multi-view construction and encoder design: To fully exploit the spatial structure and vertical distribution features embedded in LiDAR signals, this invention constructs a multi-view input. Each view corresponds to a specific spatial neighborhood (3×3, 5×5, 7×7, 9×9, and 11×11) and contains two channels: total attenuation backscattering coefficient and vertical attenuation backscattering coefficient. Each view is equipped with an independent convolutional autoencoder as its dedicated encoder, and each convolutional autoencoder adopts a differentiated architecture design. Views with different spatial ranges correspond to autoencoders with different structures or different depths to achieve adaptive feature extraction. For views with larger spatial ranges (such as 11×11 and 9×9), a deep convolutional autoencoder is used. The encoder contains three convolutional layers, which expand the receptive field through progressive downsampling to capture a large range of spatial context information; batch normalization and two-dimensional dropout layers are added after each convolutional layer to prevent overfitting; the output is flattened and mapped to a low-dimensional embedding space through a fully connected layer. For views with medium spatial range (such as 7×7 or 5×5), a convolutional autoencoder is also used, but the network depth and number of parameters are adapted and adjusted according to the input size to control the model complexity while ensuring feature extraction capabilities. For views with smaller spatial range (such as 3×3), a fully connected autoencoder is used, with stacked fully connected layers forming the encoder and decoder, avoiding information loss and overfitting risks caused by convolutional operations under small input sizes.

[0039] Each view's autoencoder maps the input to a low-dimensional embedding layer of uniform dimension, facilitating subsequent multi-view feature fusion. Dropout and batch normalization techniques are employed during training to prevent overfitting. This differentiated design allows views of different spatial extents to extract features with the most suitable network structure, avoiding the one-size-fits-all approach of a single network structure for inputs at different scales. The model training uses the Adam optimizer with an initial learning rate of 0.001, a batch size of 256, and 100 training epochs. In each training epoch, the reconstruction loss is calculated via forward propagation, the gradient is calculated via backpropagation, and the optimizer updates the weight parameters ω and ω′ of the encoder and decoder. The multi-view complementarity strategy leverages the complementary advantages of different view spatial extents: small-scale views are sensitive to changes near feature layer boundaries, which is beneficial for depicting fine contours; large-scale views effectively enhance the detection capability of weak signals under low signal-to-noise ratio conditions. Simultaneously, the two-dimensional neighborhood sampling design fully utilizes the spatial context information of adjacent profiles, improving the robustness of complex vertical structure recognition.

[0040] Physics-Guided Self-Supervised Pre-training and Denoising: Before joint training, the convolutional autoencoder for each view undergoes independent self-supervised pre-training. Based on the background molecular signal simulated in the preprocessing stage, the mean μ and standard deviation σ at different heights are statistically characterized. Gaussian noise with the same statistical properties is then generated and added to the attenuated backscattering coefficients of the model input. By minimizing the reconstruction loss function, the convolutional autoencoder is self-supervised trained. This method effectively guides the model to learn and suppress noise, achieving signal cleansing while learning features. and decoder The parameters are updated by minimizing the reconstruction error: Where n is the number of samples, Let be the center pixel of the two-dimensional neighborhood view of the i-th input model. F represents the Gaussian noise added to the input model samples; ω and ω′ represent the learnable weight parameters of the encoder and decoder networks (including the weight matrices and bias terms of each convolutional layer), respectively. ω G represents an encoder network with weight parameter ω. ω′ F represents a decoder network with weight parameters ω′. ω (x i ) represents the low-dimensional embedding features of the encoder for the noisy input and output. This represents the center pixel reconstructed by the decoder. During training, ω and ω′ are initially initialized randomly, and then iteratively updated using backpropagation and gradient descent optimizers by minimizing the reconstruction loss function until the loss function converges. By minimizing this reconstruction error, the decoder is able to recover the center pixel of the original input from the embedded features.

[0041] During training, regularization techniques such as Dropout and batch normalization are employed to prevent overfitting. This process enables the model to learn and suppress the distribution characteristics of atmospheric molecular noise, thereby enhancing the feature purity in the embedded representation and improving the representational power of the view-specific encoder. The reconstructed denoised signal provides a high-quality feature representation for subsequent clustering.

[0042] Multi-view joint training and feature fusion: After independent pre-training of each view encoder, the model enters the multi-view joint training phase. The overall objective function for joint training is defined as: in, To reconstruct the loss, The clustering loss is γ, which is a coefficient that controls the degree of deformation of the embedding space (set to 0.1 in this embodiment based on experience) to balance feature optimization and structure preservation.

[0043] Clustering loss A deep embedding clustering paradigm is adopted, based on Kullback-Leibler (KL) divergence, aiming to minimize the distance between embedded features and cluster centers. Embedded features from all views are fused into a unified multi-view representation through weighted averaging. To avoid missed or false detections due to insufficient initial cluster size, the initial cluster size is set to 30, which are then merged into two final output categories: feature layer and non-feature layer, using a cluster merging strategy. The clustering loss is defined as the KL divergence between the soft-assigned distribution Q and the target distribution P: in, It is a soft-assigned vector The j-th element represents the probability that the embedded feature i belongs to cluster j. Let be the probability that the i-th sample in the target distribution P belongs to the j-th cluster. This probability is derived from the soft-assignment distribution Q. It refines the cluster centers and enhances intra-cluster compactness by emphasizing high-confidence assignments. Its calculation method is as follows: Cluster merging: The deep clustering model outputs soft-assigned labels for 30 clusters through its clustering layers. To enhance the interpretability and physical consistency of the clustering results, a three-stage strategy is adopted to progressively merge these clusters into two final categories: the feature layer and the non-feature layer.

[0044] The first stage involves identifying reference clusters representing the clear sky background (i.e., the non-feature layer) and the feature layer. It's important to note that during training, the physically guided autoencoder first maps the original LiDAR signal to a denoised high-dimensional feature space. In this space, the feature layer and background noise are well separated. Further analysis shows that during the deep clustering stage, over 95% of the samples are clustered into two main clusters (e.g., ...). Figure 3 As shown, Figure 3 This represents the percentage of samples in each of the 30 clusters output by the statistical PG-MVC model (the percentage of samples in each cluster out of the total samples). The remaining clusters account for a very small proportion. Therefore, these two main clusters basically represent the feature layer and the non-feature layer, which are significantly different from each other. They have obvious differences in signal characteristics, and it is not difficult to identify the corresponding reference cluster.

[0045] This paper primarily identifies reference clusters representing the clear sky background (i.e., the non-feature layer) and the feature layer from the two main clusters based on the sample size proportion of each cluster and the feature space distance between clusters. The specific method is as follows: the cluster with the highest sample size proportion is determined as the non-feature layer reference cluster, because clear sky background pixels occupy the vast majority of space in lidar observations; simultaneously, the average Euclidean distance between each cluster center and all other cluster centers is calculated, and the cluster with the largest average distance is determined as the feature layer reference cluster. This cluster is most clearly distinguishable from other clusters in the feature space, typically corresponding to a feature layer where the signal intensity significantly deviates from the background.

[0046] In the second stage, empirical probability distributions of attenuated backscattering coefficients are constructed for the feature layer and non-feature layer reference samples identified in the first stage. For each remaining unlabeled cluster, the Kolmogorov-Smirnov test is used to quantify the difference between its distribution and the two reference distributions, and the corresponding significance probability value p is calculated. ks If the clustering is compared with the reference distribution of the feature layer, the resulting p... ks1 The value is simultaneously greater than p obtained by comparing it with the non-feature layer reference distribution. ks2 If the cluster is assigned a value and a preset significance threshold (α=0.05), then the cluster is assigned to the feature layer category; otherwise, it is assigned to the non-feature layer category.

[0047] The third stage involves pixel-level optimization of non-feature layer samples, primarily aimed at recalling weak feature layer signals that were misclassified as background during the first two clustering and merging steps. Because the distribution of weak feature layers overlaps with the background atmosphere, statistical tests based on the overall clustering distribution are insufficient to completely distinguish them, leading to the omission of some feature layer pixels. Therefore, the algorithm utilizes multi-view neighborhood sampling to calculate the attenuation scattering ratio of each pixel and combines it with the binomial distribution cumulative probability to assess its probability of belonging to clear sky; only when a pixel is identified as a feature layer in at least one small-range view and at least one large-range view is it corrected to be a feature layer. This multi-view... Figure 1 Consistency constraints can effectively recover missed weak signals and avoid misjudgments caused by single-view noise or smoothing, thereby improving the sensitivity of weak signal detection while maintaining a low false detection rate. Specifically, for each pixel, the probability of it belonging to clear sky is calculated based on the cumulative probability of a binomial distribution by counting the number of pixels with an attenuated scattering ratio (ASR) exceeding a preset threshold in different spatial ranges (3×3, 5×5, 7×7, 9×9, and 11×11). The ASR calculation formula is: Where r is the detection height; The backscattering coefficient; 'Represents the total attenuation backscattering coefficient, The backscattering coefficient of atmospheric molecules. The atmospheric molecule attenuation backscattering coefficient; The bidirectional transmittance of ozone; The bidirectional transmittance of the molecule.

[0048] The measured lidar signal can be considered as consisting of an ideal pure signal and Poisson noise, which approximately follows a Gaussian distribution for a sufficiently strong signal. Therefore, the probability that the measured attenuation scattering ratio (ASR) is greater than or less than the ideal ASR is considered to be the same (i.e., p = 1 / 2). Here, we choose the ideal ASR value (1.175) obtained from Mao et al. (2024). Referring to the method used by Mao et al. (2021) to calculate the probability of clear skies, for m pixels in a selected two-dimensional neighborhood sample, although there is some correlation between these pixel grid points (Vaillant de Guélis et al., 2021), we simply assume that each grid point is independent. In probability and statistics theory, the behavior of these m selected grid points is equivalent to m repeated Bernoulli trials, i.e., following a binomial distribution with parameters m and p. The probability that exactly u grid points have an ASR greater than the ideal value is: Where C(m,u) represents the binomial distribution coefficient, and in this embodiment, p equals 1 / 2. Therefore, for measurements under clear sky conditions, the probability that there are more than u pixels with an attenuation scattering ratio (ASR) greater than the ideal attenuation scattering ratio (ASR) (i.e., the probability of clear sky) can be calculated using the following formula: For a fixed value of m, the larger the attenuation scattering ratio (ASR) of a selected pixel grid point relative to the ideal attenuation scattering ratio, the lower the probability that it belongs to a clear sky. Therefore, a very small The value indicates that this situation is unlikely to occur if the atmosphere is clear, meaning that the selected pixel grid is more likely to be a feature layer.

[0049] Here, we refer to the experimental results of Mao et al. (2021) for the selection of the preset threshold, which is 0.01. This paper adopts multi-view... Figure 2This method leverages the spatial extent of different views through multi-dimensional neighborhood sampling to improve detection capabilities. Two-dimensional neighborhood samples of varying sizes are extracted centered on each pixel grid point. The probability of clear skies in each neighborhood sample is calculated. If the probability of a central pixel belonging to clear skies within any small view (3×3, 5×5, or 7×7) is less than 0.01, and simultaneously, the probability of a central pixel belonging to clear skies within any large view (9×9 or 11×11) is also less than 0.01, then that central pixel is marked as a feature layer pixel. This multi-view fusion mechanism utilizes the boundary sensitivity of small-scale views and the noise suppression capability of large-scale views, achieving robust detection of weak feature layers. Through iterative merging, all pixels are ultimately classified into two categories: feature layer and non-feature layer. This method effectively alleviates the oversegmentation problem common in unsupervised clustering, ensuring physical consistency and stable recognition results.

[0050] like Figure 2 As shown, two typical observation cases from the Sahara region are selected for feature layer detection performance analysis: Case 1 (February 19, 2019, 01:58:11 UTC, nighttime): The original signal showed a dust layer at altitudes of 0–7 km, but the signal distribution was discontinuous, and some weak signal areas were difficult to identify. After detection using the method of this invention (e.g....), Figure 2 As shown in e), the spatial distribution structure of the dust layer is reconstructed more completely, with clear layer boundaries, while effectively suppressing background noise interference. Compared to the official SIBYL algorithm, which requires resolutions of 20km and 80km to identify weak feature layers, the PG-MVC model can achieve more accurate feature layer detection at a high resolution of 5km.

[0051] Case Study 2 (January 13, 2019, 10:27:09 UTC, daytime): Under low signal-to-noise ratio conditions during the day, the original signal made it difficult to capture the complete structure of the dust layer. After detection using the method of this invention, not only was the main body of the dust layer (such as...) successfully identified... Figure 2 As shown in f), it also detected weak signal regions that the official SIBYL algorithm missed at 1–5km resolution, while avoiding the overestimation of layer range caused by low-resolution averaging (e.g. Figure 2 (as shown in h).

[0052] Example 2 This embodiment provides an unsupervised identification system for the feature layer of a spaceborne lidar, including: The data preprocessing and physical prior construction module is used to extract the attenuation backscattering coefficient and auxiliary meteorological data of the L1-level product of the spaceborne lidar and complete the preprocessing. It uses the auxiliary meteorological data to simulate the atmospheric molecule attenuation backscattering signal under ideal cloudless and aerosol-free conditions, statistically analyze its distribution characteristics, and construct a three-dimensional physical prior lookup table. The multi-view spatial context data extraction module is used to traverse the preprocessed attenuated backscattered image by sampling two-dimensional neighborhoods with multiple different spatial ranges. Each spatial neighborhood sample is treated as an independent view, and a multi-view spatial context dataset is constructed for each pixel in the image. The physics-guided autoencoder pre-training module configures an independent convolutional autoencoder for each view. Based on the geographic region, altitude layer, and day / night time period corresponding to the center pixel of each input view, it retrieves noise statistics from a physical prior lookup table, generates Gaussian noise conforming to these statistical characteristics, and adds it to the input signal of the corresponding view. The autoencoder is then trained to reconstruct the original input signal before noise addition from the noise-added signal. By minimizing the reconstruction error, the model is guided to learn and recognize background signal components that conform to the statistical characteristics of atmospheric molecules. The core of this step lies in using physics-guided priors to extract discriminative embedding features, ensuring good separation between background molecular signals and feature layer signals in the embedding space, thus providing high-quality initial feature representations for subsequent deep clustering. In other words, the pre-training of the autoencoder is not only a signal processing process but also a feature learning process serving the deep clustering task. The multi-view deep clustering joint training module is used to weightedly fuse the low-dimensional embedding features output by all view encoders to obtain a unified multi-view representation, while optimizing the autoencoder reconstruction loss and deep clustering loss. The initial clustering results are obtained by minimizing the KL divergence between the soft assignment distribution and the target distribution. The Physical Constraint Cluster Merging and Refinement module is used to perform multi-stage physical constraint cluster merging on the preliminary clustering results to obtain preliminary binary classification results of the feature layer and non-feature layer. Then, based on the cumulative probability of the binomial distribution, the clear sky probability is calculated, and pixel-level optimization is performed on the non-feature layer samples to recover weak signals that may have been missed in the cluster merging process, and the final feature layer recognition result is output.

[0053] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An unsupervised identification method for feature layers of a spaceborne lidar, characterized in that, Includes the following steps: S1. Extract the attenuation backscattering coefficient of the spaceborne lidar and auxiliary meteorological data, and perform preprocessing operations such as removing surface signals, profile averaging and median filtering on the raw data; use the auxiliary meteorological data to simulate the atmospheric molecular attenuation backscattering signal under ideal cloudless and aerosol-free conditions, statistically analyze its distribution characteristics, and construct a three-dimensional physical prior lookup table. S2. Using multiple two-dimensional neighborhood samplings with different spatial ranges, the preprocessed attenuated backscattered image is traversed. Each neighborhood sampling result is treated as an independent view. Neighborhood sampling data is extracted for each pixel in the image to construct a multi-view spatial context dataset. S3. Construct an independent convolutional autoencoder for each view. During the pre-training stage, generate Gaussian noise that conforms to statistical characteristics based on the geographical region, altitude layer, and day / night time period corresponding to the center pixel of each view and add it to the input signal of the corresponding view. Train the autoencoder to reconstruct the original input signal before adding noise from the signal after adding noise. By minimizing the reconstruction error, guide the model to learn to recognize background signals that conform to the statistical characteristics of atmospheric molecules. S4. After completing the pre-training of the autoencoders for each view, the low-dimensional embedding features output by all view encoders are weighted and fused, with the weights of each view preset to be equal, to obtain a unified multi-view representation; deep clustering training is performed on this multi-view representation space, and the clustering loss is optimized by minimizing the KL divergence between the soft assignment distribution and the target distribution to obtain preliminary clustering results, providing a basis for subsequent physical constraint merging. S5. Based on the similarity between clusters and the probability distribution of signals within each cluster, the preliminary clustering results are merged under multi-stage physical constraints to obtain preliminary binary classification results of the feature layer and non-feature layer; then, the non-feature layer probability is calculated based on the cumulative probability of the binomial distribution, and the non-feature layer samples are optimized at the pixel level to finally output the binary classification recognition results of the feature layer and non-feature layer. The cluster merging of the multi-stage physical constraints specifically includes the following steps: S51. Based on the sample ratio within each cluster and the similarity measure between clusters, identify reference clusters representing the non-feature layer and the feature layer; among them, the cluster with the highest sample frequency is selected as the non-feature layer reference cluster, and the cluster that exhibits strong separability in the feature space is designated as the feature layer reference cluster. S52. Construct empirical probability distributions of attenuation backscattering coefficients for reference clusters in the feature layer and non-feature layer, use the Kolmogorov-Smirnov test to quantify the difference between each unlabeled cluster and the two types of reference distributions, and assign categories to unlabeled clusters based on the test results. S53. Perform pixel-level optimization on non-feature layer samples to recover weak signals that may be missed during cluster merging. By counting the number of pixels with attenuation scattering ratio exceeding a preset threshold in different spatial ranges, calculate the probability of each pixel belonging to the non-feature layer based on the cumulative probability of the binomial distribution, so as to recover the weak signal feature layer that may be missed during cluster merging.

2. The unsupervised identification method for feature layers of spaceborne lidar according to claim 1, characterized in that, In step S1, the construction of the three-dimensional physical prior lookup table specifically includes: statistically analyzing the simulated atmospheric molecular attenuation backscattering signals over a long period of time (at least one year) for different geographical sub-regions, different altitude layers, and day and night periods, calculating their mean μ and standard deviation σ, and constructing a physical prior lookup table containing three dimensions: region, altitude layer, and time period.

3. The unsupervised identification method for feature layers of spaceborne lidar according to claim 1, characterized in that, In step S3, an independent convolutional autoencoder is constructed for each view using a differentiated architecture design. Views with different spatial ranges correspond to autoencoders with different structures or depths. For views with larger spatial ranges, such as 11×11 and 9×9 two-dimensional views, a deep convolutional autoencoder is used. This deep convolutional autoencoder contains three convolutional layers, each followed by batch normalization and a two-dimensional dropout layer to prevent overfitting. The output is flattened and then mapped to a low-dimensional embedding space through a fully connected layer. For views with medium spatial ranges, such as 7×7 and 5×5 two-dimensional views... The diagram also employs a convolutional autoencoder, but the network depth and number of parameters are adapted and adjusted according to the input size to control model complexity while ensuring feature extraction capabilities. For views with smaller spatial ranges, such as 3×3 two-dimensional views, a fully connected autoencoder is used. The fully connected autoencoder consists of stacked fully connected layers forming an encoder and decoder, avoiding information loss and overfitting risks caused by convolutional operations under small input sizes. The autoencoders for each view map the input to a low-dimensional embedding layer of the same dimension, and Dropout and batch normalization techniques are used during training to prevent overfitting.

4. The unsupervised identification method for feature layers of spaceborne lidar according to claim 3, characterized in that, The parameters of the encoder and decoder are updated by minimizing the reconstruction error: ; Where n is the number of samples, Let be the center pixel of the two-dimensional neighborhood view of the i-th input model. F represents the Gaussian noise added to the input sample; ω and ω′ represent the weight parameters of the encoder and decoder networks, respectively. ω G represents an encoder network with weight parameter ω. ω′ F represents a decoder network with weight parameters ω′. ω (x i ) represents the low-dimensional embedding features of the encoder for the noisy input and output. This represents the center pixel reconstructed by the decoder.

5. The unsupervised identification method for feature layers of spaceborne lidar according to claim 1, characterized in that, In step S4, after the independent pre-training of each view encoder is completed, the model enters the multi-view joint training stage. The overall objective function of the joint training is defined as: ; in, To reconstruct the loss, The clustering loss is γ, which is a coefficient controlling the degree of deformation of the embedding space, and its value is 0.

1. The clustering loss adopts a deep embedding clustering paradigm, constructed based on KL divergence, and is defined as the KL divergence between the soft-assigned distribution Q and the target distribution P: ; in, It is a soft-assigned vector The j-th element represents the probability that embedded feature i belongs to cluster j. Let be the probability that the i-th sample in the target distribution P belongs to the j-th cluster, which is derived from the soft-assigned distribution Q.

6. The unsupervised identification method for feature layers of spaceborne lidar according to claim 1, characterized in that, In step S52, the Kolmogorov-Smirnov test is used to calculate the significance probability value p obtained by comparing the distribution of the unlabeled cluster and the reference cluster of the feature layer. ks1 The significance probability value p obtained by comparing the distribution of unlabeled clusters with that of non-feature layer reference clusters. ks2 If p ks1 At the same time greater than p ks2 And a preset saliency threshold is used to assign the unlabeled cluster to a feature layer category; Otherwise, it is assigned to a non-feature layer category.

7. The unsupervised identification method for feature layers of spaceborne lidar according to claim 1, characterized in that, In step S53, pixel-level optimization is performed on the non-feature layer samples: Calculate the attenuation scattering ratio of pixels in different spatial ranges, count the number of pixels whose attenuation scattering ratio exceeds a preset threshold, and calculate the probability that the pixel belongs to the non-feature layer based on the cumulative probability of the binomial distribution; if the pixel is identified as a feature layer in at least any small neighborhood and any large neighborhood, then it is finally marked as a feature layer.

8. A spaceborne lidar feature layer unsupervised identification system, used to implement the method according to any one of claims 1-7, characterized in that, include: The data preprocessing and physical prior construction module is used to extract the attenuation backscattering coefficient of the spaceborne lidar and auxiliary meteorological data and complete the preprocessing. It uses the auxiliary meteorological data to simulate the atmospheric molecule attenuation backscattering signal under ideal cloudless and aerosol-free conditions, statistically analyze its distribution characteristics, and construct a three-dimensional physical prior lookup table. The multi-view spatial context data extraction module is used to traverse the preprocessed attenuated backscattered image by sampling two-dimensional neighborhoods with multiple different spatial ranges, and to construct a multi-view spatial context dataset for each pixel in the image, with each neighborhood sample as an independent view. The physics-guided autoencoder pre-training module is used to configure an independent convolutional autoencoder for each view. Based on the geographic region, altitude layer, and day / night time period corresponding to the center pixel of each input view, it obtains noise statistical parameters from the physical prior lookup table, generates Gaussian noise that conforms to the statistical characteristics, and adds it to the input signal of the corresponding view. The autoencoder is trained to reconstruct the original input signal before adding noise from the signal after adding noise. By minimizing the reconstruction error, the model is guided to learn to recognize background signal components that conform to the statistical characteristics of atmospheric molecules. The multi-view deep clustering joint training module is used to weightedly fuse the low-dimensional embedding features output by all view encoders to obtain a unified multi-view representation, while optimizing the autoencoder reconstruction loss and deep clustering loss. The initial clustering results are obtained by minimizing the KL divergence between the soft assignment distribution and the target distribution. The Physical Constraint Cluster Merging and Refinement module is used to perform multi-stage physical constraint cluster merging on the preliminary clustering results to obtain preliminary binary classification results of the feature layer and non-feature layer. Then, based on the cumulative probability of the binomial distribution, the clear sky probability is calculated, and pixel-level optimization is performed on the non-feature layer samples to recover weak signals that may have been missed in the cluster merging process, and the final feature layer recognition result is output.

Images