Small sample PolSAR image classification method based on DBCL-3DDC

Abstract

The invention discloses a small sample PolSAR image classification method based on DBCL-3DDC, and relates to the field of radar remote sensing image interpretation. The method comprises the steps that pixel points in a PolSAR image are sliced, and image slices are obtained and serve as input of a DBCL-3DDC network; wherein the DBCL-3DDC network comprises a comparison branch, a hybrid branch and a classification branch; in the comparison branch and the mixed branch, a random enhancement strategy is adopted to pre-process image slices to obtain a slice enhancement view, the comparison branch and the mixed branch are pre-trained on the slice enhancement view, and network parameters of the comparison branch and the mixed branch are updated based on a multi-level comparison learning updating strategy; fine adjustment based on small sample classification is carried out on the classification branches in the image slices, and network parameters of the classification branches are updated; and the updated DBCL-3DDC network is used for the classification task of the whole PolSAR image. Compared with the prior art, the method has the advantages that the feature representation of the multi-instance PolSAR image can be extracted in a heuristic mode, high-precision classification is achieved under the small sample condition, and the method is suitable for the PolSAR field with scarce label samples.

Description

Technical Field

[0001] The present invention relates to the technical field of radar remote sensing image interpretation, and more specifically, to a small sample PolSAR image classification method based on DBCL-3DDC. Background Art

[0002] Synthetic Aperture Radar (SAR) has the advantage of obtaining information in all weather conditions and all day, and its products have important application value in the fields of national economy. Early SAR imaging systems were single-frequency and single-polarization, which could not fully obtain the key information of the target scattering process contained in the scattered signal, and lacked the description of the target scattering characteristics such as phase information and polarization.

[0003] Polarimetric Synthetic Aperture Radar (PolSAR) is a multi-channel, multi-parameter radar imaging system. Unlike optical imaging systems, PolSAR can work stably in all weather conditions and has strong penetration capabilities. More importantly, compared with traditional single-polarization SAR imaging systems, PolSAR not only retains its traditional advantages, but can also record four different polarization states by utilizing the polarization characteristics of electromagnetic waves, thereby obtaining richer target information and providing more accurate data support for subsequent research and interpretation.

[0004] Remote sensing technology has shown unique advantages in large-scale monitoring based on remote sensing images due to its wide coverage, strong timeliness, economy and convenience, and has great application potential in refined classification. At present, remote sensing classification technology based on PolSAR images aims to accurately classify each pixel in the image into a specific terrain category.

[0005] In recent years, deep learning (DL) methods have made breakthrough progress in many fields. They achieve nonlinear fitting through a large amount of data-driven methods, avoid complex mathematical modeling and parameter selection, and transfer computational overhead from the prediction end to the training end, effectively improving model performance. With the continuous improvement of SAR sensors and the further development of remote sensing technology, DL technology can automatically learn the deep features of images in PolSAR image classification, and perform effective feature extraction and classification, showing its superiority in processing complex data. However, in PolSAR image classification, the existing technology faces the following problems and shortcomings: (1) High dependence on the number of labeled samples PolSAR images usually have high-dimensional features and complex scattering characteristics. However, compared with optical remote sensing images, the annotation of PolSAR images is not only dense but also requires professional domain knowledge, which makes it difficult to obtain high-quality labels. Most of the existing deep learning methods belong to supervised learning, such as CV-CNN-SE (two-dimensional complex-valued network). The quality of its classification accuracy depends on the number of labeled samples, which limits its performance in small-sample PolSAR image classification tasks.

[0006] (2) Not very targeted to the characteristics of PolSAR images PolSAR image classification is a pixel-level classification task. It usually uses a regular rectangular sliding window method to cut the area around the central pixel into image slices as the input of the network model. Therefore, it can be regarded as a special semantic segmentation problem. However, in areas at the edge of categories or irregular shapes, image slices often contain two or even multiple categories and are rich in multiple semantic information. This feature limits the application of the model in multi-instance tasks. Summary of the invention

[0007] In order to overcome the defects of the prior art described above, namely, high dependence on the number of label samples and lack of pertinence to the characteristics of PolSAR images, the present invention provides a small sample PolSAR image classification method based on DBCL-3DDC.

[0008] In order to solve the above technical problems, the technical solution of the present invention is as follows: In a first aspect, a small sample PolSAR image classification method based on DBCL-3DDC comprises: Slicing the pixels in the PolSAR image to obtain image slices as input of a DBCL-3DDC network; wherein the DBCL-3DDC network includes a comparison branch, a mixing branch, and a classification branch; In the comparison branch and the hybrid branch, a random enhancement strategy is adopted to preprocess the image slice to obtain a slice enhancement view, the comparison branch and the hybrid branch are pre-trained on the slice enhancement view, and the network parameters of the comparison branch and the hybrid branch are updated based on a multi-level contrast learning update strategy; wherein the comparison branch includes an online network and a target network, and the hybrid branch includes a hybrid network with the same structure as the online network and shared network parameters; Allowing the classification branch to perform fine-tuning based on small sample classification on the image slice, and updating the network parameters of the classification branch; The updated DBCL-3DDC network is used for the classification task of the entire PolSAR image.

[0009] In a second aspect, a computer program product comprises a computer program or computer executable instructions, wherein when the computer program or computer executable instructions are executed by a processor, the method described in the first aspect is implemented.

[0010] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: The present invention discloses a small sample PolSAR image classification method based on DBCL-3DDC. The proposed DBCL-3DDC is composed of a multi-network structure and multi-level contrast learning. It is combined with small sample PolSAR images to reduce the model's dependence on the number of labeled samples, enhance the feature learning ability of multi-instance tasks, and fully consider the diversity of data, thereby reducing the time and cost of remote sensing data annotation. It is particularly suitable for the PolSAR field where labeled samples are scarce. In addition, the present invention can achieve high-precision small sample PolSAR classification in both single-phase and multi-phase, can meet the monitoring needs in dynamic environments (such as agricultural dynamic monitoring, disaster warning, environmental protection, etc.), and improve the feasibility of practical application of the model. Compared with the prior art, the performance of the present invention is more stable when there are very few labeled samples, and has higher classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of the small sample PolSAR image classification method in Example 1 of the present application; Figure 2 This is another flowchart of the small sample PolSAR image classification method in Example 1 of the present application; Figure 3 This is a schematic diagram of the structure of the FEM component in Example 1 of the present application; Figure 4 This is a schematic diagram of the structure of the FPM component in Example 1 of the present application; Figure 5 This is a schematic diagram of the structure of the Predictor component in Example 1 of the present application; Figure 6 This is a schematic diagram of the structure of the Classification component in Example 1 of the present application; Figure 7 This is the visualized classification results of different methods in Example 2 of this application on the Flevoland 1989 dataset. DETAILED DESCRIPTION

[0012] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged in appropriate cases, which is only to describe the distinction mode adopted by the objects of the same attributes in the embodiments of the present application when describing. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment containing a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment. The term "determine" widely covers various actions, and may include acquisition, calculation, calculation, processing, derivation, investigation, search (for example, search in a table, database or other data structure), ascertainment, and similar actions, and may also include reception (for example, receiving information), access (for example, accessing data in a memory) and similar actions, and may also include generation, creation, establishment and similar actions, as well as parsing, selection, selection and similar actions, etc. The relevant definitions of other terms will be given in the following description.

[0013] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. In addition, the "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if there is transmission of electrical signals or data between the connected objects.

[0014] The drawings are for illustrative purposes only and should not be construed as limiting the present patent; In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0015] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0016] Example 1 This embodiment provides a small sample PolSAR image classification method based on DBCL-3DDC (Dense Bootstrap Contrastive Learning Method With 3-Dimensional Dynamic Convolution) (referred to as “DBCL-3DDC method”), referring to FIG1 and FIG2, including: Slicing the pixels in the PolSAR image to obtain image slices as input of a DBCL-3DDC network; wherein the DBCL-3DDC network includes a comparison branch, a mixing branch, and a classification branch; In the comparison branch and the mixing branch, a random enhancement strategy is adopted to preprocess the image slice to obtain a slice enhancement view, the comparison branch and the mixing branch are pre-trained on the slice enhancement view, and the network parameters of the comparison branch and the mixing branch are updated based on a multi-level contrast learning update strategy; wherein the comparison branch includes an online network and a target network, and the mixing branch includes a mixed network with the same structure as the online network and shared network parameters; Allowing the classification branch to perform fine-tuning based on small sample classification on the image slice, and updating the network parameters of the classification branch; The updated DBCL-3DDC network is used for the classification task of the entire PolSAR image.

[0017] It should be noted that in the above embodiment, DBCL-3DDC is composed of a multi-network structure and multi-level contrastive learning. It learns global and local representations respectively through global features (Global Feature) and local dense features (Local Dense Feature), heuristically extracts feature representations of multi-instance data, and combines small sample PolSAR images with a small amount of labeled data (only 0.2% of samples for each category) to achieve high-precision classification, which reduces the model's dependence on the number of labeled samples, reflects excellent small sample learning ability and efficient online learning ability, and effectively solves the problem that the characteristics of PolSAR images are not very targeted.

[0018] In some preferred embodiments, the online network It includes the FEM (Feature Extraction Module) component for extracting dense features based on 3DDC (3-Dimensional Dynamic Convolution), the FPM (Feature Projection Module) component for feature modeling, and the Predictor component for prediction; The hybrid network includes a FEM component, a FPM component and a Predictor component; The target network Includes FEM components and FPM components.

[0019] In the above embodiment, dense features of the image are extracted by FEM containing three-dimensional dynamic convolution (3DDC), and global features and local dense features are modeled using FPM components to enhance the feature representation capability of the network in complex data.

[0020] In some specific implementations, each pixel in a PolSAR image is represented by a A sliding window of size is defined, called an image slice, with its center pixel located at Therefore, each pixel in the PolSAR image can be converted into an image slice of size [16×16×number of channels], and the set of these image slices is used as the input of the FEM module.

[0021] In some optional embodiments, referring to FIG. 3 , the FEM component includes three connected convolution blocks, and an attention mechanism (Squeeze-and-Excitation, SE) is also provided between adjacent convolution blocks as a bypass; wherein the convolution block includes a 3DDC layer, a BN layer, a ReLU layer and a maximum pooling layer in sequence; and / or, Referring to Figure 4, the FPM component includes a parallel global feature projection sub-branch for outputting global features and a dense feature projection sub-branch for outputting local dense features; wherein the global feature projection sub-branch includes a global average pooling (Global Average Pool) layer, an FC layer, a BN (BatchNorm) layer, a ReLU layer and an FC layer in sequence, and the dense feature projection sub-branch includes a 3DDC layer, a BN layer, a ReLU layer, a 3DDC layer and an FC layer in sequence.

[0022] In some specific implementations, the 3DDC layer in the FEM component uses a 3DDC with a convolution kernel of 3×3×3, denoted as 3DDC-3.

[0023] Furthermore, between adjacent convolution blocks of the FEM component, an attention mechanism is connected between the global average pooling layer of the preceding convolution block and the 3DDC-3 layer of the succeeding convolution block as a bypass.

[0024] In some other specific implementations, the 3DDC layer in the FPM component uses a 3DDC with a convolution kernel of 1×1×1, denoted as 3DDC-1.

[0025] It should be noted that the FEM component is based on 3DDC, and its purpose is to retain local dense features to improve the model's ability to aggregate similar local representations and solve the problem that the existing technology is not very targeted to the characteristics of PolSAR images. Finally, the FEM output in each network is dense features. , which are represented in each network in the pre-training stage as , and , as the input of the FPM component in the online network, target network and hybrid network respectively. FPM consists of two parallel sub-branches (i.e., global feature projection sub-branch and dense feature projection sub-branch). The two sub-branches receive the same input, enabling the model to simultaneously learn and save global features and local dense features, and finally output global features. and local dense features , which are represented as the first global features in the online network during the pre-training phase and the first local dense feature , the second global feature in the target network and the second local dense feature , and the hybrid global features in the hybrid network and mixed local dense features .

[0026] It should also be noted that in the FPM component, the dense feature projection sub-branch removes the global average pooling layer in the global feature projection sub-branch and replaces the FC with a three-dimensional dynamic convolution layer (such as 3DDC-1), which performs local weighting in the spatial dimension, thereby more flexibly processing feature information of different dimensions. This design enables the model to process features independently in the spatial and channel dimensions, thereby improving the ability to extract and combine features.

[0027] In the above embodiment, the FEM component retains dense features (also called "dense features") based on 3D dynamic convolution, and then learns global and local representations at ratios of 30% and 70% respectively through the FPM component, heuristically extracting feature representations of multi-instance PolSAR images, thereby compensating for the problems existing in general contrastive learning methods in multi-instance data.

[0028] Furthermore, the 3DDC layer is implemented by conditional parameterized convolution, and its convolution kernel is parameterized as a linear combination of four experts, expressed as follows:

[0029] In the formula, represents the activation function; each represents a scalar weight associated with the input, computed by a routing function with learned parameters:

[0030] It should be noted that, in the above embodiment, 3DDC is designed to construct a network in a comparison structure, which can improve the feature extraction capability of the network in complex data and reduce the interference of redundant information.

[0031] In some optional embodiments, as shown in FIG5 , the Predictor component is composed of two layers of FC, a ReLU activation function and a BN layer.

[0032] It should be noted that Predictor is an important component of contrastive learning in this embodiment, located at the end of each branch network (i.e., online network and hybrid network), and is used to predict feature representation. The Predictor is composed of two layers of FC, ReLU activation function and BN layer. This design enhances the representation ability in the contrastive learning process, so that the model can more effectively extract and compare feature representations between samples.

[0033] In some specific implementations, the global features and local dense features output by the FPM components in each network include and as well as and , the Predictor components in each network generate corresponding global feature representations and local dense feature representation , specifically, including the first global representation in the online network and the first local representation , and the hybrid global representation in the hybrid network and hybrid local representation .

[0034] In some optional embodiments, the slice enhancement view includes a first slice enhancement view and the second slice enhanced view ; The contrast branch and the mixing branch are pre-trained on the slice enhanced view, including: In the comparison branch: Enhance the view with the first slice For the online network The first projection feature is generated and the corresponding online network representation is predicted; wherein the first projection feature includes the first global feature and the first local dense feature , the online network representation includes a first global representation and the first local representation ; Enhance the view with the second slice The second projection feature is input to the target network and outputs the corresponding second projection feature, which is used to provide a regression target for the training of the online network; wherein the second projection feature includes a second global feature and the second local dense feature ; In the hybrid branch: Enhance the view of the first slice Enhanced view with the second slice Blend to get a hybrid slice enhanced view ; Enhance your view with hybrid slices As input to the hybrid network, a hybrid network representation is output; wherein the hybrid network representation includes a hybrid global representation and hybrid local representation ; In each iteration, after determining the first projection feature, the second projection feature and the hybrid network representation, a multi-level contrast loss function is determined based on the online network representation and the second projection feature based on a multi-level contrast learning strategy. , and based on the multi-level contrast loss function Update the online network With the target network network parameters.

[0035] It should be emphasized that in the above embodiment, the corresponding global representation and local representation are obtained through contrast branches and mixing branches in the pre-training stage, and then multi-level contrastive learning is used to enhance the model's ability to capture features of multi-instance problems in the pre-training stage.

[0036] It should be noted that most existing contrastive learning methods focus on self-supervised training based on the global representation of the image, and often ignore the relationship between local representations in multi-semantic images. To this end, this embodiment proposes a new local dense contrastive loss, which performs multi-level contrastive learning by utilizing the spatial information of dense features, see Figure 2. Specifically, multi-level contrastive learning includes global-to-global contrastive loss and local-to-local contrastive loss, where the global contrastive loss comes from global features and the local contrastive loss comes from local dense features.

[0037] Furthermore, the multi-level contrast loss function Including global loss and local density loss , whose expression is:

[0038] Among them, the global loss Including global contrast loss and global hybrid loss , whose expression is:

[0039] In the formula, the global contrast loss Based on the first global representation and the second global feature Determine that the global mixing loss Based on hybrid global representation , the first global feature and the second global feature Sure; The local density loss Including local dense contrast loss and local dense mixing loss , whose expression is:

[0040] In the formula, the local dense contrast loss Based on the first local representation and the second local dense feature Determine that the local dense mixing loss Based on hybrid local representation , the first local dense feature and the second local dense feature .

[0041] It should be noted that the above embodiment performs end-to-end training on the two parallel projection heads of FEM and FPM by optimizing the contrast loss function on the global representation and the local representation.

[0042] Furthermore, the global contrast loss The expression is:

[0043] In the formula, express norm, represents the inner product; The global mixing loss The expression is:

[0044] In the formula, Indicates that by calculating Dimensional Features and The eigenvector generated by the maximum value of the element, ; The local dense contrast loss The expression is:

[0045] The local dense mixing loss The expression is:

[0046] In the formula, Indicates that by calculating Dimensional Features and The eigenvector generated by the maximum value of the elements.

[0047] In the above embodiment, global-global and local-local loss functions are established by learning global features and local dense features. The loss function aims to optimize the model's feature capture capability for images containing a variety of semantic information, and more effectively learn and understand the complex characteristics of the data, thereby achieving high-precision PolSAR image classification with a very small number of labeled samples.

[0048] It should be noted that in order to enable the network model to directly compare the most distinguishing features, in the above embodiment, Dimensional Features and Generate a new eigenvector by computing the maximum value of its elements ,right Dimensional Features and A new eigenvector is generated by calculating the maximum value of its elements .

[0049] It should be emphasized that in the above embodiment, by simultaneously learning the representation of global features and local dense features, the classification accuracy of the downstream PolSAR image task can be significantly improved.

[0050] It is also important to emphasize that in the above embodiments, a new contrastive learning paradigm is designed to perform dense contrastive learning on local representations, thereby comprehensively learning instance representations at the global and local levels. This multi-level contrastive learning strategy can maximize the use of filtered instance information and mutually enhance information learning from multiple perspectives.

[0051] Furthermore, the online network With the target network The network parameter update process is as follows:

[0052] In the formula, representing online network parameters about the online network; Represents the optimizer; Indicates batch; represents the learning rate; represents a target network parameter about the target network, and Online network parameters EMA (Exponential Moving Average); Indicates the decay rate of EMA.

[0053] Furthermore, the adopting of a random enhancement strategy to preprocess the image slice to obtain a slice enhanced view includes: Two groups of random reinforcement strategies are used and , respectively, slice the image Make the first ratio and the second ratio The random cropping with probability Perform random horizontal flipping to obtain the initial enhanced view; Adjusting the initial augmented view to the image slice The original size of the slice is obtained, and the number of channels C remains unchanged in the whole process, and the slice enhancement view is obtained, including the first slice enhancement view (also called the first random augmented view) and the second slice augmented view (Also called the second random augmented view).

[0054] In some specific implementations, in the random enhancement strategy, the first ratio , the second ratio .

[0055] In some optional embodiments, the classification branch includes a FEM component and a Classification component; wherein the Classification component is responsible for capturing global features, and the goal is to predict the category to which the central pixel of the image slice belongs; The step of fine-tuning the classification branch on the image slice based on small sample classification includes: Initializing the FEM components in the classification branch using the pre-trained network parameters corresponding to the FEM components in the online network; Establish a small sample classification training set based on the small sample image slices with labels, make the classification branch perform fine-tuning training on the small sample classification training set, and output the predicted category of the corresponding central pixel point through the FEM component and the Classification component in turn; The network parameters of the classification branch are updated based on the cross entropy loss between the predicted category and the label.

[0056] As a non-limiting example, as shown in FIG6 , the Classification component sequentially includes a global average pooling layer, a FC layer, a BN layer, a ReLU layer, and another FC layer.

[0057] PolSAR image classification is a pixel-level task, so the spatial information of dense features may interfere with the prediction of sample categories. Therefore, in the small sample classification stage, the network structure includes FEM components and Classification components. In the classification branch, first, a small number of labeled samples, under the guidance of the weight information obtained in the pre-training stage, work together with the FEM component to obtain high-quality feature representation; secondly, the Classification component is responsible for capturing global features and predicting the category of sample pixels; finally, the model is optimized at this stage by calculating the cross entropy loss between the label and the predicted category.

[0058] It should be noted that the goal of the Classification component is to predict the category to which the central pixel of the image slice belongs, that is, only the global features of the image slice need to be retained. The image spatial features extracted by the FEM component in the classification branch are input into the Classification component, and the high-dimensional feature map is converted into a low-dimensional global feature through global average pooling, and then the global features are finally predicted through FC, ReLU activation function and BN layer.

[0059] It should also be noted that, on a small number of unlabeled samples, after the pre-training stage, the model will have the ability to aggregate features of similar instances and generate a pre-trained weight file of the model, which can be used to initialize the FEM component in the classification branch; in the Few-Shot Classification stage, a very small number of image slices that have not undergone data preprocessing are input into the FEM component, and the model is fine-tuned in conjunction with the Classification component. Through fine-tuning of a small number of samples, the model can quickly update and adapt to semantic information of different categories, which makes this embodiment particularly suitable for real-time monitoring tasks, such as agricultural dynamic monitoring, disaster warning and other scenarios.

[0060] In some specific implementations, after the DBCL-3DDC network completes fine-tuning and updating, its classification branch (ie, the network model consisting of the FEM component and the Classification component) is applied to perform the classification task of the entire PolSAR image.

[0061] Exemplarily, the method described in the above embodiment is applied to realize the application of precision agriculture, such as crop growth monitoring, through the classification and dynamic monitoring of crops.

[0062] Exemplarily, the method described in the above embodiment is applied to monitor changes in land cover types and assess the impact of environmental changes on the ecosystem.

[0063] Exemplarily, the method described in the above embodiment is applied to provide support for observation and planning through object classification and change detection.

[0064] Example 2 This embodiment conducts a comparative experiment based on the Flevoland 1989 dataset (L-band). 5% of unlabeled samples and 0.2% of labeled samples are randomly selected from the constructed image dataset according to a certain ratio. These samples are used as the pre-training dataset and small sample classification training set of the DBCL-3DDC method proposed in Example 1, respectively.

[0065] Flevoland 1989 dataset (L-band): This dataset is a publicly available fully polarized airborne SAR data from NASA / JPL laboratory. It was collected in 1989 and the study area is located in the Flevoland area of ​​the Netherlands. The image size is There are 15 different ground object categories, namely Stem Beans, Peas, Rapeseed, Beet, Forest, Lucerne, Wheat 1, Bare Soil, Grass, Water, Barley, Wheat 2, Wheat 3, Potatoes and Buildings. This image is widely used for performance verification of PolSAR image classification methods.

[0066] During the experiment, data processing and model construction are as follows: 1) Data processing: The input data is PolSAR image, and a sliding window is used to construct image slices of [16×16×number of channels] corresponding to each pixel.

[0067] 2) Model training (pre-training stage): The dense guided contrastive learning (DBCL) framework is used to train the pre-training dataset.

[0068] 3) Model fine-tuning (small sample classification stage): FEM and classification modules are used to fine-tune the small sample classification training set.

[0069] 4) Model testing: During the testing phase, the entire image data is classified to evaluate the classification accuracy of the model.

[0070] The existing technologies used for comparison are: (a) CV-CNN (complex-valued convolutional neural network), (b) CV-CNN-SE (two-dimensional complex-valued network), (c) RV-CNN (real convolutional neural network), (d) SSPRL (self-supervised PolSAR representation learning), (e) SDF2Net (shallow to deep feature fusion network), (f) HybridCVNet (hybrid complex-valued network) and (g) 3D-CNN (three-dimensional convolutional network).

[0071] The comparative experimental results are shown in Table 1 and Figure 7.

[0072] Table 1 Classification results of different methods on the Flevoland 1989 dataset

[0073] In Figure 7, (a) shows the visual classification results of the CV-CNN method, (b) shows the visual classification results of the CV-CNN-SE method, (c) shows the visual classification results of the RV-CNN method, (d) shows the visual classification results of the SSPRL method, (e) shows the visual classification results of the SDF2Net method, (f) shows the visual classification results of the HybridCVNet method, (g) shows the visual classification results of the 3D-CNN method, and (h) shows the visual classification results of the DBCL-3DDC (Ours) method. Among them, the blue rectangle highlights the area with dense species, the red rectangle highlights the area with balanced number of adjacent species, and the white rectangle emphasizes the area with unbalanced number of adjacent species.

[0074] Combining Table 1 and Figure 7, it can be seen that the DBCL-3DDC method proposed in Example 1 can achieve high-precision classification (97.29%) with a very small number of labeled samples (0.2%), which is significantly better than the existing deep learning method. The classification accuracy under small sample conditions is improved by about 15%, which is due to the fact that DBCL can perform dense contrast learning on local representations, thereby comprehensively learning instance representations at the global and local levels. In addition, due to the introduction of local dense features, multi-level contrast learning and 3DDC, the model can effectively extract the key features of PolSAR images, heuristically perform dense classification tasks on multiple instance PolSAR images, and has stronger small sample learning capabilities, which significantly reduces the time and cost of remote sensing data annotation, and is particularly suitable for the PolSAR field where labeled samples are scarce.

[0075] It can be understood that the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described again here.

[0076] Example 3 This embodiment provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor, so that the processor executes part or all of the steps of the method provided in Embodiment 1 or 2 of the present application.

[0077] It is understood that the storage medium may be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0078] Exemplarily, the processor may be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA).

[0079] Exemplarily, the read-only memory includes but is not limited to MASK ROM, PROM, EPROM, EEPROM, Flash, etc.

[0080] Exemplarily, the random access memory includes but is not limited to DRAM, SRAM, SDRAM, DDR SDRAM, etc.

[0081] In some examples, a computer program product is provided, which can be implemented in hardware, software or a combination thereof. As a non-limiting example, the computer program product can be embodied as the storage medium, or as a software product, such as an SDK (Software Development Kit).

[0082] As a non-limiting example, a computer program product is provided, the computer program product includes a computer program or a computer executable instruction, the computer program or the computer executable instruction is stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs some or all steps of the method described in the embodiment of the present application.

[0083] In some examples, a computer program is provided, comprising a computer-readable code, and when the computer-readable code is run in a computer device, a processor in the computer device executes a part or all of the steps in the method.

[0084] This embodiment also proposes an electronic device, including a memory and a processor, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and when the processor executes the at least one instruction, at least one program, a code set or an instruction set, it implements part or all of the steps of the method described in Example 1 or 2.

[0085] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory and a communication interface; wherein the processor generally controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers through a network; the memory is configured to store instructions and applications executable by the processor, and can also cache data to be processed or processed by the processor and various modules in the electronic device (including but not limited to image data, audio data, voice communication data and video communication data), which can be implemented by flash memory (FLASH), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or random access memory (RAM).

[0086] The processor may include one or more processing elements. Thus, the processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. In addition, each integrated circuit may include circuits (e.g., a first circuit, a second circuit, and other circuits, etc.) configured to perform the functions of the processor.

[0087] Furthermore, data may be transmitted between the processor, the communication interface and the memory via a bus, which may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together.

[0088] It can be understood that the options in the above-mentioned embodiment 1 or 2 are also applicable to this embodiment, so they will not be described again here.

[0089] The same or similar reference numerals correspond to the same or similar components; The terms used to describe the positional relationship in the drawings are only used for illustrative purposes and should not be construed as limiting the present application; It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0090] In different specific implementations, the method or system described in the present application can be implemented in software, hardware or a combination thereof. In addition, the order of the steps of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0091] Obviously, the above-mentioned embodiments of the present application are merely examples for clearly illustrating the present application, and are not intended to limit the implementation methods of the present application, and are not intended to limit the present application. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. Each discrete structure / function module or unit can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part, and the structure and function of the discrete components can be implemented as a combined structure or component. It is not necessary and impossible to enumerate all the implementation methods here. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A small sample PolSAR image classification method based on DBCL-3DDC, characterized in that: include: Slicing the pixels in the PolSAR image to obtain image slices as input of a DBCL-3DDC network; wherein the DBCL-3DDC network includes a comparison branch, a mixing branch, and a classification branch; In the comparison branch and the hybrid branch, a random enhancement strategy is adopted to preprocess the image slice to obtain a slice enhancement view, the comparison branch and the hybrid branch are pre-trained on the slice enhancement view, and the network parameters of the comparison branch and the hybrid branch are updated based on a multi-level contrast learning update strategy; wherein the comparison branch includes an online network and a target network, and the hybrid branch includes a hybrid network with the same structure as the online network and shared network parameters; Allowing the classification branch to perform fine-tuning based on small sample classification on the image slice, and updating the network parameters of the classification branch; The updated DBCL-3DDC network is used for the classification task of the entire PolSAR image.

2. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 1 is characterized in that: The online network It includes FEM component for extracting dense features based on 3DDC, FPM component for feature modeling and Predictor component for prediction; The hybrid network includes a FEM component, a FPM component and a Predictor component; The target network Includes FEM components and FPM components.

3. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 2 is characterized in that: The FEM component includes three connected convolution blocks, and an attention mechanism is also provided between adjacent convolution blocks as a bypass; wherein the convolution block includes a 3DDC layer, a BN layer, a ReLU layer and a maximum pooling layer in sequence; and / or, The FPM component includes a parallel global feature projection sub-branch for outputting global features and a dense feature projection sub-branch for outputting local dense features; wherein the global feature projection sub-branch includes a global average pooling layer, an FC layer, a BN layer, a ReLU layer and an FC layer in sequence, and the dense feature projection sub-branch includes a 3DDC layer, a BN layer, a ReLU layer, a 3DDC layer and an FC layer in sequence.

4. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 3 is characterized in that: The 3DDC layer is implemented by conditional parameterized convolution, whose convolution kernel is parameterized as a linear combination of four experts, expressed as follows: In the formula, represents the activation function; each represents a scalar weight associated with the input, computed by a routing function with learned parameters: 。 5. A small sample PolSAR image classification method based on DBCL-3DDC according to any one of claims 2 to 4, characterized in that: The slice enhancement view includes a first slice enhancement view and the second slice enhanced view ; The contrast branch and the mixing branch are pre-trained on the slice enhanced view, including: In the comparison branch: Enhance the view with the first slice For the online network The first projection feature is generated and the corresponding online network representation is predicted; wherein the first projection feature includes the first global feature and the first local dense feature , the online network representation includes a first global representation and the first local representation ; Enhance the view with the second slice The second projection feature is input to the target network and outputs the corresponding second projection feature, which is used to provide a regression target for the training of the online network; wherein the second projection feature includes a second global feature and the second local dense feature ; In the hybrid branch: Enhance the view of the first slice Enhanced view with the second slice Blend to get a blended slice enhanced view ; Enhance your view with hybrid slices As input to the hybrid network, a hybrid network representation is output; wherein the hybrid network representation includes a hybrid global representation and hybrid local representation ; In each iteration, after determining the first projection feature, the second projection feature and the hybrid network representation, a multi-level contrast loss function is determined based on the online network representation and the second projection feature based on a multi-level contrast learning strategy. , and based on the multi-level contrast loss function Update the online network With the target network network parameters.

6. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 5 is characterized in that: The online network With the target network The network parameter update process is as follows: In the formula, representing online network parameters about the online network; Represents the optimizer; Indicates batch; represents the learning rate; represents a target network parameter about the target network, and Online network parameters Exponential Moving Average EMA; Indicates the decay rate of EMA.

7. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 5 is characterized in that: The multi-level contrast loss function Including global loss and local density loss , whose expression is: Among them, the global loss Including global contrast loss and global hybrid loss , whose expression is: In the formula, the global contrast loss Based on the first global representation and the second global feature Determine that the global mixing loss Based on hybrid global representation , the first global feature and the second global feature Sure; The local density loss Including local dense contrast loss and local dense mixing loss , whose expression is: In the formula, the local dense contrast loss Based on the first local representation and the second local dense feature Determine that the local dense mixing loss Based on hybrid local representation , the first local dense feature and the second local dense feature .

8. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 7 is characterized by: The global contrast loss The expression is: In the formula, express norm, represents the inner product; The global mixing loss The expression is: In the formula, Indicates that by calculating Dimensional Features and The eigenvector generated by the maximum value of the element, ; The local dense contrast loss The expression is: The local dense mixing loss The expression is: In the formula, Indicates that by calculating Dimensional Features and The eigenvector generated by the maximum value of the elements.

9. The small sample PolSAR image classification method based on DBCL-3DDC according to claim 5, characterized in that: The step of adopting a random enhancement strategy to preprocess the image slice to obtain a slice enhancement view includes: Two groups of random reinforcement strategies are used and , respectively, slice the image Make the first ratio and the second ratio The random cropping with probability Perform random horizontal flipping to obtain the initial enhanced view; Adjusting the initial augmented view to the image slice The original size of the slice is obtained, and the number of channels C remains unchanged in the whole process, and the slice enhancement view is obtained, including the first slice enhancement view and the second slice enhanced view .

10. A small sample PolSAR image classification method based on DBCL-3DDC according to any one of claims 2 to 4, characterized in that: The classification branch includes a FEM component and a Classification component; wherein the Classification component is responsible for capturing global features, and the goal is to predict the category to which the central pixel of the image slice belongs; The step of fine-tuning the classification branch on the image slice based on small sample classification includes: Initializing the FEM components in the classification branch using the pre-trained network parameters corresponding to the FEM components in the online network; Establish a small sample classification training set based on the small sample image slices with labels, make the classification branch perform fine-tuning training on the small sample classification training set, and output the predicted category of the corresponding central pixel point through the FEM component and the Classification component in turn; The network parameters of the classification branch are updated based on the cross entropy loss between the predicted category and the label.

Images