Cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment

Abstract

The present invention discloses a cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment. It belongs to the field of remote sensing image processing, and the steps are: generating a rotation invariant HOG image according to source domain sample data and target domain sample data; inputting the original image and the rotation invariant HOG image into a convolutional neural network as input to obtain rotation invariant high-level semantic features of the source domain and target domain images; calculating the subclass centers of the source domain and target domain samples according to the high-level semantic features, calculating the moving direction of the target domain image and moving the target domain features according to the neighborhood subclass center alignment method; training an SVM classifier using the source domain features and the moved target domain features, and inputting the unknown target domain image into the classifier for testing to obtain the final scene classification result. The present invention can effectively improve the model's adaptability to spectral shifts caused by cross-domain scene image angle changes and intra-class diversity, and provide a new approach to improving the accuracy of cross-domain remote sensing image scene classification.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing, and in particular relates to a cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment. Background Art

[0002] With the rapid launch of high-resolution remote sensing satellites, optical high-resolution remote sensing imagery has become one of the most important data sources for land cover classification. Remote sensing image scene classification leverages the rich spatial detail of high-resolution remote sensing imagery to capture high-level semantic information about land cover types, making it a research hotspot in remote sensing image scene understanding. Existing remote sensing image scene classification methods typically rely on abundant training samples to achieve land cover classification results. However, the training sample annotation process is often labor-intensive and resource-intensive, and ground-truth samples are sometimes difficult to obtain. Consequently, scholars both domestically and internationally have begun to explore the problem of cross-domain scene classification, leveraging existing sample data from other regions to perform scene classification on target domain images, where training samples are difficult to obtain. However, due to significant differences in scale, spectrum, and spatial distribution between target and source domain images, significant spectral shifts occur between the source and target domains, resulting in suboptimal scene classification accuracy. Furthermore, varying sensor angles and intra-class variations in images can exacerbate spectral shifts in cross-domain images, further complicating cross-domain remote sensing image scene classification. Summary of the Invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment. In the case of sample missing, the feature distribution difference between the source domain and target domain images is reduced through rotation invariant HOG images and cross-domain subclass alignment, so as to achieve high-precision cross-domain remote sensing image scene classification, which can effectively improve the model's adaptability to cross-domain unknown complex scene images.

[0004] Technical Solution: The cross-domain remote sensing image scene classification method based on rotation-robust feature subclass center alignment described in the present invention. By proposing high-level semantic rotation-invariant features of remote sensing images and a method for spatial alignment of neighborhood subclass center features, this method can effectively improve the model's adaptability to angular changes and spectral shifts in cross-domain unknown target domain scene images, and achieve high-precision remote sensing image scene classification in the absence of target domain samples. The specific steps are as follows:

[0005] Step 1: Input source domain sample and target domain sample image, respectively use rotation invariant HOG operator to extract rotation invariant features of source domain and target domain sample images, and generate rotation invariant HOG feature images;

[0006] Step 2: For all source and target domain images, create image pairs of original images and rotation-invariant HOG feature images, and feed them into the convolutional neural network ResNet101 as input to obtain high-level semantic rotation-invariant features of the source and target domain images.

[0007] Step 3: Calculate the subclass center of the source domain and target domain high-level semantic rotation invariant features, use the neighborhood subclass center alignment method to calculate the moving direction of the target domain image features and move the target domain features;

[0008] Step 4: Use the source domain sample features and the moved target domain sample features to train an SVM classifier. Input the target domain image of unknown category into the aforementioned SVM classifier to obtain the scene classification result of the target domain image.

[0009] Beneficial effects: Compared with the prior art, the present invention is characterized in that: the present invention addresses the spectral offset problem of source domain and target domain images caused by different shooting angles and intra-class diversity in cross-domain scene classification, and can effectively improve the classification accuracy of the cross-domain scene classification algorithm in the case of missing samples by designing high-level semantic features with rotation invariance and shortening the feature distance between the source domain and the target domain in the feature space. The ability of features to express image rotation changes can be improved by designing high-level semantic features with rotation invariance. Considering the impact of intra-class spectral differences on the distribution deviation of source domain and target domain features, a neighborhood subclass center alignment method is proposed to move the target domain features in the direction of their corresponding source domain features, which can effectively reduce the distribution difference between the source domain and target domain features, improve the ability of source domain features to describe target domain images, and provide support for applications such as land use surveys, national geographic surveys, and natural resource monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is an operational flow chart of the present invention;

[0011] Figure 2 It is the rotation-invariant HOG feature map of different color spaces in the present invention;

[0012] Figure 3 is a schematic diagram of the UC Merced dataset sample used in this invention;

[0013] Figure 4 3 is a feature space distribution diagram before and after optimization by the neighborhood subclass center alignment method in an embodiment of the present invention; wherein, (a) is the feature distribution diagram before optimization, and (b) is the feature distribution diagram after optimization by the neighborhood subclass center alignment method. DETAILED DESCRIPTION

[0014] In order to more clearly illustrate the technical solution of the present invention, the technical solution of the present invention is further described in detail below with reference to the accompanying drawings:

[0015] like Figure 1-4 As shown, the present invention describes a cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment. First, the method generates a rotation invariant HOG image based on source domain sample data and target domain sample data; then, the original image and the rotation invariant HOG image are input into the convolutional neural network as input to obtain the rotation invariant high-level semantic features of the source domain and target domain images; then, the subclass centers of the source domain and target domain samples are calculated based on the high-level semantic features, and the moving direction of the target domain image is calculated based on the neighborhood subclass center alignment method and the target domain features are moved until the moving convergence threshold δ is met; finally, the source domain features and the moved target domain features are used to train the SVM classifier to establish a classification model, and the target domain image is input for testing to obtain the final scene classification result. Specifically, Figure 1 As shown, the specific steps are as follows:

[0016] (1) Input source domain sample and target domain sample images to generate rotation-invariant HOG feature images;

[0017] (2) The original image and the image with rotation-invariant HOG features are fed into the convolutional neural network as input to obtain high-level semantic rotation-invariant features of the source and target domain images;

[0018] (3) Calculate the subclass center of the high-level semantic rotation-invariant features of the source domain and the target domain, use the neighborhood subclass center alignment method to calculate the moving direction of the target domain image features and move the target domain features;

[0019] (4) Using the source domain sample features and the moved target domain sample features, the SVM classifier is trained. The target domain image of unknown category is input into the aforementioned SVM classifier to obtain the scene classification result of the target domain image.

[0020] Furthermore, the specific operation process of step (1) is:

[0021] (1.1) Calculate the gradient of the source domain and target domain sample images; use the gradient operators [-1,0,1]] and [-1,0,1] T Perform convolution operations on the image respectively, calculate the horizontal and vertical gradients of the image, and obtain the gradient image D in the X and Y directions x , D y ;

[0022] (1.2) According to the following formula, the Fourier HOG gradient image is generated for the original image of the source domain and target domain samples and the gradient image in the X and Y directions Among them, K1 is the convolution kernel of the spatial set, K2 is the local regularization convolution kernel; Φ(D(x)) is the gradient direction, and D represents the gradient domain image; is the Fourier transform; e is a natural constant, approximately 2.71828, ||D(x)|| is the gradient field; im represents the imaginary part of the Fourier basis;

[0023]

[0024]

[0025] (1.3) According to the following formula, the gradient image regenerated in step (1.2) Calculate the basic function U using the basis function convolution kernel with circular consistency j,k and regional feature image B i ;

[0026]

[0027]

[0028] Where r represents the distance in the polar coordinate system; represents the angle in the polar coordinate system; δ represents the impulse function; r j Indicates the polar coordinate distance corresponding to (j,k); represents the Fourier basis;

[0029] (1.4) Generate the final rotation invariant feature map;

[0030]

[0031] U j1，k1 represents the above Uj,k calculated with (j1,k1) as parameters; represents the Fourier HOG gradient image calculated with integer m1; U j2，k2 Denote that (j2, k2) is the above Uj,k calculated by parameters; Represents the Fourier HOG gradient image calculated by the integer m2; k1 represents an arbitrary positive integer; m1 represents an integer; k2 represents an arbitrary positive integer; m2 represents an integer.

[0032] Furthermore, the specific operation process of step (2) is:

[0033] (2.1) Create an image pair by combining the original sample image and the rotation-invariant HOG feature image;

[0034] (2.2) The training sample image pairs are simultaneously fed into the convolutional neural network to generate high-level semantic convolutional features with rotation invariance.

[0035] Furthermore, the specific operation process of step (3) is:

[0036] (3.1) Assuming that there are C types of land cover, according to the high-level semantic features X of the source domain samples extracted above, s The Kmeans method is used to divide each category into k subcategories, forming a total of k×C subcategories. The subcategory labels can be expressed as Ω=[Ω 11 ,...,Ω 1k ,...,Ω C1 ,...,Ω Ck ];

[0037] (3.2) Using the source domain sample feature X s and a small number of target domain sample features X t Train the SVM classifier;

[0038] (3.3) When the number of iterations l is 1, obtain the predicted labels of k×C subcategories obtained by the aforementioned SVM classifier

[0039] (3.4), according to the true value label Ω and the predicted label Calculate the source domain subclass center U s and target domain subclass center

[0040] (3.5) Calculate the moving direction d of each subclass based on the sample data with the target domain label ij ; and Represent the mean and number of samples of the source domain label belonging to the i-th category and the j-th subcategory, and Respectively represent the mean and number of samples of the target domain predicted to belong to the jth subclass of the i-th category;

[0041]

[0042]

[0043]

[0044] (3.6), each target domain feature Find M nearest neighbor points and calculate the target domain features The final moving direction d:

[0045]

[0046] (3.7) According to the characteristics of the target domain and the corresponding moving direction to obtain new target domain features

[0047]

[0048] (3.8) Use the classifier trained in step (3.2) to classify the target domain features Update the number of iterations l = l + 1 to obtain the new predicted label Y t l ;

[0049] (3.9) Repeat steps (3.2-3.8) until the predicted label Y t l Convergence, stop iteration.

[0050] Furthermore, the specific operation process of step (4) is:

[0051] For the cross-domain scene classification model obtained in step (3), the target domain image is obtained to obtain the unknown target domain label, where the target domain dataset is an example of the UC Merced dataset. Figure 3 As shown in Table 1, the classification results of this patented method compared with other representative methods on the UCMerced dataset are shown.

[0052] Table 1 Comparison of classification accuracy of different methods on the UC Merced dataset

[0053]

[0054] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment, characterized by: By proposing a method for aligning high-level semantic rotation-invariant features of remote sensing images and neighborhood subclass center feature space, the model's adaptability to cross-domain unknown target domain scene image angle changes and spectral shifts is improved, achieving high-precision remote sensing image scene classification in the absence of target domain samples. The specific steps are as follows: Step (1) input source domain sample and target domain sample image to generate rotation invariant HOG feature image; Step (2) The original image and the rotation-invariant HOG feature image are fed into a convolutional neural network as input to obtain high-level semantic rotation-invariant features of the source and target domain images. Step (3) Calculate the subclass center of the source domain and target domain high-level semantic rotation invariant features, use the neighborhood subclass center alignment method to calculate the moving direction of the target domain image features and move the target domain features; The specific operation process: (3.1) Assuming that there are C types of land cover, according to the high-level semantic rotation-invariant features X of the source domain samples extracted above s The Kmeans method is used to divide each category into k subcategories, forming a total of k×C subcategories. The subcategory labels can be expressed as Ω=[Ω 11 ,...,Ω 1k ,...,Ω C1 ,...,Ω Ck ]; (3.2) Using source domain sample features X s and a small number of target domain sample features X t Train the SVM classifier; (3.3) When the number of iterations l is 1, obtain the predicted labels Y of k×C subcategories obtained by the above SVM classifier t l ; (3.4) According to the true value label Ω and the predicted label Y t l , calculate the source domain subclass center U s and target domain subclass center (3.5) Calculate the moving direction d of each subclass based on the sample data with the target domain label ij ; and Represent the mean and number of samples of the source domain label belonging to the i-th category and the j-th subcategory, and Respectively represent the mean and number of samples of the target domain predicted to belong to the jth subclass of the i-th category; (3.6) Each target domain feature Find M nearest neighbor points and calculate the target domain features The final moving direction d: (3.7) According to the characteristics of the target domain and the corresponding moving direction to obtain new target domain features (3.8) Use the classifier trained in step (3.2) to classify the target domain features Update the number of iterations l = l + 1 to obtain the new predicted label Y t l ; (3.9) Repeat steps (3.2-3.8) until the predicted label Y t l Convergence, stop iteration; Step (4) uses the source domain sample features and the moved target domain sample features to train an SVM classifier; inputs the target domain image of unknown category into the aforementioned SVM classifier to obtain the scene classification result of the target domain image.

2. The cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment according to claim 1 is characterized in that: The operation process of step (1) is: extracting rotation-invariant HOG features from the source domain and target domain training samples respectively, and generating corresponding rotation-invariant HOG feature images; specifically, the following steps are performed: (1.1) Calculate the gradient of the source domain and target domain sample images; use the gradient operators [-1,0,1] and [-1,0,1] T Perform convolution operations on the image respectively, calculate the horizontal and vertical gradients of the image, and obtain the gradient image D in the X and Y directions x , D y ; (1.2) According to the following formula, the Fourier HOG gradient image is generated for the original image of the source domain and target domain samples and the gradient image in the X and Y directions Among them, K1 is the convolution kernel of the spatial set, K2 is the local regularization convolution kernel; Φ(D(x)) is the gradient direction, and D represents the gradient domain image; (1.3) According to the following formula, the gradient image regenerated in step (1.2) Calculate the basic function U using the basis function convolution kernel with circular consistency j,k and regional feature image B i ; (14) Generate the final rotation invariant feature map; 3. The cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment according to claim 1 is characterized by: The specific operation process of step (2) is: (2.1) The original image of the training sample and the rotation-invariant HOG feature image are combined into an image pair and input into the convolutional neural network for feature extraction; (2.2) The training sample image pairs are simultaneously fed into the convolutional neural network to obtain high-level semantic rotation-invariant features of the source and target domain images.

4. The cross-domain remote sensing image scene classification method based on rotation robust feature subclass center alignment according to claim 1 is characterized by: The specific process of step (4) is: for the cross-domain scene classification model obtained in step (3), finally obtaining the target domain image to obtain the unknown target domain label.

Images