Remote sensing image classification and recognition method and device, computer device and storage medium

Abstract

The application relates to a remote sensing image classification and recognition method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a remote sensing image to be classified and recognized, and extracting deep feature information of the remote sensing image; determining a feature channel corresponding to the deep feature information and a channel weight of the feature channel; obtaining a channel feature according to the channel weight of the feature channel and the deep feature information of the feature channel; performing feature aggregation processing on the channel feature to obtain a classification and recognition feature; inputting the classification and recognition feature into a pre-trained remote sensing image classification model to obtain an image content classification and recognition result of the remote sensing image through the remote sensing image classification model. The method can improve the accuracy of image content classification and recognition of the remote sensing image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a remote sensing image classification and recognition method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the continuous development of science and technology, remote sensing images, as an important carrier of geomorphological information visualization, have been efficiently applied in various environments. Remote sensing images are mainly composed of dozens or hundreds of data points from the same scene or the same spectral channel. Pixels with the same spectral dimension are fused to form a remote sensing image, which contains the magnitude and characteristics of electromagnetic waves from various ground features. Because remote sensing images contain a wide variety of features, they are easily affected by noise, leading to significant errors in image classification.

[0003] In existing technologies, remote sensing image classification methods based on multi-level cross-layer bilinear fusion establish multi-level feature extraction models using data collected on-site, calculate feature information at each regional scale in the image, and classify according to the hierarchical arrangement of pixel features in the model. Remote sensing image classification methods based on graph convolutional networks establish neural network image convolutional models, use image brightness as the division scale to initially divide the image into bright and dark regions, design a differentiation model, and further divide the image regions according to the relationship between brightness and darkness.

[0004] However, the applicant discovered during the implementation process that the existing technology has a problem with large classification errors. Summary of the Invention

[0005] Therefore, it is necessary to provide a remote sensing image classification and recognition method, apparatus, computer equipment, and storage medium that can improve the classification and recognition of remote sensing image content, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a remote sensing image classification and recognition method, the method comprising:

[0007] Acquire the remote sensing image to be classified and identified, and extract the depth feature information of the remote sensing image;

[0008] Determine the feature channels corresponding to the depth feature information, and the channel weights of the feature channels;

[0009] Channel features are obtained based on the channel weights and depth feature information of the feature channels;

[0010] The channel features are aggregated to obtain classification and recognition features;

[0011] The classification and recognition features are input into a pre-trained remote sensing image classification model, and the image content classification and recognition results of the remote sensing image are obtained through the remote sensing image classification model.

[0012] In one embodiment, the remote sensing image classification model includes a classifier and a generator;

[0013] The classification and recognition features are input into a pre-trained remote sensing image classification model. The model then yields the image content classification and recognition results for the remote sensing image, including:

[0014] The classification and recognition features are input into a pre-trained remote sensing image classification model, and the image content classification and recognition results of the remote sensing image are obtained through the classifier.

[0015] In one embodiment, the remote sensing image classification model is generated in the following ways:

[0016] Obtain sample classification and recognition features; the sample classification and recognition features include first sample classification and recognition features and second sample classification and recognition features. The first sample classification and recognition features are sample classification and recognition features that carry image content labels, and the second sample classification and recognition features are sample classification and recognition features that do not carry image content labels.

[0017] Obtain a random vector for the sample classification and recognition features, and input the random vector into the generator to obtain pseudo-sample classification and recognition features;

[0018] The first sample classification and recognition features, the second sample classification and recognition features, and the pseudo sample classification and recognition features are input into the classifier. The classifier obtains the sample classification and recognition feature recognition results and the image content classification prediction results. The image content classification prediction results include the image content classification prediction results corresponding to the first sample classification and recognition features.

[0019] The first loss value is determined based on the sample classification and recognition feature recognition result, the sample classification and recognition feature, and the pseudo sample classification and recognition feature; and the second loss value is determined based on the image content label of the first sample classification and recognition feature and the image content classification prediction result corresponding to the first sample classification and recognition feature.

[0020] The total loss value of the classifier is determined by the first loss value and the second loss value, and the classifier is trained using the total loss value to obtain the trained remote sensing image classification model.

[0021] In one embodiment, determining the feature channels corresponding to the depth feature information and the channel weights of the feature channels includes:

[0022] Obtain the average weight and the maximum weight of the feature channel;

[0023] The channel weights of the feature channels are determined based on the depth feature information, the average weight, and the maximum weight.

[0024] In one embodiment, the channel features are subjected to feature aggregation processing to obtain classification and recognition features, including:

[0025] Obtain multiple feature aggregation parameters for channel features;

[0026] Based on multiple feature aggregation parameters and channel features, classification and recognition features are obtained.

[0027] In one embodiment, before extracting depth feature information from the remote sensing image, the process includes:

[0028] Acquire high-frequency and low-frequency information from remote sensing images;

[0029] Based on high-frequency and low-frequency information, the remote sensing image is denoised to obtain a denoised remote sensing image.

[0030] Extracting depth feature information from remote sensing images, including:

[0031] Extract depth feature information from the denoised remote sensing image.

[0032] In one embodiment, the remote sensing image is denoised based on high-frequency and low-frequency information to obtain a denoised remote sensing image, including:

[0033] Obtain the residual mapping relationship pre-constructed for high-frequency information;

[0034] Based on high-frequency information, low-frequency information, and residual mapping relationships, a loss function for pixels in remote sensing images is obtained.

[0035] Based on the pixel loss function, high-frequency information, and low-frequency information, a denoised remote sensing image is obtained from the remote sensing image.

[0036] Secondly, this application also provides a remote sensing image classification and recognition device, the device comprising:

[0037] The depth feature extraction module is used to acquire the remote sensing image to be classified and identified, and extract the depth feature information of the remote sensing image;

[0038] The channel weight determination module is used to determine the feature channels corresponding to the depth feature information, as well as the channel weights of the feature channels;

[0039] The channel feature acquisition module is used to obtain channel features based on the channel weights and depth feature information of the feature channels.

[0040] The classification feature determination module is used to perform feature aggregation processing on channel features to obtain classification recognition features;

[0041] The recognition result acquisition module is used to input the classification and recognition features into a pre-trained remote sensing image classification model, and obtain the image content classification and recognition results of the remote sensing image through the remote sensing image classification model.

[0042] Thirdly, this application also provides a computer device. This computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0043] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0044] The aforementioned remote sensing image classification and recognition method, apparatus, computer equipment, and storage medium acquire a remote sensing image to be classified and extract its depth feature information; determine the feature channels corresponding to the depth feature information and the channel weights of the feature channels; obtain channel features based on the channel weights and depth feature information of the feature channels; perform feature aggregation processing on the channel features to obtain classification and recognition features; input the classification and recognition features into a pre-trained remote sensing image classification model, and obtain the image content classification and recognition result of the remote sensing image through the remote sensing image classification model. Compared with traditional technologies, this application obtains channel features by processing the depth feature information and the channel weights of the feature channels, and then performs feature aggregation on the channel features to obtain classification and recognition features. By using a remote sensing image classification model and classifying and recognizing the image content of the remote sensing image based on the classification and recognition features, it can effectively and accurately identify the image content of the remote sensing image, reduce the image content classification error of the remote sensing image in the prior art, and thus improve the accuracy of image content classification and recognition of remote sensing images. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a remote sensing image classification and recognition method in one embodiment;

[0046] Figure 2 This is a flowchart illustrating the generation method of a remote sensing image classification model in one embodiment;

[0047] Figure 3 This is a flowchart illustrating the steps for obtaining classification and recognition features in one embodiment;

[0048] Figure 4 This is a flowchart illustrating the steps for obtaining a denoised remote sensing image in one embodiment.

[0049] Figure 5 This is a schematic diagram illustrating the impact of the bilinear fusion method on the classification results in one embodiment;

[0050] Figure 6 This is a schematic diagram illustrating the impact of a convolutional network method on the classification results in one embodiment;

[0051] Figure 7 This is a schematic diagram illustrating the impact of a remote sensing image classification and recognition method on the classification results in one embodiment;

[0052] Figure 8 This is a structural block diagram of a remote sensing image classification and recognition device in one embodiment;

[0053] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1 As shown, a remote sensing image classification and recognition method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0056] S102, acquire the remote sensing image to be classified and identified, and extract the depth feature information of the remote sensing image.

[0057] The remote sensing image to be classified and identified can be a remote sensing image for which image content classification and identification are to be performed. The remote sensing image can be an information carrier of the remote sensing target. Depth feature information refers to the depth value of the image, which can be used as the pixel value of the remote sensing image.

[0058] For example, a remote sensing image to be classified and identified can be acquired. Multiple depth feature information of the remote sensing image is extracted, and each depth feature information is further processed to obtain feature information that is beneficial for remote sensing image classification.

[0059] S104, determine the feature channels corresponding to the depth feature information, and the channel weights of the feature channels.

[0060] Here, feature channels refer to the channel information corresponding to depth feature information. Channel weights can be the weight information corresponding to feature channels.

[0061] For example, the feature channels corresponding to each depth feature information, as well as the channel weights of each feature channel, can be determined. For instance, semantic attention channels can be established according to the feature relationships of different regions in a remote sensing image.

[0062] S106. Based on the channel weights of the feature channels and the depth feature information of the feature channels, the channel features are obtained.

[0063] Among them, channel features can be feature information for feature channels obtained by processing depth feature information, or they can be spatial dimension features.

[0064] For example, the channel weights of the feature channels can be multiplied by the depth feature information of the feature channels to obtain the channel features.

[0065] For example, channel features can be obtained using the following expression:

[0066]

[0067] Where F1 is the channel feature, M s (f 1c ) represents the channel weight, f 2c This refers to depth feature information.

[0068] S108, perform feature aggregation processing on the channel features to obtain classification and recognition features.

[0069] Among them, classification and recognition features can be features used to classify and recognize the image content of remote sensing images.

[0070] For example, feature aggregation can be performed on channel features based on pre-set feature aggregation parameters for the remote sensing image to be classified, thereby obtaining classification and recognition features. The feature aggregation parameters can be obtained by performing multiple convolution processes on the channel features.

[0071] S110: Input the classification and recognition features into the pre-trained remote sensing image classification model, and obtain the image content classification and recognition results of the remote sensing image through the remote sensing image classification model.

[0072] The remote sensing image classification model can be a model used to classify and identify the content of remote sensing images. The image content classification and identification result can be the result obtained after classifying and identifying the image content of the remote sensing image. For example, the image content of the remote sensing image can be classified into multiple image content categories, such as image content A, B, C, and D. By using the remote sensing image classification model to classify and identify the image content A, B, C, and D contained in the remote sensing image, the image content of the remote sensing image can be accurately identified and classified.

[0073] For example, classification features obtained from remote sensing image processing can be input into a pre-trained remote sensing image classification model. The remote sensing image classification model then performs classification and prediction of the image content of the remote sensing image, yielding the image content classification result. This effectively enables the classification and recognition of the image content of remote sensing images.

[0074] In this embodiment, a remote sensing image to be classified is acquired, and its depth feature information is extracted. The feature channels corresponding to the depth feature information and their channel weights are determined. Channel features are obtained based on the channel weights and depth feature information of the feature channels. Feature aggregation processing is performed on the channel features to obtain classification and recognition features. These classification and recognition features are then input into a pre-trained remote sensing image classification model, which provides the image content classification and recognition result for the remote sensing image. Compared with traditional technologies, this application obtains channel features by processing the depth feature information and channel weights, and then aggregates these features to obtain classification and recognition features. By using a remote sensing image classification model and classifying and recognizing the image content based on these classification and recognition features, the image content of the remote sensing image can be effectively and accurately identified. This reduces the image content classification error in existing technologies, thereby improving the accuracy of image content classification and recognition for remote sensing images.

[0075] In one embodiment, the remote sensing image classification model includes a classifier and a generator;

[0076] The classification and recognition features are input into a pre-trained remote sensing image classification model. The model then yields the image content classification and recognition results for the remote sensing image, including:

[0077] The classification and recognition features are input into a pre-trained remote sensing image classification model, and the image content classification and recognition results of the remote sensing image are obtained through the classifier.

[0078] The classifier can be used to classify and recognize the content of remote sensing images. The generator can be used to train the classifier.

[0079] For example, the classification and recognition features obtained from S108 can be input into the classifier of the remote sensing image classification model. The classifier then performs classification and recognition prediction on the image content of the remote sensing image to obtain the image content classification and recognition result. In this way, the image content of the remote sensing image can be effectively classified and recognized.

[0080] In this embodiment, a classifier trained through semi-supervised learning can be used to classify, identify, and predict the image content of remote sensing images, thereby improving the accuracy of image content classification and identification in remote sensing images.

[0081] In one embodiment, such as Figure 2 As shown, the methods for generating remote sensing image classification models include:

[0082] S202, Obtain sample classification and recognition features; the sample classification and recognition features include a first sample classification and recognition feature and a second sample classification and recognition feature, the first sample classification and recognition feature is a sample classification and recognition feature carrying image content labels, and the second sample classification and recognition feature is a sample classification and recognition feature not carrying image content labels.

[0083] The sample classification features can be used to train the remote sensing image classification model. These features are obtained through processing the remote sensing image in steps S102-S108. The first sample classification feature can be a classification feature carrying image content labels, where the image content labels are classification labels for the image content of the remote sensing image. This first sample classification feature with image content labels can be used for supervised training. The second sample classification feature can be a classification feature without image content labels, and this second sample classification feature without image content labels can be used for unsupervised training. Semi-supervised learning training of the remote sensing image classification model can be achieved through the first and second sample classification features.

[0084] For example, a first sample classification feature carrying image content labels can be obtained, and a second sample classification feature without image content labels can be obtained. The image content labels of the first sample classification feature can be manually labeled. By obtaining the sample classification feature, training data can be provided for training the remote sensing image classification model.

[0085] S204: Obtain a random vector for the sample classification and recognition features, and input the random vector into the generator to obtain pseudo-sample classification and recognition features.

[0086] The random vector can be an image vector used to generate features for pseudo-sample classification and recognition.

[0087] For example, a random image vector for sample classification and recognition features can be obtained, and the random image vector can be preset by the user. The random image vector is input into a generator, which can then obtain pseudo-sample classification and recognition features. Furthermore, the classifier can be trained using these pseudo-sample classification and recognition features.

[0088] S206, the first sample classification recognition feature, the second sample classification recognition feature, and the pseudo sample classification recognition feature are input into the classifier, and the sample classification recognition feature recognition result and the image content classification prediction result are obtained through the classifier; the image content classification prediction result includes the image content classification prediction result corresponding to the first sample classification recognition feature.

[0089] Among them, the sample classification feature recognition result refers to the prediction result regarding whether the sample classification features are true. The image content classification prediction result refers to the classification prediction result of the remote sensing image content.

[0090] For example, the first sample classification features, the second sample classification features, and the pseudo-sample classification features can be used as inputs to a classifier. These features are fed into the classifier for training. The classifier identifies the real sample classification features and the pseudo-sample classification features, along with the image content classification prediction results, and is then used to train the classifier. For instance, the classifier can identify whether the first sample classification features, the second sample classification features, and the pseudo-sample classification features are real or pseudo-sample classification features, and obtain the image content classification prediction results for the remote sensing image based on the real classification features, thus facilitating the training of the classifier using these identification results.

[0091] S208, determine the first loss value based on the sample classification recognition feature recognition result, the sample classification recognition feature and the pseudo sample classification recognition feature, and determine the second loss value based on the image content label of the first sample classification recognition feature and the image content classification prediction result corresponding to the first sample classification recognition feature.

[0092] The first loss value can be the loss value determined based on the result of the classifier identifying the authenticity, and the second loss value can be the loss value determined by the image content classification prediction result of the first sample classification feature identified by the classifier and the real image content label of the first sample classification feature.

[0093] For example, a first loss value can be determined based on the predicted recognition result of whether the sample classification recognition feature is real, the sample classification recognition feature, and the pseudo-sample classification recognition feature. For instance, the first loss value can be determined based on the probability that the classifier recognizes the sample classification recognition feature as a real classification recognition feature, and the probability that the classifier recognizes the pseudo-sample classification recognition feature as a pseudo-classification recognition feature.

[0094] The second loss value can be determined based on the real image content label of the first sample classification and recognition features, and the image content classification prediction result corresponding to the first sample classification and recognition features predicted by the classifier.

[0095] Furthermore, the total loss value of the classifier can be determined by the first loss value and the second loss value, so that the classifier can be trained using the total loss value.

[0096] S210: Determine the total loss value of the classifier using the first loss value and the second loss value, and train the classifier using the total loss value to obtain the trained remote sensing image classification model.

[0097] The total loss value can be the loss value used to train the classifier.

[0098] For example, the total loss value of the classifier can be determined by fusing the first loss value and the second loss value. The model parameters of the classifier can be adjusted using the total loss value to train the classifier and obtain a trained classifier, thus obtaining a trained remote sensing image classification model.

[0099] In this embodiment, by determining the first and second loss values ​​of the classifier, and then using these first and second loss values ​​to determine the total loss value of the classifier, the classifier can be trained to obtain a trained remote sensing image classification model. This trained remote sensing image classification model can then be used to predict and classify the content of remote sensing images, improving the accuracy of image content classification and recognition.

[0100] In one specific embodiment, a semi-supervised semi-learning algorithm is used to capture the authenticity of attribute features of different categories of remote sensing images. Labels are added to feature points with strong authenticity, and the values ​​are input into the classification model. The loss function is expanded, and the function outputs real samples and pseudo samples of the target to be classified. The probability value between the sample and a certain type is calculated, and effective classification is achieved by comparing the probability values.

[0101] The generator and discriminator each contain 1024 and 128 units, respectively, with different units representing different image feature types. The generator is expressed as:

[0102] X f =G(α,β)

[0103] In the formula, X f Let G(α, β) represent the generator; G(α, β) represent the probability that sample α belongs to β independent types in the classifier. Based on the pixel characteristics of different regions in multi-label remote sensing images, we define the true region as the region with a large and dense distribution of pixels, and the false region as the region with a small and sparse distribution of pixels. We then calculate the log-likelihood for classifying these two regions, expressed by the following formula:

[0104] L s =E[lg P(S=real|X r )+E[lg P(S=f|X f )]]

[0105] L c =E[lg P(C=c|X r )+E[lg P(C=c|X f )]]

[0106] In the formula, (L s +L c (L) represents the parameters to be maximized during classifier training; s -L c ) represents the maximum parameter for training the generator; X r This represents the classification result of the real area in the remote sensing image; X f This indicates the classification result of false regions in a remote sensing image.

[0107] In one embodiment, determining the feature channels corresponding to the depth feature information and the channel weights of the feature channels includes:

[0108] Obtain the average weight and the maximum weight of the feature channel;

[0109] The channel weights of the feature channels are determined based on the depth feature information, the average weight, and the maximum weight.

[0110] The average weight can be the average of the weights of each depth feature contained in the feature channel. The maximum weight can be the largest weight value corresponding to the depth feature contained in the feature channel. Channel weights can be weights specific to a feature channel.

[0111] For example, the average weight of a feature channel can be determined based on the weights of each depth feature information contained in the feature channel, and the maximum weight value corresponding to the depth feature information contained in the feature channel can be obtained. The channel weight of a feature channel can be determined based on the depth feature information, the average weight, and the maximum weight. For instance, the channel weight of a feature channel can be obtained by multiplying the sum of the average weight and the maximum weight by the depth feature information.

[0112] For example, channel weights can be determined using the following formula:

[0113] M c (f n )=ξ((a(f n )+max p(f n )))

[0114] Among them, M c (f n ) represents the channel weights, ξ represents the depth feature information, and a(f) represents the channel weights. n ) represents the weighted average, and max p(f) n ) represents the maximum weight.

[0115] In this embodiment, the channel weights of the feature channels are determined by using depth feature information, average weight, and maximum weight, which is beneficial for extracting classification and recognition features, thereby improving the accuracy of image content classification and recognition of remote sensing images.

[0116] In one embodiment, such as Figure 3 As shown, feature aggregation is performed on the channel features to obtain classification and recognition features, including:

[0117] S302, Obtain multiple feature aggregation parameters for channel features;

[0118] S304, based on multiple feature aggregation parameters and channel features, obtains classification and recognition features.

[0119] The feature aggregation parameters can be pre-set parameters or parameters obtained by convolving the channel features. The classification and recognition features can be features used to classify and recognize the image content of remote sensing images.

[0120] For example, multiple pre-set feature aggregation parameters can be obtained, and channel features can be processed using these parameters to obtain classification and recognition features. For instance, classification and recognition features can be obtained using the following formula:

[0121]

[0122] Where F1 represents the channel features, and g represents the semantic feature aggregation function; A1k This represents the feature aggregation parameters. P1 represents the classification and recognition features. M indicates that this parameter is omitted.

[0123] In this embodiment, the channel features are aggregated using multiple feature aggregation parameters to obtain classification and recognition features, thereby improving the accuracy of image content classification and recognition of remote sensing images.

[0124] In one embodiment, before extracting depth feature information from the remote sensing image, the process includes:

[0125] Acquire high-frequency and low-frequency information from remote sensing images;

[0126] Based on high-frequency and low-frequency information, the remote sensing image is denoised to obtain a denoised remote sensing image.

[0127] Extracting depth feature information from remote sensing images, including:

[0128] Extract depth feature information from the denoised remote sensing image.

[0129] High-frequency information can be high-frequency noise in the remote sensing image. Low-frequency information can be low-frequency noise in the remote sensing image.

[0130] For example, a noisy remote sensing image can be filtered using a low-frequency filter to separate high-frequency and low-frequency noise levels, thereby improving the accuracy of subsequent classification. The remote sensing image can then be denoised using both high-frequency and low-frequency information to obtain a denoised remote sensing image, and the depth feature information of the denoised remote sensing image can be extracted.

[0131] In this embodiment, the remote sensing image is denoised using high-frequency and low-frequency information to obtain a denoised remote sensing image. The depth feature information of the denoised remote sensing image is then extracted, which can improve classification accuracy and the accuracy of image content classification and recognition of the remote sensing image.

[0132] In one embodiment, such as Figure 4 As shown, the remote sensing image is denoised based on high-frequency and low-frequency information to obtain the denoised remote sensing image, including:

[0133] S402, Obtain the residual mapping relationship pre-constructed for high-frequency information;

[0134] S404, based on high-frequency information, low-frequency information and residual mapping relationship, obtains the loss function for pixels of remote sensing image;

[0135] S406: Based on the pixel loss function, high-frequency information, and low-frequency information, the remote sensing image is processed to obtain a denoised remote sensing image.

[0136] The residual mapping relationship can be a pre-set relationship, which can be obtained based on high-frequency information. The pixel loss function can be a pixel-comparison-based loss function.

[0137] For example, a pre-constructed residual mapping relationship can be obtained using high-frequency information. The high-frequency and low-frequency information are then processed using the residual mapping relationship to obtain a loss function for the pixels of the remote sensing image. Furthermore, the remote sensing image can be processed using the loss function for the pixels of the remote sensing image, along with the high-frequency and low-frequency information, to obtain a denoised remote sensing image.

[0138] In this embodiment, the loss function of pixels in the remote sensing image is determined by residual mapping relationship, high-frequency information, and low-frequency information; and based on the pixel loss function, high-frequency information, and low-frequency information, a denoised remote sensing image is obtained. This enables denoising of the remote sensing image, improving the accuracy of image content classification and recognition.

[0139] In one specific embodiment, with the continuous development of science and technology, remote sensing images, as an important carrier of geomorphological information visualization, have been efficiently applied in various environments. Remote sensing images are mainly composed of dozens or hundreds of data points from the same scene or the same spectral channel. Pixels with the same spectral dimension are fused to form a remote sensing image, which contains the magnitude and characteristics of electromagnetic waves from various ground features. Because remote sensing images contain a wide variety of features, they are easily affected by noise, leading to significant errors in image classification. To address this, existing technologies propose a remote sensing image classification method based on multi-level cross-layer bilinear fusion. This method uses data collected in the field to establish a multi-level feature extraction model, calculates the feature information at each regional scale in the image, and classifies the pixels according to their hierarchical arrangement within the model. However, this method does not consider the possibility of dual-scale variations for the same feature in a remote sensing image, resulting in significant extraction errors and affecting the classification results. Another existing technology employs a remote sensing image classification method based on graph convolutional networks. This method establishes a neural network image convolutional model, uses image brightness as a dividing scale to initially divide the image into bright and dark regions, and designs a differentiation model to further divide the image regions according to the brightness difference. However, this method is too sensitive to capturing the brightness and darkness of images, has a high computational cost for scale classification, and a limited scope of application. To address these issues, this application presents a multi-label remote sensing image classification strategy based on dual-semantic data augmentation. Considering the impact of noise on classification, a fuzzy loss function is used to denoise the remote sensing images, improving image clarity and texture representation, providing significant assistance for subsequent classification. A dual-semantic data augmentation space for remote sensing images is established, calculating the semantic changes of pixels in different dimensions within this space. Based on this, feature labels are added to pixels with the same semantic meaning. An image feature label discriminator is established according to the meaning of the corresponding labels for different regions of the remote sensing image. The linear synchronization parameters of images with the same semantic relationship are calculated, and efficient classification is achieved based on parameter comparison. Experiments demonstrate that this classification method has practical application capabilities and can achieve efficient classification in various remote sensing environments.

[0140] Before performing multi-label remote sensing image classification, a perceptual loss function is used to locate pixels with lost details and blurred images. The signal-to-noise ratio (SNR) residual values ​​of the original and defective images are calculated. The noisy image is then passed through a low-frequency filter to separate high-frequency and low-frequency noise levels, improving subsequent classification accuracy. An SNR residual compilation network is designed. Through training, it learns residual mappings and adjusts the spectral values ​​of noisy locations according to the residual mapping values ​​to achieve SNR point balance and effectively denoise the image.

[0141] Loss function L of remote sensing images σ The calculation formula is:

[0142]

[0143] In the formula, L represents the pixel-based loss function for remote sensing images (the pixel loss function of remote sensing images); υ This represents the loss function based on feature comparison. To improve the accuracy and effectiveness of denoising, the loss is calculated using both pixel loss and feature loss functions, and the root mean square error of the loss is determined. The feature loss and pixel loss are then compared and matched for the same noisy image. The formula is as follows:

[0144]

[0145] In the formula, W represents the label of the noisy image; x h Indicates high-frequency pixels; x l Represents low-frequency pixels; E represents normal image labels; F(x) h ) represents the signal-to-noise ratio residual mapping relationship.

[0146] Typically, noise points in remote sensing images have multiple causes. Introducing a loss function allows this method to effectively capture the characteristic changes of noise points through the loss residual relationship between pixels and feature points in the image. Based on this, a global skipping algorithm is used to derive the final denoising formula according to the high- and low-frequency distribution relationship of noise points within the noisy image:

[0147] y(x)=x h -x l F(L ζ )

[0148] Where y(x) is the denoised remote sensing image.

[0149] A dual semantic data augmentation space is established. The denoised remote sensing images are input into the space, and the semantic detail features in the images are captured and searched through a genetic learning algorithm. After continuous iterative augmentation training, different feature images are assigned label categories. Subsequent classification algorithms can perform accurate classification according to the semantic label strength.

[0150] A dual semantic data augmentation spatial hierarchy is established, with each hierarchy corresponding to different remote sensing image semantics, and the semantic weight value for each hierarchy is set to f. n The channel weights of the semantic attention channel (feature channel) are established according to the feature relationships of different regions in the image as follows:

[0151] M c (f n )=ξ((a(f n )+max p(f n )))

[0152] In the formula, M c (fn ) represents the channel weight value of the semantic attention channel (feature channel); ξ represents the initial feature (deep feature information); a(f n ) represents the average weight; max p represents the maximum weight. A dimensional concept is introduced into the dual-semantic data augmentation space, meaning that images of different dimensions receive different semantic weight types in this space. This avoids classification confusion caused by images with the same dimension and weight, maximizing classification accuracy. Introducing the spatial dimension concept yields:

[0153]

[0154] In the formula, F1 represents the channel feature, and M... s (f 1c ) represents the spatial weights (channel weights of feature channels) that include the concept of dimension; f 1c f 2c This represents dual semantic focus features (deep feature information).

[0155] The above process yields the feature value F1 (channel feature) after semantic focusing. Employing the concept of bilinear feature convergence, and starting from the semantic relationships of image features, this ensures that each dimension of the classification target in the image to be classified contains a key semantic element P1. The formula is as follows:

[0156]

[0157] In the formula, g represents the semantic feature aggregation function; A 1k The two semantic parameters represent the target to be classified, where F1 is the channel feature, P1 is the classification and recognition feature, and M is omitted.

[0158] The bi-semantic calculation results of the targets to be classified in remote sensing images given above can provide important assistance to subsequent image classification algorithms. By using the bi-semantic concept of each image, the problem of missing features can be made up for, errors can be reduced as much as possible, and classification accuracy and efficiency can be improved.

[0159] Based on image denoising and dual semantic weight calculation, a multi-label remote sensing image classification algorithm based on semi-supervised semi-learning is established. Typically, a general classifier determines whether a subsample is correctly classified and meets the classification criteria. However, in practical applications, differences between images can lead to errors. Therefore, a semi-supervised semi-learning algorithm is used to capture the authenticity of attribute features of different categories of remote sensing images. Labels are added to feature points with strong authenticity, and these labels are input into the classification model. The loss function is expanded, and the function outputs real and pseudo samples of the target to be classified. The probability value between a sample and a certain type is calculated, and effective classification is achieved by comparing the probability values.

[0160] The generator and discriminator each contain 1024 and 128 units, respectively, with different units representing different image feature types. The generator is expressed as:

[0161] X f =G(α,β)

[0162] In the formula, X f Let G(α, β) represent the generator; G(α, β) represent the probability that sample α belongs to β independent types in the classifier. Based on the pixel characteristics of different regions in multi-label remote sensing images, two types of classification regions are defined: real regions (where pixels are abundant and densely distributed) and false regions (where pixels are scarce and sparsely distributed). The log-likelihood for classifying these two regions is calculated, expressed by the formula:

[0163] L s =E[lg P(S=real|X r )+E[lg P(S=f|X f )]]

[0164] L c =E[lg P(C=c|X r )+E[lg P(C=c|X f )]]

[0165] In the formula, (L s +L c (L) represents the parameters to be maximized during classifier training; s -L c ) represents the maximum parameter for training the generator; X r This represents the classification result of the real area in the remote sensing image; X f This indicates the classification result of false regions in a remote sensing image.

[0166] Using a remote sensing image of a suburban area of ​​a city as the test object, and to meet the testing requirements of multiple features and labels, the image contains various landforms, including grasslands, land, residential areas, and factories. The image was acquired using a visible-infrared imaging spectrometer, with a pixel size of 155×155 and containing 224 spectral bands, including both low- and high-frequency bands. The wavelength influence range is 2.5μm. After removing spectral dispersion or overlap caused by duplicate coverage and illumination effects, the final actual image size is 155×155×220.

[0167] The original image contains areas such as grasslands, land, residential areas, factories, and wheat fields, with residential areas and land accounting for a larger proportion.

[0168] The experimental results were obtained by comparing the effects of different image sizes and component numbers on the classification results. Figures 5-7 As shown, Figure 5 The classification results are affected by the bilinear fusion method. Figure 6 The classification results are affected by convolutional network methods. Figure 7 This relates to the classification effect results of the remote sensing image classification and recognition method proposed in this application. The principal component count is a necessary value for generating the image, representing the size of the composition of pixels in different dimensions of the image. From... Figures 5-7 As can be seen, there is a significant difference in the three-dimensional classification accuracy of the three methods. Based on the comparison of different image patch sizes and principal component vertical axis values, it was found that at test points with the same image patch size and principal component number, the proposed method has higher classification accuracy, while the classification accuracy of the other two methods is worse. This indicates that the classification algorithm proposed in this application is better and has stronger practical performance.

[0169] In this embodiment, a remote sensing image classification method combining dual semantic data augmentation is presented, addressing the challenges of numerous feature types and complex spectral distributions in multi-label remote sensing images. Considering the impact of noisy images on classification accuracy, a perceptual loss function algorithm is employed to calculate the spectral difference between noisy and normal images. Effective denoising is achieved through difference compensation, significantly improving the accuracy of subsequent classification. A dual semantic data augmentation space is established, and semantic features are solved based on the dimensionality coefficients of the target points to be classified. The final classification algorithm uses the probability values ​​between the semantic meaning of the target and the feature category to achieve classification. Experimental data demonstrates that the proposed method achieves high classification accuracy and efficient classification for images of varying sizes.

[0170] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0171] Based on the same inventive concept, this application also provides a remote sensing image classification and recognition device for implementing the remote sensing image classification and recognition method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the remote sensing image classification and recognition device provided below can be found in the limitations of the remote sensing image classification and recognition method described above, and will not be repeated here.

[0172] In one embodiment, such as Figure 8 As shown, a remote sensing image classification and recognition device 800 is provided, including: a depth feature extraction module 810, a channel weight determination module 820, a channel feature acquisition module 830, a classification feature determination module 840, and a recognition result acquisition module 850, wherein:

[0173] The depth feature extraction module 810 is used to acquire the remote sensing image to be classified and identified, and extract the depth feature information of the remote sensing image;

[0174] The channel weight determination module 820 is used to determine the feature channels corresponding to the depth feature information, as well as the channel weights of the feature channels.

[0175] The channel feature acquisition module 830 is used to obtain channel features based on the channel weights and depth feature information of the feature channels.

[0176] The classification feature determination module 840 is used to perform feature aggregation processing on the channel features to obtain classification recognition features;

[0177] The recognition result acquisition module 850 is used to input the classification and recognition features into a pre-trained remote sensing image classification model, and obtain the image content classification and recognition result of the remote sensing image through the remote sensing image classification model.

[0178] In one embodiment, the remote sensing image classification model includes a classifier and a generator; the recognition result acquisition module is used to input the classification and recognition features into the pre-trained remote sensing image classification model, and obtain the image content classification and recognition result of the remote sensing image through the classifier.

[0179] In one embodiment, the device further includes a sample feature acquisition module, a random vector acquisition module, a prediction result acquisition module, a loss value determination module, and a training module.

[0180] The sample feature acquisition module acquires sample classification and recognition features, including a first sample classification and recognition feature and a second sample classification and recognition feature. The first sample classification and recognition feature carries image content labels, while the second sample classification and recognition feature does not. The random vector acquisition module acquires random vectors for the sample classification and recognition features and inputs these random vectors into a generator to obtain pseudo-sample classification and recognition features. The prediction result acquisition module inputs the first sample classification and recognition feature, the second sample classification and recognition feature, and the pseudo-sample classification and recognition feature into a classifier to obtain the sample classification and recognition feature recognition result and the image content classification prediction result. The image content classification prediction result includes the image content classification prediction result corresponding to the first sample classification and recognition feature. The loss value determination module determines a first loss value based on the sample classification and recognition feature recognition result, the sample classification and recognition feature, and the pseudo-sample classification and recognition feature, and determines a second loss value based on the image content labels of the first sample classification and recognition feature and the image content classification prediction result corresponding to the first sample classification and recognition feature. The training module is used to determine the total loss value of the classifier using the first loss value and the second loss value, and to train the classifier using the total loss value to obtain the trained remote sensing image classification model.

[0181] In one embodiment, the channel weight determination module includes an average value maximum unit and a channel weight calculation unit.

[0182] The average and maximum weight unit is used to obtain the average weight and maximum weight of the feature channel. The channel weight calculation unit is used to determine the channel weight of the feature channel based on the depth feature information, the average weight, and the maximum weight.

[0183] In one embodiment, the classification feature determination module includes an aggregation parameter acquisition unit and a classification feature determination unit.

[0184] The aggregation parameter acquisition unit is used to acquire multiple feature aggregation parameters for channel features. The classification feature determination unit is used to obtain classification and recognition features based on multiple feature aggregation parameters and channel features.

[0185] In one embodiment, the device further includes a high- and low-frequency acquisition module and a noise reduction processing module.

[0186] The high-frequency and low-frequency acquisition module acquires high-frequency and low-frequency information from the remote sensing image. The denoising module denoises the remote sensing image based on the high-frequency and low-frequency information to obtain a denoised remote sensing image. The depth feature extraction module extracts depth feature information from the denoised remote sensing image.

[0187] In one embodiment, the denoising processing module includes a mapping relationship unit, a loss function unit, and a denoised image determination unit.

[0188] The mapping relationship unit is used to obtain the pre-constructed residual mapping relationship for high-frequency information. The loss function unit is used to obtain the loss function for the pixels of the remote sensing image based on the high-frequency information, low-frequency information, and residual mapping relationship. The denoising image determination unit is used to obtain the denoised remote sensing image based on the pixel loss function, high-frequency information, and low-frequency information.

[0189] Each module in the aforementioned remote sensing image classification and recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0190] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores remote sensing images. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a remote sensing image classification and recognition method.

[0191] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0192] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0193] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0194] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0195] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0196] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A remote sensing image classification and recognition method, characterized in that, The method includes: Acquire the remote sensing image to be classified and identified, and extract the depth feature information of the remote sensing image; Determine the feature channels corresponding to the depth feature information, and the channel weights of the feature channels; Based on the channel weights of the feature channels and the depth feature information of the feature channels, channel features are obtained; The channel features are subjected to feature aggregation processing to obtain classification and recognition features; The classification and recognition features are input into a pre-trained remote sensing image classification model, and the image content classification and recognition results of the remote sensing image are obtained through the remote sensing image classification model. The determination of the feature channels corresponding to the depth feature information and the channel weights of the feature channels includes: A dual semantic data augmentation spatial hierarchy is established, with each hierarchy corresponding to different remote sensing image semantics; Obtain the average weight and the maximum weight corresponding to the feature channel; The channel weights of the feature channels are determined based on the depth feature information, the average weight, and the maximum weight. The determination of the channel weights of the feature channels includes: Set the semantic weight value corresponding to each level as follows: Based on the characteristic relationships of different regions in the remote sensing image, the channel weights of the feature channels are established as follows: ;in, The channel weight values ​​for the feature channels; The depth feature information; The average value of the weights; This represents the maximum value of the weight.

2. The method according to claim 1, characterized in that, The remote sensing image classification model includes a classifier and a generator; The step of inputting the classification and recognition features into a pre-trained remote sensing image classification model, and obtaining the image content classification and recognition result of the remote sensing image through the remote sensing image classification model, includes: The classification and recognition features are input into a pre-trained remote sensing image classification model, and the image content classification and recognition results of the remote sensing image are obtained through the classifier.

3. The method according to claim 2, characterized in that, The remote sensing image classification model is generated in the following ways: Obtain sample classification and recognition features; the sample classification and recognition features include a first sample classification and recognition feature and a second sample classification and recognition feature, wherein the first sample classification and recognition feature is a sample classification and recognition feature carrying image content tags, and the second sample classification and recognition feature is a sample classification and recognition feature not carrying image content tags; Obtain a random vector for the classification and recognition features of the sample, and input the random vector into the generator to obtain pseudo-sample classification and recognition features through the generator; The first sample classification and recognition feature, the second sample classification and recognition feature, and the pseudo-sample classification and recognition feature are input into the classifier, and the classifier obtains the sample classification and recognition feature recognition result and the image content classification prediction result; the image content classification prediction result includes the image content classification prediction result corresponding to the first sample classification and recognition feature. A first loss value is determined based on the sample classification and recognition feature recognition result, the sample classification and recognition feature, and the pseudo sample classification and recognition feature; and a second loss value is determined based on the image content label of the first sample classification and recognition feature and the image content classification prediction result corresponding to the first sample classification and recognition feature. The total loss value of the classifier is determined by using the first loss value and the second loss value, and the classifier is trained using the total loss value to obtain a trained remote sensing image classification model.

4. The method according to claim 1, characterized in that, The process of performing feature aggregation on the channel features to obtain classification and recognition features includes: Obtain multiple feature aggregation parameters for the channel features; The classification and recognition features are obtained based on the multiple feature aggregation parameters and the channel features.

5. The method according to claim 1, characterized in that, Before extracting the depth feature information of the remote sensing image, the process includes: Acquire the high-frequency information and low-frequency information of the remote sensing image; Based on the high-frequency and low-frequency information, the remote sensing image is denoised to obtain a denoised remote sensing image. The extraction of depth feature information from the remote sensing image includes: Extract the depth feature information of the denoised remote sensing image.

6. The method according to claim 5, characterized in that, The step of denoising the remote sensing image based on the high-frequency and low-frequency information to obtain a denoised remote sensing image includes: Obtain the pre-constructed residual mapping relationship for the high-frequency information; Based on the high-frequency information, the low-frequency information, and the residual mapping relationship, a loss function for the pixels of the remote sensing image is obtained; Based on the loss function of the pixels, the high-frequency information, and the low-frequency information, the denoised remote sensing image is obtained from the remote sensing image.

7. A remote sensing image classification and recognition device, characterized in that, The device includes: The depth feature extraction module is used to acquire the remote sensing image to be classified and identified, and extract the depth feature information of the remote sensing image; The channel weight determination module is used to determine the feature channels corresponding to the depth feature information, and the channel weights of the feature channels; The channel feature acquisition module is used to obtain channel features based on the channel weights of the feature channels and the depth feature information of the feature channels; The classification feature determination module is used to perform feature aggregation processing on the channel features to obtain classification recognition features; The recognition result acquisition module is used to input the classification and recognition features into a pre-trained remote sensing image classification model, and obtain the image content classification and recognition result of the remote sensing image through the remote sensing image classification model; The channel determination module is also used to: establish a dual semantic data augmentation spatial hierarchy, where each hierarchy corresponds to different remote sensing image semantics; The channel weight determination module includes: The average maximum value unit is used to obtain the average weight and the maximum weight corresponding to the feature channel; The channel weight calculation unit determines the channel weight of the feature channel based on the depth feature information, the average weight, and the maximum weight. The determination of the channel weights of the feature channels includes: Set the semantic weight value corresponding to each level as follows: Based on the characteristic relationships of different regions in the remote sensing image, the channel weights of the feature channels are established as follows: ;in, The channel weight values ​​for the feature channels; The depth feature information; The average value of the weights; This represents the maximum value of the weight.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images