Aerial remote sensing image target detection and identification method based on multi-scale feature fusion
Through the multi-scale feature fusion method, using denoising autoencoder and SIFT, beamlet features, combined with multi-scale dictionary learning and linear support vector machine, the problems of coherent speckle noise and geometric distortion in SAR images are solved, and the efficiency and robustness of target detection are improved.
Patent Information
- Application Number
- CN202510860268.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-25
- Publication Date
- 2025-10-17
AI Technical Summary
The presence of coherent speckle noise and geometric distortion in SAR images makes target detection and recognition difficult.
A multi-scale feature fusion method is adopted to remove noise through a denoising autoencoder, combine SIFT and beamlet features, use multi-scale dictionary learning for sparse coding and feature fusion, and finally perform classification through a linear support vector machine.
It effectively overcomes the influence of coherent speckle noise and geometric distortion, improves the computational efficiency and robustness of SAR image target detection, reduces the dependence on the number of training samples, and enhances the generalization ability of features.
Smart Images
Figure CN120808210A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of aerial remote sensing image target detection and recognition, and particularly relates to an aerial remote sensing image target detection and recognition method based on multi-scale feature fusion. BACKGROUND
[0002] In the field of aerial remote sensing, SAR (Synthetic Aperture Radar) is an active sensor using microwaves for sensing, and is one of the most rapidly developed and most effective sensors in microwave remote sensing devices. Compared with other sensors such as optical and infrared sensors, SAR has unique observation advantages in remote sensing detection, and is widely used in crop growth monitoring, forest volume monitoring, resource and mineral exploration, ecological environment protection monitoring, marine ecological environment monitoring, earthquake and geological disaster monitoring, topographic mapping, military reconnaissance, weapon guidance and navigation, etc. Compared with target recognition of optical images, the detection and recognition of SAR image targets face special difficulties, the most important of which are: (1) speckle noise: due to the SAR correlation imaging mechanism, there are a large number of speckle noises in the SAR image, which seriously affect the quality of the SAR image. (2) geometric distortion: due to the side-looking imaging mode of SAR, the target objects in the image will show special geometric distortion phenomena, including perspective contraction, overlap, and upslope shortening. Therefore, the present application proposes an aerial remote sensing image target detection and recognition method based on multi-scale feature fusion. SUMMARY
[0003] The present application aims to provide an aerial remote sensing image target detection and recognition method based on multi-scale feature fusion to solve the problems raised in the background.
[0004] To achieve the above-mentioned purpose, the present application provides the following technical solution: an aerial remote sensing image target detection and recognition method based on multi-scale feature fusion, comprising the following steps:
[0005] Step S1, SAR image denoising: based on a denoising autoencoder, a deep denoising autoencoder capable of feature fusion is constructed by training;
[0006] Step S2, extracting dense scale-invariant local features (SIFT) and beamlet features: the image enhanced by step S1 is denoised, then SIFT feature points and beamlet feature points are extracted from each image, and a training feature subset is formed by random sampling method; for the test image, dense SIFT features and beamlet features are also extracted, and all features are reserved for sparse coding;
[0007] Step S3, sparse representation of features: for the training feature subset obtained in step S2, a global multi-scale dictionary learning method is used to learn a global multi-scale dictionary, and then all dense SIFT features and beamlet features extracted from the image in step S2 (training images and test images) are sparsely coded to obtain sparse representation of features, and the sparse representation features are used to replace the SIFT feature points and beamlet feature points extracted at the corresponding positions of the original image;
[0008] Step S4, feature fusion: based on the SIFT feature points and beamlet feature points extracted in step S3, feature fusion is performed through a feature fusion network, and the feature vectors after fusion of the multi-scale SIFT feature points and beamlet feature points are obtained;
[0009] Step S5, linear support vector machine classification: after the final image description vector of the training image in step S4 is obtained, a classifier is trained for SAR image target recognition.
[0010] Preferably, the denoising process of step S1 specifically comprises: combining image sub-blocks with similar structures into a three-dimensional array using a three-dimensional module matching algorithm (BM3D), performing image filtering in the transform domain using a joint filtering method, and finally obtaining a denoised image through inverse transformation.
[0011] Preferably, the deep denoising autoencoder of step S1 comprises: an input layer, a plurality of hidden layers and an output layer; a full convolutional network layer is further arranged between the input layer and the first hidden layer, thereby synchronously extracting convolutional features and fusing them with SIFT features and beamlet features.
[0012] Preferably, the feature extraction process of step S2 first uses a sliding window method for dense sampling, selects different window sizes and sliding steps, extracts dense SIFT features and beamlet features at multiple scales in the same image, and then performs Gaussian blur and direction matching calculation to ensure that 128-dimensional SIFT descriptors and beamlet descriptors are extracted in each sliding window.
[0013] Preferably, the step S3 adds multi-scale features when extracting local features, and uses multi-scale features as input to train the dictionary and learn multi-scale characteristics.
[0014] Preferably, the algorithm of step S3 has the following steps:
[0015] (1) Dense SIFT feature and beamlet feature extraction: through the method of step S2, the 128-dimensional SIFT feature and the beamlet feature of multiple scales are extracted from the training image by equidistant dense sampling, and for a 64*64 size SAR image, the dense SIFT and beamlet features are extracted in [0, 1, 2] as a scale respectively, and the feature number reaches 59;
[0016] (2) Random down-sampling: a large number of SIFT feature descriptors and beamlet descriptors are extracted by dense sampling, and a large amount of redundancy exists, and the random sampling is carried out, the sampling ratio is selected according to the training set, and a multi-scale training feature subset is obtained;
[0017] (3) Multi-scale dictionary learning: the multi-scale dense SIFT feature set and the beamlet feature set are used as input, the RLS-DLA algorithm is used to learn the global multi-scale dictionary, and the multi-scale dictionary D is obtained;
[0018] (4) Sparse representation: for the training set, the original dense SIFT feature and the beamlet feature are subjected to multi-scale sparse representation, which is used for training the classifier later, and for the test set image, the dense SIFT feature and the beamlet feature of each picture are also extracted, and then the multi-scale dictionary is solved to obtain sparse expression, which is used as the input of the later classifier.
[0019] Compared with the prior art, the present application has the beneficial effects that:
[0020] The present application extracts SAR image features through a multi-scale analysis method, and combines beamlet and SIFT features, overcomes the difficulty of target recognition caused by the coherent speckle noise unique to SAR images, and has better calculation efficiency and robustness compared with the traditional feature extraction method, constructs a feature fusion model through a deep denoising auto-encoding network, extracts high-level features with good discrimination from SAR images by fusing multi-scale features, enhances the generalization ability of the features, further overcomes the difficulty of SAR image target recognition caused by geometric distortion at the high-level feature level, reduces the depth of the neural network and the number of hyperparameters compared with the existing deep learning method, enhances the interpretability, and reduces the dependence on the number of training samples. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION
[0022] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.
[0023] Embodiments
[0024] Please refer to Figure 1 , a kind of based on multi-scale feature fusion aerial remote sensing image target detection and recognition method in the illustration, including the following steps:
[0025] Step S1, SAR image denoising: based on denoising auto-encoder, by training to build the deep denoising auto-encoder that can carry out feature fusion;
[0026] Step S2, extract dense scale invariant local feature (SIFT) and beamlet feature: the image enhancement map of step S1 denoised image is carried out, then the SIFT feature point and beamlet feature point of each image are extracted, and the training feature subset is formed by the method of random sampling;For test image, dense SIFT feature box beamlet feature is also extracted, and all features are reserved for sparse coding;
[0027] Step S3, sparse representation of feature: for the training feature subset obtained in step S2, a global multi-scale dictionary is learned by using a multi-scale dictionary learning method, and then all dense SIFT features and beamlet features (training image and test image) extracted in step S2 are sparse coded, to obtain sparse representation of feature, and the SIFT feature point and beamlet feature point extracted at the corresponding position of the original image are replaced by sparse representation feature;
[0028] Step S4, feature fusion: based on the SIFT feature point and beamlet feature point extracted in step S3, feature fusion is carried out through feature fusion network, and the feature vector after the fusion of multi-scale SIFT feature point and beamlet feature point belongs to;
[0029] Step S5, linear support vector machine classification: after the final image description vector of training image in step S4, the classifier is trained to identify SAR image target.
[0030] In this embodiment, the step S1 of the denoising process is specifically as follows: a three-dimensional module matching algorithm (BM3D) is used to combine image sub-blocks with similar structures into a three-dimensional array, a joint filtering method is used to filter the image in the transform domain, and finally an inverse transform is performed to obtain the denoised image. The deep denoising autoencoder in the step S1 comprises an input layer, a plurality of hidden layers and an output layer. A full convolutional network layer is further arranged between the input layer and the first hidden layer, so as to synchronously extract convolutional features and fuse the SIFT features and the beamlet features.
[0031] Further, the step S2 of the feature extraction process is specifically as follows: a sliding window method is used for dense sampling, different window sizes and sliding steps are selected, dense SIFT features and beamlet features at multiple scales are extracted in the same image, and then Gaussian blur and direction matching calculation are performed to ensure that 128-dimensional SIFT descriptors and beamlet descriptors are extracted in each sliding window.
[0032] Further, the step S3 of adding multi-scale features during local feature extraction is further added, and the multi-scale features are used as input to train the dictionary and learn the multi-scale characteristics.
[0033] In this embodiment, the steps of the algorithm in the step S3 are as follows:
[0034] (1) Dense SIFT feature and beamlet feature extraction: through the method of step S2, multi-scale 128-dimensional SIFT features and beamlet features are extracted from the training image at equal intervals, and for a 64*64 size SAR image, [0, 1, 2] is used as a scale to extract dense SIFT and beamlet features, and the number of features reaches 59;
[0035] (2) Random downsampling: a large number of SIFT feature descriptors and beamlet descriptors are extracted through dense sampling, and a large amount of redundancy exists, so random sampling is performed, a sampling ratio is selected according to the training set, and a multi-scale training feature subset is obtained;
[0036] (3) Multi-scale dictionary learning: the multi-scale dense SIFT feature set and the beamlet feature set are used as input, the RLS-DLA algorithm is used to learn a global multi-scale dictionary, and a multi-scale dictionary D is obtained;
[0037] (4) Sparse representation: for the training set, the original dense SIFT features and beamlet features are subjected to multi-scale sparse representation, which is used for training the classifier later, and for the test set image, the dense SIFT features and beamlet features of each picture are also extracted, and then the multi-scale dictionary is solved to obtain sparse representation, which is used as the input of the classifier later.
[0038] It has to be noted that, in the present document, the terms "first", "second", etc. merely serve to identify a subject or action, without necessarily requiring or implying any such actual relationship or order between such subjects or actions. Moreover, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusions, such that a process, method, article, or apparatus that comprises a list of elements does not include those elements solely, but can include other elements not expressly listed, or can include elements inherent in such process, method, article, or apparatus.
[0039] While embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, combinations, and variations can be undertaken by those skilled in the art without departing from the spirit and scope of the present application, which is defined by the appended claims and their equivalents.
Claims
1. A method for target detection and recognition in aerial remote sensing images based on multi-scale feature fusion, characterized in that: The steps include: Step S1, SAR image denoising: Based on the denoising autoencoder, a deep denoising autoencoder capable of feature fusion is constructed through training; Step S2, extracting dense scale-invariant local features (SIFT) and beamlet features: the image denoised in step S1 is enhanced, and then SIFT feature points and beamlet feature points are extracted for each image, and a training feature subset is formed by random sampling; For the test image, the dense SIFT feature box beamlet features are also extracted, and all features are retained for sparse coding; Step S3, sparse representation of features: for the training feature subset obtained in step S2, a global multi-scale dictionary is learned using a multi-scale dictionary learning method, and then all dense SIFT features and Beamlet features extracted from the images in step S2 (training images and test images) are sparsely encoded to obtain a sparse representation of the features, and the sparse representation features are used to replace the SIFT feature points and Beamlet feature points extracted from the corresponding positions of the original image; Step S4, feature fusion: Based on the SIFT feature points and beamlet feature points extracted in step S3, feature fusion is performed through a feature fusion network to obtain a feature vector after the fusion of multi-scale SIFT feature points and beamlet feature points; Step S5, linear support vector machine classification: after the final image description vector of the training image in step S4 is obtained, a classifier is trained to perform SAR image target recognition.
2. The method for target detection and recognition in aerial remote sensing images based on multi-scale feature fusion according to claim 1, characterized in that: The denoising process of step S1 is specifically to combine image sub-blocks with similar structures into a three-dimensional array using a three-dimensional module matching algorithm (BM3D), perform image filtering in the transform domain using a joint filtering method, and finally obtain the denoised image through inverse transformation.
3. The method for target detection and recognition in aerial remote sensing images based on multi-scale feature fusion according to claim 2, characterized in that: The deep denoising autoencoder in step S1 includes: an input layer, several hidden layers and an output layer; a full convolutional network layer is further provided between the input layer and the first hidden layer, thereby synchronously extracting convolutional features and fusing them with SIFT features and beamlet features.
4. The method for target detection and recognition in aerial remote sensing images based on multi-scale feature fusion according to claim 3, characterized in that: In the feature extraction process of step S2, a sliding window method is first used for dense sampling. Different window sizes and sliding steps are selected to extract dense SIFT features and Beamlet features at multiple scales in the same image. Gaussian blur and directional matching calculations are then performed to ensure that 128-dimensional SIFT descriptors and Beamlet descriptors are extracted in each sliding window.
5. The method for target detection and recognition in aerial remote sensing images based on multi-scale feature fusion according to claim 4, characterized in that: In step S3, multi-scale features are added during local feature extraction, and the multi-scale features are used as input to train the dictionary and learn the multi-scale characteristics.
6. The method for target detection and recognition in aerial remote sensing images based on multi-scale feature fusion according to claim 5, characterized in that: The steps of the algorithm of step S3 are as follows: (1) Dense SIFT and Beamlet feature extraction: Using the method in step S2, multi-scale 128-dimensional SIFT features and Beamlet features are extracted from the training image by dense sampling at equal intervals. For a 64*64 SAR image, [0, 1, 2] is used as a scale to extract dense SIFT and Beamlet features, and the number of features reaches 59. (2) Random downsampling: A large number of SIFT feature descriptors and beamlet descriptors are extracted through dense sampling, among which there is a lot of redundancy. They are randomly sampled and the sampling ratio is selected according to the training set to obtain a multi-scale training feature subset; (3) Multi-scale dictionary learning: Taking the multi-scale dense SIFT feature set and the beamlet feature set as input, the RLS-DLA algorithm is used to learn the global multi-scale dictionary to obtain the multi-scale dictionary D; (4) Sparse representation: For the training set, the original dense SIFT features and beamlet features are sparsely represented at multiple scales and used for training the classifier in the later stage. For the test set images, the dense SIFT features and beamlet features of each image are also extracted, and then the sparse representation is solved by multi-scale dictionary and used as the input of the later classifier.