A remote sensing image target detection method and system based on hierarchical feature similarity calculation

Abstract

The invention discloses a remote sensing image target detection method and system based on hierarchical feature similarity calculation, comprising: loading remote sensing images and label information thereof, extracting features through a hierarchical stacked convolution module; fusing through a feature fusion network using a convolution operator, a feature splicing operator and a feature reshaping operator; decoupling through a detection head network optimized based on a hierarchical feature similarity calculation module, predicting position and category information, wherein the detection head network comprises a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function; using focal loss and rotation frame intersection-over-union loss to perform task training on a target detection model network, obtaining a trained remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module, and testing a remote sensing image to be tested; the invention comprehensively utilizes information of feature maps of different hierarchies for joint prediction modeling, thereby effectively improving the performance of remote sensing image ship target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image target detection method and system based on hierarchical feature similarity calculation. Background Art

[0002] Remote sensing image ship target detection technology has always been one of the research focuses in the remote sensing field. It has broad application prospects in the fields of marine safety, waterway monitoring, spatial planning, etc.; due to the complex sea environment, the task of remote sensing image ship target detection is extremely difficult and challenging.

[0003] Early methods for ship target detection in remote sensing images were mainly based on visual saliency and template matching. These methods were designed for specific scenarios and had poor performance when faced with complex ocean scene backgrounds and environmental noise, and were not robust enough for ship target detection.

[0004] At present, the ship target detection method in remote sensing images based on deep learning automatically extracts multi-level feature maps of remote sensing images based on feature extraction operators, mines the deep feature information of ship targets, and makes accurate predictions after feature fusion, showing good precision performance; in the feature maps extracted by deep learning models, shallow feature maps often contain rich local texture detail features (such as color, texture), which help the model understand the external representation of remote sensing targets and then understand the image content; while deep feature maps often contain global semantic features, which contain high-level abstract information related to objects and scenes in the image. This information covers the category, shape, structure of objects, and their position and relationship in the environment, which helps the model understand the deep essential representation of remote sensing targets, which is crucial for correctly identifying and locating potential target objects in remote sensing images.

[0005] However, remote sensing images have the characteristics of complex imaging background and drastic changes in target scale. Conventional ship target detection models cannot perform fine processing of feature information at different levels in the feature decoupling stage. As a result, a lot of valuable target information is lost in the interpretation stage, making it difficult to achieve accurate ship target detection.

[0006] Therefore, how to finely process shallow features and deep features to fully mine feature information at different levels in the decoupling stage and achieve accurate positioning of ship targets in remote sensing images is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0007] In view of this, the present invention provides a remote sensing image target detection method and system based on hierarchical feature similarity calculation, aiming to solve the problem that traditional deep learning models cannot effectively and comprehensively utilize feature map information at different levels to achieve accurate interpretation. By performing hierarchical feature similarity calculation in the feature decoupling stage to comprehensively utilize the information of feature maps at different levels for joint predictive modeling, the performance of remote sensing image ship target detection is effectively improved, a more reliable and efficient solution is provided for the field of remote sensing image ship target detection, and strong support is provided for future remote sensing application fields.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] A remote sensing image target detection method based on hierarchical feature similarity calculation includes the following steps:

[0010] S1. Load remote sensing images and their label information, including location and category information;

[0011] S2. extracting features from the input remote sensing image through a feature extraction network including a hierarchical stacked convolution module to obtain a multi-level feature map;

[0012] S3. The multi-level feature maps are fused by using a convolution operator, a feature concatenation operator, and a feature reshaping operator through a feature fusion network to obtain multi-scale characteristic information of potential targets, and the fused feature maps are output;

[0013] S4. Decouple the fused feature map through a detection head network optimized based on a hierarchical feature similarity calculation module, predict the location and category information of the potential target area, and output the prediction result. The detection head network includes a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function;

[0014] S5. Based on the feature extraction network, feature fusion network and detection head network optimized based on hierarchical feature similarity calculation module, an object detection model combined with multi-scale semantic feature attention fusion network is obtained;

[0015] S6. Use focal loss and rotation frame intersection-over-union loss to train the target detection model network for remote sensing image target detection tasks, and obtain a trained remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module;

[0016] S7. Use the trained remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module to test the remote sensing image to be tested.

[0017] Preferably, each level of stacked convolution modules in step S2 includes a basic convolution operator, a normalization module and an activation function;

[0018] The basic convolution operator calculates the channel information of the input remote sensing image at a pixel-by-pixel level. The normalization module normalizes the calculated channel results to the standard range. The activation function activates and inhibits the feature matrix of the potential target area to help the model distinguish between foreground and background information.

[0019] Preferably, the calculation process of the feature fusion network in step S3 is expressed as:

[0020] P n =Conv(F n )

[0021] P n-2 =Concat(Conv(F n-2 ), Resize(P n ))

[0022] P n-4 =Concat(Conv(F n-4 ), Resize(P n-2 ))

[0023] Among them, [F n-4 , F n-2 , F n ] is the feature map from three different levels in the multi-level feature map extracted by the feature extraction network, n is the number of layers of the convolution module of the feature extraction network, [P n-4 , P n-2 , P n ] is [F n-4 , F n-2 , F n ] The corresponding fused feature map, Concat represents the feature concatenation operator, Conv represents the convolution operator, and Resize represents the feature reshaping operator.

[0024] Preferably, in step S4, the method for decoupling the fused feature graph based on the detection head network optimized by the hierarchical feature similarity calculation module is:

[0025] S41. The convolution module is used to receive the fused feature map and process its shape and channel dimension;

[0026] S42. The hierarchical feature similarity calculation module is used to extract similarity feature information of three feature maps from different levels, wherein the three feature maps from different levels include a multi-level feature map extracted by a feature extraction network and a feature map fused by a corresponding feature fusion network and a feature map processed by a convolution module of a detection head network;

[0027] S43. The feature addition module is used to fuse the output feature map of the hierarchical feature similarity calculation module and the convolution module of the detection head network;

[0028] S44. The prediction function is used to predict the location and category information of the potential target area and output the prediction result.

[0029] Preferably, in step S42, the processing method of the hierarchical feature similarity calculation module is specifically as follows:

[0030] S421. Perform upsampling, feature dimension reduction and multi-layer perceptron feature extraction processing on three feature maps from different levels respectively;

[0031] S422. Perform multi-layer perceptron feature extraction on the feature graph after upsampling and feature dimension reduction, and then perform feature addition;

[0032] S423 performs a linear change operation on the feature graph after the multilayer perceptron feature extraction in step S421 to obtain a value matrix, and performs a linear change operation on the feature graph after the feature addition in step S422 to obtain a query matrix and a key value matrix;

[0033] S424. The feature similarities of the value matrix, query matrix and key-value matrix are calculated through the self-attention mechanism, the information expression of the multi-level feature graph in terms of long-distance dependency features is mined, and the feature information of the feature graph is extracted through point convolution. The final output feature graph is the output feature graph of the hierarchical feature similarity calculation module.

[0034] A remote sensing image target detection system based on hierarchical feature similarity calculation, based on the remote sensing image target detection method based on hierarchical feature similarity calculation, comprising a remote sensing image input module, a remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module, and a model training module;

[0035] The remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module includes a feature extraction network, a feature fusion network and a detection head network optimized based on the hierarchical feature similarity calculation module;

[0036] The feature extraction network includes a hierarchical stacked convolution module, which is used to extract features from the input remote sensing image and obtain a multi-level feature map;

[0037] The feature fusion network includes convolution operators, feature concatenation operators, and feature reshaping operators, which are used to fuse multi-level feature maps to obtain multi-scale characteristic information of potential targets and output fused feature maps.

[0038] The detection head network optimized based on the hierarchical feature similarity calculation module includes a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function, which is used to decouple the fused feature map, predict the location and category information of the potential target area, and output the prediction result;

[0039] The model training module is used to train the target detection model network for remote sensing image target detection tasks using focal loss and rotation frame intersection-over-union loss, and obtain a remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module.

[0040] Preferably, each level of stacked convolution modules includes a basic convolution operator, a normalization module and an activation function.

[0041] Preferably, the hierarchical feature similarity calculation module includes an upsampling module, a feature dimensionality reduction module, a multi-layer perceptron, a linear transformation module, a feature addition module, a self-attention module and a point convolution module.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a remote sensing image target detection method based on hierarchical feature similarity calculation.

[0043] A processing terminal includes a memory and a processor. The memory stores a computer program that can be run on the processor. When the processor executes the computer program, a remote sensing image target detection method based on hierarchical feature similarity calculation is implemented.

[0044] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image target detection method and system based on hierarchical feature similarity calculation. The hierarchical feature similarity calculation module can help the target detection model fully mine the target feature information under different receptive fields, avoid the problem of insufficient interpretation accuracy existing in conventional deep learning models, realize the complementary use of feature information, and effectively avoid the problem of information loss in the fusion process, so as to help the model correctly identify and locate potential targets in remote sensing images, and effectively improve the performance of ship target detection in remote sensing images.

[0045] Based on the detection head network of the hierarchical feature similarity calculation module, an optimization module is designed in the feature decoupling stage to comprehensively utilize the information of feature maps at different levels for joint prediction modeling, thereby improving the multi-level feature interpretation ability of the model to solve the problem of insufficient accuracy in the ship target detection task of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0047] Figure 1 A schematic diagram of a remote sensing image target detection method based on hierarchical feature similarity calculation provided by the present invention;

[0048] Figure 2 A schematic diagram of a processing method of a hierarchical feature similarity calculation module provided by the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The embodiment of the present invention discloses a remote sensing image target detection method based on hierarchical feature similarity calculation, such as Figure 1 , including the following steps:

[0051] S1. Load remote sensing images and their label information, including location and category information;

[0052] S2. extracting features from the input remote sensing image through a feature extraction network including a hierarchical stacked convolution module to obtain a multi-level feature map;

[0053] S3. The multi-level feature maps are fused by using a convolution operator, a feature concatenation operator, and a feature reshaping operator through a feature fusion network to obtain multi-scale characteristic information of potential targets, and the fused feature maps are output;

[0054] S4. Decouple the fused feature map through a detection head network optimized based on a hierarchical feature similarity calculation module, predict the location and category information of the potential target area, and output the prediction result. The detection head network includes a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function;

[0055] S5. Based on the feature extraction network, feature fusion network and detection head network optimized based on hierarchical feature similarity calculation module, an object detection model combined with multi-scale semantic feature attention fusion network is obtained;

[0056] S6. Use focal loss and rotation frame intersection-over-union loss to train the target detection model network for remote sensing image target detection tasks, and obtain a trained remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module;

[0057] S7. Use the trained remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module to test the remote sensing image to be tested.

[0058] In this embodiment, in step S1, the calculation process of loading the remote sensing image and its label information is expressed as:

[0059] data=f dataloder (imgs, labels)

[0060] Among them, imgs∈R H×W×3 represents an input remote sensing image with a size of H×W and a channel number of 3. labels is the annotation information of the target in the image, including location and category information. dataloder It is a data loader function used to load data and data augmentation.

[0061] The process of feature extraction network extracting features from input remote sensing images is expressed as:

[0062] [F1, F2, …, F i , …, F n ] = f backbone (imgs)

[0063] in, Represents the feature map obtained after feature extraction, with a size of H i ×W i , and there is C i channels, f backbone It is a feature extraction network that obtains feature information of the input remote sensing image by stacking convolution modules in layers. Typical backbone feature extraction networks include: ResNet, VGG, EfficientNet, etc. This embodiment takes the basic convolution module as an example, but it should be noted that through simple model replacement, the remote sensing image ship target detection model based on hierarchical feature similarity calculation can be adapted to the general basic backbone feature extraction network.

[0064] In order to further implement the above technical solution, each level of stacked convolution modules in step S2 includes a basic convolution operator, a normalization module and an activation function;

[0065] The basic convolution operator calculates the channel information of the input remote sensing image at a pixel-by-pixel level to achieve the purpose of feature extraction. The normalization module normalizes the calculated channel results to the standard range to facilitate subsequent calculations while avoiding gradient optimization anomalies. The activation function activates and inhibits the feature matrix of the potential target area to help the model distinguish between foreground and background information.

[0066] In this embodiment, the calculation process of stacked convolution modules at each level can be formally expressed as:

[0067] F1 = Relu(BN(Conv(imgs)))

[0068] Among them, Conv represents the basic convolution operation, BN is the normalization module, which keeps the number of channels for receiving and outputting features unchanged, imgs is the input original remote sensing image, F1 is the output feature map obtained after feature extraction, and Relu is the activation function responsible for activating the potential target area of ​​the output feature map;

[0069] Conv is defined as a convolution module containing a basic convolution operator, a normalization module, and an activation function. Therefore, the deep feature extraction process of a deep neural network can be formally expressed as:

[0070] F n =Conv n (Conv...(Conv2(Conv1(img))))

[0071] Among them, CONV n represents the convolution module of the nth layer, img is the input image, F n It is a multi-level feature map extracted by ordinary convolution operator. Based on this, we can obtain n feature maps of remote sensing images after deep feature extraction;

[0072] Then, the feature fusion network is responsible for receiving the feature graphs of the backbone feature extraction network for fusion to obtain the multi-scale characteristic information of the potential target and selecting the feature graphs of the backbone feature extraction network f backbone The extracted feature map [F n-4 , F n-2 , F n ] is passed to the feature fusion network for feature fusion.

[0073] In order to further implement the above technical solution, the calculation process of the feature fusion network in step S3 is expressed as:

[0074] P n =Conv(F n )

[0075] P n-2=Concat(Conv(F n-2 ), Resize(P n ))

[0076] P n-4 =Concat(Conv(F n-4 ), Resize(P n-2 ))

[0077] Among them, [F n-4 , F n-2 , F n ] is the feature map from three different levels in the multi-level feature map extracted by the feature extraction network, n is the number of layers of the convolution module of the feature extraction network, [P n-4 , P n-2 , P n ] is [F n-4 , F n-2 , F n ] The corresponding fused feature map, Concat represents the feature concatenation operator, Conv represents the convolution operator, and Resize represents the feature reshaping operator.

[0078] In order to further implement the above technical solution, the method for decoupling the fused feature map based on the detection head network optimized by the hierarchical feature similarity calculation module in step S4 is:

[0079] S41. The convolution module is used to receive the fused feature map and process its shape and channel dimension;

[0080] S42. The hierarchical feature similarity calculation module is used to extract similarity feature information of three feature maps from different levels, wherein the three feature maps from different levels include a multi-level feature map extracted by a feature extraction network and a feature map fused by a corresponding feature fusion network and a feature map processed by a convolution module of a detection head network;

[0081] S43. The feature addition module is used to fuse the output feature map of the hierarchical feature similarity calculation module and the convolution module of the detection head network;

[0082] S44. The prediction function is used to predict the location and category information of the potential target area and output the prediction result.

[0083] In this embodiment, the calculation process of the detection head is expressed as:

[0084] D n =ADD(Conv(F n ), LFAM(F n , P n , Conv(P n )))

[0085] D n-2 =ADD(Conv(F n-2 ), LFAM(F n-2 , P n-2 , Conv(P n-2 )))

[0086] D n-4 =ADD(Conv(F n-4 ), LFAM(F n-4 , P n-4 , Conv(P n-4 )))

[0087] Among them, LFAM is a hierarchical feature similarity calculation module;

[0088] The prediction function calculation process is expressed as:

[0089]

[0090] in, To predict the classification information obtained, To predict the location information, D i is the feature map to be interpreted, f cls is the classification function, f bbox is the rectangular box prediction function.

[0091] In order to further implement the above technical solution, Figure 2 In step S42, the processing method of the hierarchical feature similarity calculation module is specifically as follows:

[0092] S421. Perform upsampling, feature dimension reduction and multi-layer perceptron feature extraction processing on three feature maps from different levels respectively;

[0093] S422. Perform multi-layer perceptron feature extraction on the feature graph after upsampling and feature dimension reduction, and then perform feature addition;

[0094] S423. Perform a linear change operation on the feature map after the multilayer perceptron feature extraction in step S421 to obtain a value matrix, and perform a linear change operation on the feature map after the feature addition in step S422 to obtain a query matrix and a key value matrix;

[0095] S424. The feature similarities of the value matrix, query matrix and key-value matrix are calculated through the self-attention mechanism, the information expression of the multi-level feature graph in terms of long-distance dependency features is mined, and the feature information of the feature graph is extracted through point convolution. The final output feature graph is the output feature graph of the hierarchical feature similarity calculation module.

[0096] In this embodiment, step S421 defines is the feature map of the i-th level, i∈{1, 2, …, n}, with a size of H×W and C channels; input feature map F A , F B and F C Represented as feature maps of different levels, namely F A , F B and F C The three inputs of the proposed corresponding level feature similarity calculation module are Figure 1 It can be seen that F A F is extracted by the feature extraction network. B is obtained after feature fusion by the feature fusion network, F C F B After being passed to the detection head network, it is obtained through a layer of convolution module. The three are respectively subjected to upsampling, feature dimension reduction and multi-layer perceptron feature extraction processing to achieve the unification of feature map latitude. The calculation process can be formally expressed as:

[0097] F A_1 =Upsample(F A )

[0098] F B_1 =FDR(F B )

[0099] F C_1 =Mlp(F C )

[0100] Among them, Upsample is an upsampling module, which is used to standardize the shape of the feature map for subsequent calculations; FDR is a feature dimension reduction module, which is used to map the high-dimensional channel features of the feature map to lower latitudes to standardize the dimensional shape of the feature map and facilitate subsequent calculations; after upsampling and feature dimension reduction operations, the feature maps F from different levels containing different receptive field feature information A and F B is processed into and feature map F C The same shape; using different feature dimension processing methods, the difference information contained in different feature maps is retained to the greatest extent without being lost due to operator characteristics;

[0101] In step S422, the same multi-layer perceptron module is used to implement feature extraction:

[0102] F A_2 =Mlp(F A_1 )

[0103] F B_2 =Mlp(F B_1 )

[0104] Among them, Mlp represents a multi-layer perceptron module, and the number of neurons in its input layer is F C The number of channels is C, the number of neurons in the hidden layer is 10×C, and the number of neurons in the output layer is 2×C;

[0105] For the feature map F A_2 and F B_2 Perform feature addition operations to achieve the fusion of feature maps at different levels, paving the way for the subsequent stage to supplement the complete target feature information:

[0106] F AB_2 =Add(F B_2 , F A_2 )

[0107] Among them, Add represents the feature addition module, which adds two feature maps of the same shape to increase the content of information without changing the feature latitude. AB_2 is the feature map obtained after feature addition, and its feature latitude is the same as F C Consistency;

[0108] In step S423, the feature map F is transformed using a linear transformation operation. AB_2 and F C_1 For processing, the specific linear change method can be implemented by point convolution or full connection matrix:

[0109] QK=Linear(F AB_2 )

[0110] V=Linear(F C_1 )

[0111] Among them, Linear is a linear transformation matrix, the purpose is to obtain the corresponding query matrix Q, key value matrix K, and value matrix V; it should be noted that here in order to calculate F AB_2 and F C_1 The feature similarity information of F is used to further mine the difference feature information of potential targets in feature graphs of different levels. The query matrix Q and the key value matrix K are composed of F AB_2 Get, and the value matrix V is composed of F C_1 get;

[0112] In step S424, the self-attention mechanism is used to calculate feature similarity, and its calculation process can be formally expressed as:

[0113]

[0114] Among them, the self-attention mechanism maps the input vector to the word vector probability of a given output to obtain the prediction of the content of each sequence, d represents the dimension of its feature map, and FAbc is the feature map obtained by self-attention calculation, and its feature latitude is the same as F C Stay consistent;

[0115] In step S424, feature information of the feature map is extracted by point convolution, the purpose of which is to mine the feature information of the feature map in the channel dimension and adjust the number of channels of the feature map to facilitate subsequent feature fusion operations. The calculation process can be formally expressed as:

[0116] F out =Conv1(F Abc )

[0117] Among them, Conv1 represents the point convolution operation, F out Represents the feature map of the final output.

[0118] Furthermore, in this embodiment, focal loss (FocalLoss) and rotated box intersection over union loss (RotatedIoU) are used to train the network for remote sensing image target detection tasks, which is formally expressed as:

[0119]

[0120] Among them, FlcalLoss represents focal loss, which is used for the classification prediction branch of remote sensing image target detection tasks; RotatedIoU represents the rotated box IoU loss function, which is used to calculate the distance between the predicted box and the target true value label corresponding to the input remote sensing image; Loss represents the total loss used in network training; when the network is trained until the loss no longer decreases, it means that the network training has reached stability and the training process is completed, and the trained target detection model combined with the rotation feature perception module is obtained.

[0121] Furthermore, the remote sensing image to be tested is tested, which is formally expressed as:

[0122]

[0123] Among them, Detector represents the trained remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module, x Test represents the remote sensing image to be tested, Indicates the corresponding ship target detection result.

[0124] A remote sensing image target detection system based on hierarchical feature similarity calculation, based on a remote sensing image target detection method based on hierarchical feature similarity calculation, including a remote sensing image input module, a remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module, and a model training module;

[0125] The remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module includes a feature extraction network, a feature fusion network and a detection head network optimized based on the hierarchical feature similarity calculation module;

[0126] The feature extraction network includes a hierarchical stacked convolution module, which is used to extract features from the input remote sensing image and obtain a multi-level feature map;

[0127] The feature fusion network includes convolution operators, feature concatenation operators, and feature reshaping operators, which are used to fuse multi-level feature maps to obtain multi-scale characteristic information of potential targets and output fused feature maps.

[0128] The detection head network optimized based on the hierarchical feature similarity calculation module includes a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function, which is used to decouple the fused feature map, predict the location and category information of the potential target area, and output the prediction result;

[0129] The model training module is used to train the target detection model network for remote sensing image target detection tasks using focal loss and rotation frame intersection-over-union loss, and obtain a remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module.

[0130] In order to further implement the above technical solution, each level of stacked convolution modules includes a basic convolution operator, a normalization module and an activation function.

[0131] In order to further implement the above technical solution, the hierarchical feature similarity calculation module includes an upsampling module, a feature dimensionality reduction module, a multi-layer perceptron, a linear transformation module, a feature addition module, a self-attention module and a point convolution module.

[0132] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a remote sensing image target detection method based on hierarchical feature similarity calculation.

[0133] A processing terminal comprises a memory and a processor. The memory stores a computer program that can be run on the processor. When the processor executes the computer program, a remote sensing image target detection method based on hierarchical feature similarity calculation is implemented.

[0134] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0135] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image target detection method based on hierarchical feature similarity calculation, characterized in that: The following steps are involved: S1. Load remote sensing images and their label information, including location and category information; S2. extracting features from the input remote sensing image through a feature extraction network including a hierarchical stacked convolution module to obtain a multi-level feature map; S3. The multi-level feature maps are fused by using a convolution operator, a feature concatenation operator, and a feature reshaping operator through a feature fusion network to obtain multi-scale characteristic information of potential targets, and the fused feature maps are output; S4. Decouple the fused feature map through a detection head network optimized based on a hierarchical feature similarity calculation module, predict the location and category information of the potential target area, and output the prediction result. The detection head network includes a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function; S5. Based on the feature extraction network, feature fusion network and detection head network optimized based on hierarchical feature similarity calculation module, an object detection model combined with multi-scale semantic feature attention fusion network is obtained; S6. Use focal loss and rotation frame intersection-over-union loss to train the target detection model network for remote sensing image target detection tasks, and obtain a trained remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module; S7. Use the trained remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module to test the remote sensing image to be tested.

2. The remote sensing image target detection method based on hierarchical feature similarity calculation according to claim 1, characterized in that: In step S2, each level of stacked convolution modules includes a basic convolution operator, a normalization module and an activation function; The basic convolution operator calculates the channel information of the input remote sensing image at a pixel-by-pixel level. The normalization module normalizes the calculated channel results to the standard range. The activation function activates and inhibits the feature matrix of the potential target area to help the model distinguish between foreground and background information.

3. The remote sensing image target detection method based on hierarchical feature similarity calculation according to claim 1, characterized in that: The calculation process of the feature fusion network in step S3 is expressed as: P n =Conv(F n ) P n-2 =Concat(Conv(F n-2 ),Resize(P n )) P n-4 =Concat(Conv(F n-4 ),Resize(P n-2 )) Among them, [F n-4 , F n-2 , F n ] is the feature map from three different levels in the multi-level feature map extracted by the feature extraction network, n is the number of layers of the convolution module of the feature extraction network, [P n-4 , P n-2 , P n ] is [F n-4 , F n-2 , F n ] The corresponding fused feature map, Concat represents the feature concatenation operator, Conv represents the convolution operator, and Resize represents the feature reshaping operator.

4. The remote sensing image target detection method based on hierarchical feature similarity calculation according to claim 1, characterized in that: In step S4, the method for decoupling the fused feature map based on the detection head network optimized by the hierarchical feature similarity calculation module is: S41. The convolution module is used to receive the fused feature map and process its shape and channel dimension; S42. The hierarchical feature similarity calculation module is used to extract similarity feature information of three feature maps from different levels, wherein the three feature maps from different levels include a multi-level feature map extracted by a feature extraction network and a feature map fused by a corresponding feature fusion network and a feature map processed by a convolution module of a detection head network; S43. The feature addition module is used to fuse the output feature map of the hierarchical feature similarity calculation module and the convolution module of the detection head network; S44. The prediction function is used to predict the location and category information of the potential target area and output the prediction result.

5. The remote sensing image target detection method based on hierarchical feature similarity calculation according to claim 4, characterized in that: In step S42, the processing method of the hierarchical feature similarity calculation module is specifically as follows: S421. Perform upsampling, feature dimension reduction and multi-layer perceptron feature extraction processing on three feature maps from different levels respectively; S422. Perform multi-layer perceptron feature extraction on the feature graph after upsampling and feature dimension reduction, and then perform feature addition; S423 performs a linear change operation on the feature graph after the multilayer perceptron feature extraction in step S421 to obtain a value matrix, and performs a linear change operation on the feature graph after the feature addition in step S422 to obtain a query matrix and a key value matrix; S424. The feature similarities of the value matrix, query matrix and key-value matrix are calculated through the self-attention mechanism, the information expression of the multi-level feature graph in terms of long-distance dependency features is mined, and the feature information of the feature graph is extracted through point convolution. The final output feature graph is the output feature graph of the hierarchical feature similarity calculation module.

6. A remote sensing image target detection system based on hierarchical feature similarity calculation, characterized in that: A remote sensing image target detection method based on hierarchical feature similarity calculation according to any one of claims 1 to 5, comprising a remote sensing image input module, a remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module, and a model training module; The remote sensing image ship target detection model combined with the hierarchical feature similarity calculation module includes a feature extraction network, a feature fusion network and a detection head network optimized based on the hierarchical feature similarity calculation module; The feature extraction network includes a hierarchical stacked convolution module, which is used to extract features from the input remote sensing image and obtain a multi-level feature map; The feature fusion network includes convolution operators, feature concatenation operators, and feature reshaping operators, which are used to fuse multi-level feature maps to obtain multi-scale characteristic information of potential targets and output fused feature maps. The detection head network optimized based on the hierarchical feature similarity calculation module includes a basic convolution module, a hierarchical feature similarity calculation module, a feature addition module and a prediction function, which is used to decouple the fused feature map, predict the location and category information of the potential target area, and output the prediction result; The model training module is used to train the target detection model network for remote sensing image target detection tasks using focal loss and rotation frame intersection-over-union loss, and obtain a remote sensing image ship target detection model combined with a hierarchical feature similarity calculation module.

7. A remote sensing image target detection system based on hierarchical feature similarity calculation according to claim 6, characterized in that: Each level of stacked convolution modules includes a basic convolution operator, a normalization module, and an activation function.

8. A remote sensing image target detection system based on hierarchical feature similarity calculation according to claim 6, characterized in that: The hierarchical feature similarity calculation module includes an upsampling module, a feature dimension reduction module, a multi-layer perceptron, a linear transformation module, a feature addition module, a self-attention module and a point convolution module.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image target detection method based on hierarchical feature similarity calculation described in any one of claims 1 to 5 is implemented.

10. A processing terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the remote sensing image target detection method based on hierarchical feature similarity calculation as described in any one of claims 1 to 5 is implemented.

Images