A remote sensing image target detection method based on feature fusion

Abstract

The present invention discloses a remote sensing image target detection method based on feature fusion, comprising: acquiring optical remote sensing image information; performing positioning and recognition processing on the optical remote sensing image information using a remote sensing target detection model to obtain an output feature map information set; the output feature map information set includes a plurality of output feature map information; and post-processing the output feature map information set to obtain a target image detection information set. The remote sensing target detection model comprises an input end, a backbone feature extraction network, a feature fusion network, and an output end in order from input to output. The present invention integrates a context conversion module into the backbone feature extraction network and the feature fusion network, which constructs a computing unit that is more suitable for computer vision through the operation of grouped convolution, extracts static and dynamic context information of the input feature variables, and constructs a new loss function to optimize the remote sensing target detection capability of the model.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing technology, and in particular relates to a remote sensing image target detection method based on feature fusion. Background Art

[0002] With the development of remote sensing technology, remote sensing image detection has been widely used in both military and civilian fields. Using satellite-captured remote sensing images for target detection can greatly facilitate maritime search and rescue, military intelligence reconnaissance, and traffic monitoring. However, unlike optical image detection in natural scenes, remote sensing image target detection faces the challenges of drastic changes in target scale, a high proportion of small targets, and complex image scenes. These issues lead to frequent false detections and omissions, which in turn severely impact detection accuracy and efficiency, limiting its application in satellite remote sensing technology.

[0003] Traditional remote sensing image target detection methods are usually based on digital image processing methods, that is, texture feature extraction is first performed, and then template matching, shallow learning, and background modeling are used to detect and identify the target. However, such methods have poor generalization ability for remote sensing images and poor detection results.

[0004] Existing intelligent target detection methods for remote sensing images, which are improvements to classic target detection algorithms, mostly still use convolutional neural networks. While these methods can improve remote sensing target detection to varying degrees, these improved convolutional neural networks are only able to model local information and lack the ability to model and perceive long-range information. Because remote sensing targets are often globally densely distributed in images, remote sensing image target detection algorithms based solely on convolutional neural networks lack long-range modeling and perception capabilities and have weak visual representation capabilities, which can easily lead to low remote sensing target detection rates. Furthermore, traditional artificial neural network models struggle to strike a balance between detection accuracy and model lightweightness, often sacrificing real-time detection for improved accuracy, or even improving real-time performance while still maintaining low detection accuracy. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that traditional remote sensing image target detection methods have poor generalization ability for remote sensing images and poor detection effect. Existing remote sensing image target intelligent detection methods lack the ability of long-distance modeling and perception, and have weak visual expression ability, which easily leads to a low remote sensing target detection rate.

[0006] In order to solve the above technical problems, the first aspect of the embodiments of the present invention discloses a method for remote sensing image target detection based on feature fusion, the method comprising:

[0007] Acquiring optical remote sensing image information; the optical remote sensing image information includes a plurality of optical remote sensing images;

[0008] Performing positioning and recognition processing on the optical remote sensing image information using a preset remote sensing target detection model to obtain an output feature map information set; the output feature map information set includes a plurality of output feature map information;

[0009] The output feature map information set is post-processed to obtain a target image detection information set; the target image detection information set includes a plurality of target image detection information.

[0010] The remote sensing target detection model includes an input end (Input), a backbone feature extraction network (Backbone), a feature fusion network (Neck) and an output end (Head) in sequence from input to output.

[0011] The input end is used to receive the acquired optical remote sensing image information and preprocess it.

[0012] Preprocessing is to process the acquired optical remote sensing image information using data enhancement methods, and then use adaptive image scaling methods to unify the sizes of all optical remote sensing images.

[0013] The backbone feature extraction network includes a downsampling module (Focus layer), a feature extraction module (CBS layer), a residual module (C3), and a spatial pyramid pooling module (SPP). The backbone feature extraction network is used to extract features of optical remote sensing image information.

[0014] The downsampling module is used to perform interval slicing operations on the preprocessed optical remote sensing image information in the vertical and horizontal directions of the image to obtain discrete slice information, then splice the discrete slice information, and finally convolve the spliced information to obtain the first mapping feature.

[0015] The feature extraction module is used to perform two-dimensional convolution (Conv2d), normalization (BatchNorm) and activation layer operations on the first mapping features in sequence to obtain second mapping features.

[0016] The residual module includes several classic residual structures (Bottleneck). The residual module is used to perform a convolution layer operation on the second mapping feature of its input, and add the value obtained after the convolution layer operation to the original value of the second mapping feature to obtain the third mapping feature, thereby achieving the completion of residual feature transfer without increasing the output depth.

[0017] The spatial pyramid pooling module is used to perform several maximum pooling operations of different sizes on the third mapping features, and then splice the results of the maximum pooling operations to obtain the image features of the optical remote sensing image.

[0018] The feature fusion network, including the Feature Pyramid Networks (FPN) and Path Aggregation Networks (PAN), is used to fuse image features at different levels of optical remote sensing images. Image features include category features and location features.

[0019] The feature pyramid structure includes, from input to output, a context conversion module, a feature extraction module, and an upsampling module in sequence; the feature output by the context conversion module passes through the feature extraction module and then the upsampling module to obtain a fourth mapping feature, which is spliced with the third mapping feature output by the residual module in the backbone feature extraction network to obtain a fifth mapping feature as the output of the feature pyramid structure; the path aggregation network structure includes, from input to output, an input module, a residual module, a feature extraction module, and a context conversion module in sequence; the input module receives the fifth mapping feature output by the feature pyramid structure, and then passes it through the residual module and the feature extraction module respectively to obtain a sixth mapping feature, splicing the sixth mapping feature with the output of the feature extraction module in the feature pyramid structure to obtain a seventh mapping feature, and splicing the feature obtained after the seventh mapping feature passes through the residual module and the feature extraction module in sequence with the output of the feature extraction module of the feature pyramid structure, and then the spliced feature obtained passes through the context conversion module to obtain an output feature map information set;

[0020] The context conversion module simultaneously realizes the integration functions of context information mining and self-attention learning, and promotes self-attention learning by making full use of the context information between targets in adjacent remote sensing images, thereby enhancing the expressive power of the output feature map.

[0021] The context conversion module first performs context encoding on all adjacent keys in the k×k image grid obtained by segmentation of the remote sensing image for its input feature variable X to obtain a feature matrix K with static context information. 1 ; Then K 1 It is concatenated with the Q space, and then two consecutive 1×1 convolution operations are performed on the concatenated result to obtain the static context attention matrix A. The calculation process is:

[0022] A=[K 1 ,Q]W θ W δ ,

[0023] In the above formula, W θ is the first 1×1 convolution operation matrix, W δ It is the second 1×1 convolution operation matrix.

[0024] Subsequently, the context attention matrix A is multiplied by the 1×1 convolution matrix V to obtain the feature map matrix K with dynamic context information. 2 , and its calculation process is:

[0025] K 2 =Conv 1×1 (V)A,

[0026] Among them, Conv 1×1 (V) represents the matrix V after 1×1 convolution.

[0027] Finally, K 2 With K 1 The fusion is performed to obtain the output matrix Y with global information and local information.

[0028] The output end is used to evaluate the difference between the output feature map information obtained by the feature fusion network and the real feature map information, and update the parameters of the remote sensing target detection model according to the evaluation results.

[0029] The remote sensing target detection loss function is used to evaluate the difference between the output feature map information obtained by the feature fusion network and the real feature map information. The remote sensing target detection loss function is obtained by calculating the overlap loss, the center distance loss, and the width and height loss. Its formula is:

[0030]

[0031] Where, I EIOU represents the remote sensing target detection loss function, B represents the target real box, B i Represents the target prediction box in the output feature map information, is the ratio of the intersection area and the union area of the target real box and the target prediction box, b and b gt are the center points of the target prediction box and the target real box respectively, ρ is the Euclidean distance between the two center points, c is the diagonal distance of the minimum bounding rectangle covering the target prediction box and the target real box, w and w gt are the lengths of the target prediction box and the target real box, h and h respectively. gt are the widths of the target prediction box and the target real box, respectively, C w and C h are the width and length of the minimum bounding rectangle covering the target prediction box and the target real box, respectively.

[0032] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0033] The present invention integrates the context conversion module into the backbone feature extraction network and feature fusion network. It constructs an attention calculation unit that is more suitable for computer vision through group convolution and 1×1 convolution operations, effectively improves the correlation between K and Q, extracts the static and dynamic context information of the input feature variables, and constructs a new loss function to optimize the model's remote sensing target detection capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 Schematic diagram of the composition of a remote sensing target detection model used in a remote sensing image target detection method based on feature fusion disclosed in an embodiment of the present invention;

[0036] Figure 2 It is a schematic diagram of the composition of the context conversion module disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0039] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0040] Figure 1 Schematic diagram of the composition of a remote sensing target detection model used in a remote sensing image target detection method based on feature fusion disclosed in an embodiment of the present invention; Figure 2 FIG. 1 is a schematic diagram of the composition of the context conversion module disclosed in the embodiment of the present invention, which will be described in detail below.

[0041] Example 1

[0042] In order to solve the above technical problems, the first aspect of the embodiments of the present invention discloses a method for remote sensing image target detection based on feature fusion, the method comprising:

[0043] Acquiring optical remote sensing image information; the optical remote sensing image information includes a plurality of optical remote sensing images;

[0044] Performing positioning and recognition processing on the optical remote sensing image information using a preset remote sensing target detection model to obtain an output feature map information set; the output feature map information set includes a plurality of output feature map information;

[0045] The output feature map information set is post-processed to obtain a target image detection information set; the target image detection information set includes a plurality of target image detection information.

[0046] The remote sensing target detection model includes an input end (Input), a backbone feature extraction network (Backbone), a feature fusion network (Neck) and an output end (Head) in sequence from input to output.

[0047] The input end is used to receive the acquired optical remote sensing image information and preprocess it.

[0048] Preprocessing is to process the acquired optical remote sensing image information using data enhancement methods such as Mosaic and flipping, and then use the adaptive image scaling method to unify the size of all optical remote sensing images.

[0049] The preprocessing also includes smoothing the acquired optical remote sensing images to overcome the acquisition errors that occur within a certain period of time. Specifically, it includes grayscale processing of the optical remote sensing images acquired within a period of time to obtain corresponding grayscale matrices, and calculating the eigenvectors of each grayscale matrix to obtain the eigenvector group [x1, x2, ..., x N ], N is the number of optical remote sensing images collected within a period of time, the cross-correlation matrix C of the eigenvector group is calculated, and the eigenvalue decomposition of the cross-correlation matrix C is performed to obtain:

[0050] C=VDV H ,

[0051] Among them, V is the eigenvector matrix, D is the eigenvalue matrix, and the diagonal elements of the matrix D are normalized and used as weight vectors. The optical remote sensing images collected over a period of time are weighted summed to obtain the smoothed values of the optical remote sensing images collected over a period of time as the preprocessed data.

[0052] The backbone feature extraction network includes a downsampling module (Focus layer), a feature extraction module (CBS layer), a residual module (C3), and a spatial pyramid pooling module (SPP). The backbone feature extraction network is used to extract features of optical remote sensing image information.

[0053] The downsampling module is used to perform interval slicing operations on the preprocessed optical remote sensing image information in the vertical and horizontal directions of the image to obtain discrete slice information, then splice the discrete slice information, and finally convolve the spliced information to obtain the first mapping feature.

[0054] The feature extraction module sequentially performs two-dimensional convolution (Conv2d), normalization (BatchNorm), and activation layer operations on the first mapping features to obtain the second mapping features. The two-dimensional convolution further extracts the target features, and the normalization ensures that the inputs of each neural network layer maintain a consistent distribution. The activation layer operation is implemented using the SiLU activation function.

[0055] The residual module includes several classic residual structures (Bottleneck). The residual module is used to perform a convolution layer operation on the second mapping feature of its input, and add the value obtained after the convolution layer operation to the original value of the second mapping feature to obtain the third mapping feature, thereby achieving the completion of residual feature transfer without increasing the output depth.

[0056] The spatial pyramid pooling module is used to perform multiple maximum pooling operations of varying sizes on the third-map features, and then concatenates the results of these maximum pooling operations to obtain the image features of the optical remote sensing image. The spatial pyramid pooling module primarily addresses the issue of non-uniform input feature map sizes. Most object detection networks typically use a fully connected layer as the final output layer, which requires the input feature map to be of a fixed size. The SPP module, using fixed-block pooling operations, can achieve outputs of the same size for inputs of varying sizes, thus avoiding this issue. Furthermore, the fusion of features of varying sizes within the SPP is beneficial when the sizes of objects in the image to be detected vary significantly.

[0057] The feature fusion network, including the Feature Pyramid Networks (FPN) and Path Aggregation Networks (PAN), is used to fuse image features at different levels of optical remote sensing images. Image features include category features and location features.

[0058] The feature pyramid structure includes, from input to output, a context conversion module, a feature extraction module, and an upsampling module in sequence; the feature output by the context conversion module passes through the feature extraction module and then the upsampling module to obtain a fourth mapping feature, the fourth mapping feature is spliced with the third mapping feature output by the residual module in the backbone feature extraction network to obtain a fifth mapping feature, and the fifth mapping feature is used as the output of the feature pyramid structure; the path aggregation network structure includes, from input to output, an input module, a residual module, a feature extraction module, and a context conversion module in sequence; the input module receives the fifth mapping feature output by the feature pyramid structure, and then passes it through the residual module and the feature extraction module respectively to obtain a sixth mapping feature, the sixth mapping feature is spliced with the output of the feature extraction module in the feature pyramid structure to obtain a seventh mapping feature, the seventh mapping feature is spliced with the output of the feature extraction module in the feature pyramid structure after it passes through the residual module and the feature extraction module in sequence, and the spliced feature is then passed through the context conversion module to obtain an output feature map information set;

[0059] Specifically, the feature fusion network is composed of several feature extraction modules (CBS layers), residual modules (C3), upsampling modules, and C3_CoT modules, forming a feature pyramid network (FPN) and a path aggregation network (PAN). The feature pyramid structure consists of high-level features output by the context transformation module, which are passed through the CBS module and then upsampled. These features are then concatenated with the features output by the third C3 structure (the 8th layer of the network structure) in the backbone feature extraction network. These features are then passed through the C3 and CBS modules, upsampled again, and finally concatenated with the features generated by the second C3 module (the 4th layer of the network structure) in the backbone feature extraction network.

[0060] The path aggregation network consists of a feature pyramid structure that outputs features. These features are passed through the C3 structure and the CBS module, where they are combined with the feature map output by the 16th-layer CBS module. After passing through the C3 and CBS structures, the features are combined with the features output by the 12th-layer CBS module, and then passed through the C3_CoT module. This is primarily used to fuse features from different layers of the feature map.

[0061] As the number of convolutions in a convolutional network increases, the feature hierarchy shifts from low to high levels. Low-level features are closer to the visual content of the image itself, with prominent positional features of large objects and prominent categorical and positional features of small objects within them. High-level features are more abstract and difficult for humans to directly understand, and they contain a richer categorical representation of large objects. This is because as the number of convolutions in a convolutional network increases, the feature hierarchy shifts from low to high levels. Low-level features are closer to the visual content of the image itself, with prominent positional features of large objects and prominent categorical and positional features of small objects within them. High-level features are more abstract and difficult for humans to directly understand, and they contain a richer categorical representation of large objects within them. Therefore, using a feature fusion network can prevent image feature loss. The feature pyramid structure transfers the categorical features of medium and large objects from its high-level modules to the small objects in its low-level modules. The path aggregation network structure transfers the positional features of large objects, small objects, and categorical features from its low-level modules to the medium and small objects in its high-level modules. The two complement each other, overcoming their respective limitations and enhancing the model's feature extraction capabilities. For large, medium and small targets, targets with a size less than 32×32 pixels are considered small targets, targets with a size greater than or equal to 32×32 pixels and less than 96×96 pixels are considered medium targets, and targets with a size greater than or equal to 96×96 pixels are considered large targets. Regarding the high-level modules, according to the direction of information input to output, the module that inputs information first is the high-level module, and the module that inputs information later is the low-level module. The Head module is a detection structure that inputs features of three different sizes into the Detect module and identifies large, medium and small-scale remote sensing targets respectively. This effectively overcomes the limitations of the top features of the CNN network.

[0062] A context conversion module is introduced into the backbone feature extraction network and feature fusion network to ensure the local feature extraction capability while improving the model's global information acquisition capability. It fully utilizes the input context information and guides the learning of the dynamic attention matrix, thereby improving the visual expression capability. The remote sensing target detection loss function is used to improve the accuracy of the remote sensing target recognition prediction box.

[0063] The structure of the context switching module is as follows Figure 2 As shown in the figure, it can be found that the original visual Transformer calculation unit does not fully consider the connection between different spaces, and they are independent of each other. It only learns the paired query key relationship and ignores the rich context between adjacent keys. Therefore, the present invention draws on CoTNet to improve the C3 structure and proposes a context transformation module, which simultaneously realizes the integration function of context information mining and self-attention learning, and promotes self-attention learning by making full use of the context information between targets in adjacent remote sensing images, thereby enhancing the expressive ability of the output feature map.

[0064] The context conversion module first performs context encoding on all adjacent keys in the k×k image grid obtained by segmentation of the remote sensing image for its input feature variable X to obtain a feature matrix K with static context information. 1 ; Then K 1 It is concatenated with the Q space, and then two consecutive 1×1 convolution operations are performed on the concatenated result to obtain the static context attention matrix A. The calculation process is:

[0065] A=[K 1 ,Q]W θ W δ ,

[0066] In the above formula, W θ is the first 1×1 convolution operation matrix, W δ It is the second 1×1 convolution operation matrix.

[0067] Subsequently, the context attention matrix A is multiplied by the 1×1 convolution matrix V to obtain the feature map matrix K with dynamic context information. 2 , and its calculation process is:

[0068] K 2 =Conv 1×1 (V)A,

[0069] Among them, Conv 1×1 (V) represents the matrix V after 1×1 convolution.

[0070] Finally, K 2 With K 1 The fusion is performed to obtain the output matrix Y with global information and local information. The calculation process is expressed as:

[0071] Y=Fusion(K 1 ,K 2 ).

[0072] The backbone network in the target detection model is a key component for extracting hidden information from the input image. However, the original backbone network and the C3 structure in the feature fusion network are fully convolutional structures. Although they have excellent local feature extraction capabilities, they lack the ability to obtain global information. Therefore, in order to enable the model to further improve its global information acquisition capabilities while ensuring local feature extraction capabilities, the present invention introduces a contextual transformation module into the original network model, improves the ResNet structure, and completes the construction of the C3 structure using the concept of CoTNet. This new contextual transformation module enables the network model to acquire global information and improves the detection effect of remote sensing targets.

[0073] The output end is used to evaluate the difference between the output feature map information obtained by the feature fusion network and the real feature map information, and update the parameters of the remote sensing target detection model according to the evaluation results.

[0074] The remote sensing target detection loss function is used to evaluate the difference between the output feature map information obtained by the feature fusion network and the real feature map information. The loss function is positively correlated with the performance of the training model. However, the traditional CIOU loss function is too complicated to measure the aspect ratio, resulting in too slow convergence speed, and the aspect ratio cannot replace the length and width alone. Therefore, the present invention proposes a remote sensing target detection loss function, which solves the problem of large errors in the horizontal and vertical directions of the CIOU loss function, enhances the sensitivity to width and height, and improves the convergence speed and regression accuracy. The remote sensing target detection loss function is realized by calculating the overlap loss, the center distance loss, and the width and height loss, and its formula is:

[0075]

[0076] Where, I EIOU represents the remote sensing target detection loss function, B represents the target real box, B i Represents the target prediction box in the output feature map information, is the ratio of the intersection area and the union area of the target real box and the target prediction box, b and b gt are the center points of the target prediction box and the target real box respectively, ρ is the Euclidean distance between the two center points, c is the diagonal distance of the minimum bounding rectangle covering the target prediction box and the target real box, w and w gt are the lengths of the target prediction box and the target real box, h and h respectively. gt are the widths of the target prediction box and the target real box, respectively, C w and C h are the width and length of the minimum bounding rectangle covering the target prediction box and the target real box, respectively.

[0077] The evaluation of the difference between the output feature map information obtained by the feature fusion network and the real feature map information includes:

[0078] The collected satellite remote sensing image data is treated as a stationary random process. For the output feature map information and the true feature map information, corresponding autoregressive moving average models, namely ARMA models, are established. A first ARMA model and a second ARMA model are obtained, respectively. The coefficients of the two ARMA models are calculated, and their cross-correlation matrices are calculated. The maximum eigenvalue of the cross-correlation matrix is calculated and used to discriminate the difference between the output feature map information and the true feature map information. Simultaneously, the parameters of the remote sensing target detection model are updated based on the maximum eigenvalue.

[0079] The remote sensing target detection model is obtained through the following training steps:

[0080] Get the original image information set;

[0081] Performing labeling and data enhancement processing on the original image information set to obtain a first training image information set; the first training image information set includes a plurality of first training image information;

[0082] determining target training image information based on the first training image information set;

[0083] Calculating a loss function using target training image information and target real image information, and training the first training model using the loss function to obtain a second training model;

[0084] Determine whether the model training parameter information corresponding to the second training model meets the training termination condition, and obtain a termination judgment result;

[0085] When the termination judgment result is no, the first training model is updated by using the second training model, and the step of determining target training image information based on the third training image information set is triggered;

[0086] When the termination judgment result is yes, the second training model is determined to be the remote sensing target detection model.

[0087] The post-processing of the output feature map information set to obtain a target image detection information set includes:

[0088] Performing detection frame decoding processing on the output feature map information set to obtain a detection frame information set; the detection frame information set includes a plurality of detection frame information;

[0089] The detection frame information set is subjected to category discrimination processing to obtain an image category information set; the image category information set includes a plurality of image category information.

[0090] The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0091] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0092] Finally, it should be noted that the remote sensing image target detection method based on feature fusion disclosed in the embodiment of the present invention is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that it is still possible to modify the technical solutions recorded in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A remote sensing image target detection method based on feature fusion, characterized in that: The method comprises: Acquiring optical remote sensing image information; the optical remote sensing image information includes a plurality of optical remote sensing images; Performing positioning and recognition processing on the optical remote sensing image information using a remote sensing target detection model to obtain an output feature map information set; the output feature map information set includes a plurality of output feature map information; Post-processing the output feature map information set to obtain a target image detection information set; the target image detection information set includes a plurality of target image detection information; The remote sensing target detection model includes an input end, a backbone feature extraction network, a feature fusion network and an output end in sequence from input to output; The backbone feature extraction network includes a downsampling module, a feature extraction module, a residual module, and a spatial pyramid pooling module; the backbone feature extraction network is used to extract features of optical remote sensing image information; The downsampling module is used to perform interval slicing operations on the pre-processed optical remote sensing image information in the vertical and horizontal directions of the image to obtain discrete slice information, then splice the discrete slice information, and finally convolve the spliced information to obtain the first mapping feature; The feature extraction module is used to perform two-dimensional convolution, normalization and activation layer operations on the first mapping feature in sequence to obtain a second mapping feature; The residual module includes several classic residual structures. The residual module is used to perform a convolution layer operation on the second mapping feature of its input, and add the value obtained after the convolution layer operation to the original value of the second mapping feature to obtain a third mapping feature, thereby achieving the completion of residual feature transfer without increasing the output depth; The spatial pyramid pooling module is used to perform multiple maximum pooling operations of different sizes on the third mapping feature, and then perform a splicing operation on the results of the maximum pooling operation to obtain image features of the optical remote sensing image; The feature fusion network includes a feature pyramid structure and a path aggregation network structure, which is used to achieve the fusion of image features at different levels of optical remote sensing images; The feature pyramid structure includes, from input to output, a context conversion module, a feature extraction module, and an upsampling module in sequence; the feature output by the context conversion module passes through the feature extraction module and then the upsampling module to obtain a fourth mapping feature, the fourth mapping feature is spliced with the third mapping feature output by the residual module in the backbone feature extraction network to obtain a fifth mapping feature, and the fifth mapping feature is used as the output of the feature pyramid structure; the path aggregation network structure includes, from input to output, an input module, a residual module, a feature extraction module, and a context conversion module in sequence; the input module receives the fifth mapping feature output by the feature pyramid structure, and then passes it through the residual module and the feature extraction module respectively to obtain a sixth mapping feature, the sixth mapping feature is spliced with the output of the feature extraction module in the feature pyramid structure to obtain a seventh mapping feature, the seventh mapping feature is spliced with the output of the feature extraction module in the feature pyramid structure after it passes through the residual module and the feature extraction module in sequence, and the spliced feature is then passed through the context conversion module to obtain an output feature map information set; The method comprises: The input end is used to receive the acquired optical remote sensing image information and pre-process it; Preprocessing is to process the acquired optical remote sensing image information using data enhancement methods, and then use adaptive image scaling methods to unify the sizes of all optical remote sensing images.

2. The method for remote sensing image target detection based on feature fusion according to claim 1, wherein: The method comprises: The context conversion module simultaneously realizes the integration function of context information mining and self-attention learning, and promotes self-attention learning by making full use of the context information between the targets of adjacent remote sensing images, thereby enhancing the expressive power of the output feature map. The context conversion module performs context encoding on all adjacent keys in the k×k image grid obtained by segmentation of the remote sensing image to obtain a feature matrix K with static context information. 1 , K 1 It is concatenated with the Q space, and then two consecutive 1×1 convolution operations are performed on the concatenated result to obtain the static context attention matrix A. The calculation process is: A=[K 1 ,Q]W θ W δ , In the above formula, W θ is the first 1×1 convolution operation matrix, W δ is the second 1×1 convolution operation matrix; For the context attention matrix A, it is multiplied by the matrix V after 1×1 convolution to obtain the feature map matrix K with dynamic context information 2 , and its calculation process is: K 2 =Conv 1×1 (V)A, Among them, Conv 1×1 (V) represents the matrix V after 1×1 convolution; K 2 With K 1 The fusion is performed to obtain the output matrix Y with global information and local information.

3. The remote sensing image target detection method based on feature fusion according to claim 1, characterized in that: The method comprises: The output end is used to evaluate the difference between the output feature map information obtained by the feature fusion network and the real feature map information, and update the parameters of the remote sensing target detection model according to the evaluation results; The evaluation of the difference between the output feature map information obtained by the feature fusion network and the real feature map information includes: The collected satellite remote sensing image data is regarded as a stationary random process. For the output feature map information and the real feature map information, corresponding autoregressive-sliding average models, namely ARMA models, are established respectively to obtain the first ARMA model and the second ARMA model respectively. The coefficients of the two ARMA models are calculated, and their cross-correlation matrices are calculated. The maximum eigenvalue of the cross-correlation matrix is calculated, and the maximum eigenvalue is used to distinguish the difference between the output feature map information and the real feature map information; according to the maximum eigenvalue, the parameters of the remote sensing target detection model are updated.

4. The method for remote sensing image target detection based on feature fusion according to claim 3, wherein: The method comprises: The remote sensing target detection loss function is used to evaluate the difference between the output feature map information obtained by the feature fusion network and the real feature map information.

5. The method for remote sensing image target detection based on feature fusion according to claim 4, wherein: The method comprises: The remote sensing target detection loss function is obtained by calculating the overlap loss, the center distance loss, and the width and height loss. Its formula is: Where, I EIOU represents the remote sensing target detection loss function, B represents the target real box, B i Represents the target prediction box in the output feature map information, is the ratio of the intersection area and the union area of the target real box and the target prediction box, b and b gt are the center points of the target prediction box and the target real box respectively, ρ is the Euclidean distance between the two center points, c is the diagonal distance of the minimum bounding rectangle covering the target prediction box and the target real box, w and w gt are the lengths of the target prediction box and the target real box, h and h respectively. gt are the widths of the target prediction box and the target real box, respectively, C w and C h are the width and length of the minimum bounding rectangle covering the target prediction box and the target real box, respectively.

Images