An image processing method and system for remote sensing target detection scenes
By adopting digital processing, composite feature extraction, embedded attention mechanism and upsampled feature extraction methods in remote sensing image processing, the problem of small-sized object detection in remote sensing images is solved, the accuracy and efficiency of object detection are improved, and the feature expression ability is enhanced .
Patent Information
- Application Number
- CN202411113594.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The prior art is difficult to accurately and quickly detect small-sized objects in remote sensing images, especially in drone remote sensing application scenarios, resulting in low target detection accuracy and efficiency.
An image processing method is adopted, including digital processing, composite feature extraction, embedded attention mechanism feature extraction, rapid spatial pyramid pooling and upsampled feature extraction, and feature maps are extracted through multi-scale fusion to enhance feature expression capabilities and suppress complex irrelevant information. Interference, improve detection accuracy and efficiency.
By extending the feature extraction backbone network, exposing multiple feature maps at different scales, enhancing feature fusion, improving the detection capability of small targets, and improving the target detection accuracy and efficiency in drone remote sensing application scenarios.
Smart Images

Figure CN118644666B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and system for remote sensing target detection scenes. Background Art
[0002] Target detection technology is one of the important issues in the field of computer vision and a key link in the development of remote sensing technology today. It plays an important role in military, meteorological monitoring, environmental protection, urban planning and other fields. With the rapid development of remote sensing technology and the diversification of data acquisition methods, how to effectively process and utilize these large amounts of remote sensing data has become a research hotspot. Among them, multi-information fusion and multiplexing technology for remote sensing target detection scenarios improves the accuracy and robustness of target detection by combining information from different layers of neural network sources.
[0003] Remote sensing images usually cover large geographical areas, ranging from cities and farmlands to pristine natural environments, which may contain confusing terrain. In addition, there are many types of objects in remote sensing images, including various geographical elements such as vehicles and buildings. The shapes and sizes of these objects vary greatly with the shooting altitude and meteorological conditions, and their edge features are difficult to identify due to similar backgrounds. A large number of small objects may be stacked together, resulting in aggregation and occlusion. This poses a great challenge to target detection in remote sensing scenes.
[0004] With the vigorous development of remote sensing related technologies, target detection in high-resolution remote sensing images has become a research focus in the field of computer vision. Among these related technologies, deep learning has become an important way of image processing due to its high precision and high efficiency. As a classic network in deep learning, deep convolutional neural network has been used for target detection in high-resolution remote sensing images.
[0005] It is worth noting that in remote sensing images, in addition to normal-sized objects, there are always a large number of small-sized objects, which are characterized by ratios less than 0.1 or bounding boxes with resolutions less than 32×32 pixels. This brings great challenges to accurately and quickly detecting it in practical applications. Summary of the invention
[0006] The purpose of the present invention is to provide an image processing method and system for remote sensing target detection scenarios, so as to solve the existing problem that it is difficult to accurately and quickly detect small-sized objects.
[0007] To achieve the above-mentioned purpose, the present invention provides the following technical scheme: an image processing method for remote sensing target detection scenes, the processing method comprising the following steps: S1, digitizing the original image to form a digitized feature map; S2, using the digitized feature map as the initial input, and outputting multiple intermediate multidimensional feature maps and a final multidimensional feature map through composite feature extraction processing; S3, using the final multidimensional feature map as the initial input, and outputting multiple intermediate global feature maps and a final global feature map through feature extraction processing embedded with an attention mechanism; S4, performing fast spatial pyramid pooling processing on the final global feature map to form a standard feature map; S5, using the standard feature map as the initial input, and outputting multiple intermediate global feature maps and a final global feature map through feature extraction processing embedded with an attention mechanism; Sampling feature extraction processing, outputting multiple intermediate deep feature maps and a final deep feature map; S6, performing multi-scale fusion on the final multidimensional feature map, the intermediate multidimensional feature map, the intermediate global feature map, the intermediate deep feature map or the final deep feature map with the same spatial size to form a fused feature map; S7, extracting different detection results according to different fused feature maps; wherein the extraction rules of the composite feature extraction processing, the feature extraction processing embedded with the attention mechanism and the upsampling feature extraction processing all include: based on the input feature map, outputting the extracted output feature map, the input feature map includes the initial input feature map and the intermediate feature map, and the intermediate feature map includes the output feature map obtained based on its last input image.
[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: the processing method in the present application, on the one hand, extends the feature extraction backbone network through upsampling feature extraction processing, and exposes multiple feature maps of different scales to the feature extraction process; on the other hand, the feature map source during multi-scale fusion is increased through upsampling feature extraction processing, wherein the feature map extracted through upsampling feature extraction processing can become a good input source for the feature fusion part, so that shallow features and deep features are integrated, and the feature expression ability of the processing method is enhanced; in addition, the feature extraction processing embedded with the attention mechanism enables the processing method to focus more on more important feature information and suppress the interference of complex irrelevant information on the detection process. This improves the accuracy and efficiency of target detection in the application scenario of UAV remote sensing.
[0009] Preferably, the above S2 specifically includes the following steps:
[0010] S21: performing convolution processing on the digitized feature map twice in succession, and then performing C3 processing once to form a first intermediate multi-dimensional feature map;
[0011] S22: performing convolution processing on the first intermediate multidimensional feature map, and then performing C3 processing twice in succession to form a second intermediate multidimensional feature map;
[0012] S23: Perform convolution processing on the second intermediate multi-dimensional feature map to form a final multi-dimensional feature map.
[0013] Preferably, the C3 process in S21 and S22 above comprises the following steps:
[0014] S201, splitting the input feature map into a first input feature map and a second input feature map along the channel dimension;
[0015] S202, performing CBS processing on the first input feature map, and then performing multiple sequentially connected bottleneck processing to form a first branch feature map;
[0016] S203, performing CBS processing on the second input feature map to form a second branch feature map;
[0017] S204, concatenating the first branch feature map and the second branch feature map to form a concatenated feature map;
[0018] S205. Perform CBS processing on the concatenated feature map to form an output feature map.
[0019] Preferably, the above S3 specifically includes the following steps:
[0020] S31: Perform C3 processing of embedding space and channel attention three times continuously on the final multidimensional feature map to form a first intermediate global feature map;
[0021] S32: performing convolution processing on the first intermediate global feature map to form a second intermediate global feature map;
[0022] S33: Perform C3 processing of embedding space and channel attention on the second intermediate global feature map to form the final global feature map.
[0023] Preferably, the above S5 specifically includes the following steps:
[0024] S51, performing convolution processing on the standard feature map to form a first intermediate deep feature map;
[0025] S52, performing upsampling processing on the first intermediate deep feature map, and then performing convolution processing to form a second intermediate deep feature map;
[0026] S53, performing upsampling processing on the second intermediate deep feature map, and then performing convolution processing to form a third intermediate deep feature map;
[0027] S54, performing upsampling processing on the third intermediate deep feature map, and then performing convolution processing to form a fourth intermediate deep feature map;
[0028] S55, performing convolution processing on the fourth intermediate deep feature map to form a fifth intermediate deep feature map;
[0029] S56, performing convolution processing on the fifth intermediate deep feature map twice in succession to form a sixth intermediate deep feature map;
[0030] S57. Upsample the sixth intermediate deep feature map to form a final deep feature map.
[0031] Preferably, the above S6 specifically includes the following steps:
[0032] S61, performing multi-scale fusion on the final deep feature map, the first intermediate global feature map, the second intermediate deep feature map, and the fifth intermediate deep feature map to form a first fused feature map;
[0033] S62, performing C3 processing of embedding space and channel attention on the first fused feature map to form a first auxiliary feature map;
[0034] S63, performing convolution processing on the first auxiliary feature map to form a second auxiliary feature map;
[0035] S64, performing upsampling processing on the second auxiliary feature map to form a third auxiliary feature map;
[0036] S65, performing multi-scale fusion on the third auxiliary feature map, the third intermediate deep feature map, the fourth intermediate deep feature map, and the second intermediate multi-dimensional feature map to form a second fused feature map;
[0037] S66, performing C3 processing of embedding space and channel attention on the second fused feature map to form a fourth auxiliary feature map;
[0038] S67, performing convolution processing on the fourth auxiliary feature map to form a fifth auxiliary feature map;
[0039] S68, performing multi-scale fusion on the fifth auxiliary feature map, the second auxiliary feature map, the final multi-dimensional feature map, and the second intermediate deep feature map to form a third fused feature map;
[0040] S69, performing C3 processing of embedding space and channel attention on the third fused feature map to form a sixth auxiliary feature map;
[0041] S610, performing convolution processing on the sixth auxiliary feature map to form a seventh auxiliary feature map;
[0042] S611, performing multi-scale fusion on the seventh auxiliary feature map, the sixth intermediate deep feature map, the second intermediate global feature map, and the first intermediate deep feature map to form a fourth fused feature map;
[0043] S612: Perform convolution processing on the fourth fused feature map to form an eighth auxiliary feature map.
[0044] Preferably, the multi-scale fusion method in S6 is:
[0045]
[0046] Where: Represents a splicing operation; Represents n different input feature maps; Indicates that the feature map is transformed along the channel dimension Concatenate into the output feature map.
[0047] Based on another object of the present invention, the present invention provides the following technical solution: an image processing system for remote sensing target detection scenes, the processing system using the above-mentioned processing method, including:
[0048] A digitization module, used for digitizing the original image to form a digitized feature map;
[0049] A composite feature extraction module is used to take the digitized feature map as the initial input, and output a plurality of intermediate multi-dimensional feature maps and a final multi-dimensional feature map through composite feature extraction processing;
[0050] A feature extraction module embedded with an attention mechanism is used to take the final multi-dimensional feature map as an initial input, and output multiple intermediate global feature maps and a final global feature map through feature extraction processing embedded with the attention mechanism;
[0051] The spatial pyramid pooling module is used to perform fast spatial pyramid pooling on the final global feature map to form a standard feature map;
[0052] An upsampling feature extraction processing module is used to take the standard feature map as the initial input, and output multiple intermediate deep feature maps and a final deep feature map through upsampling feature extraction processing;
[0053] A fusion module, used for performing multi-scale fusion of a final multi-dimensional feature map, an intermediate multi-dimensional feature map, an intermediate global feature map, an intermediate deep feature map or a final deep feature map with the same spatial size to form a fused feature map;
[0054] The output module is used to extract different detection results based on different fusion feature maps.
[0055] Based on another purpose of the present invention, the present invention provides the following technical solution: a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.
[0056] Based on another object of the present invention, the present invention provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program implements the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 A schematic flow chart of an image processing method for remote sensing target detection scenarios;
[0058] Figure 2 is a system structure diagram of the present invention;
[0059] Figure 3 is a schematic flow chart of the feature map extraction process;
[0060] Figure 4 It is a schematic flow chart of the feature map fusion process;
[0061] Figure 5 Schematic flow chart of C3 processing and C3 processing embedded with channel attention and spatial attention;
[0062] Figure 6 Schematic diagram of Sigmoid activation function, ReLU activation function and SiLU activation function.
[0063] In the figure: 10, processing system; 110, digitization module; 120, composite feature extraction module; 130, feature extraction module embedded with attention mechanism; 140, spatial pyramid pooling module; 150, upsampling feature extraction processing module; 160, fusion module; 170, output module. DETAILED DESCRIPTION
[0064] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0065] In this article, the terms "upper, lower, inside, outside" and the like are established based on the positional relationships shown in the drawings. The corresponding positional relationships may also change depending on the drawings, and therefore cannot be understood as absolute limitations on the scope of protection; moreover, relational terms such as "first" and "second" and the like are merely used to distinguish one component from another with the same name, and do not necessarily require or imply any actual relationship or order between these components.
[0066] Embodiment 1
[0067] like Figure 1 , Figure 3 and Figure 4As shown, this embodiment provides an image processing method for remote sensing target detection scenarios, and the processing method includes the following steps: S1, digitally processing the original image to form a digital feature map; the original image refers to the remote sensing image, and the remote sensing image specifically refers to the aerial image and the satellite image. There is no specific restriction on the source of the original image. In this embodiment, the original image comes from the aerial image taken by the drone. The original image is digitally processed, and the digital feature map is stored in the tensor matrix. Before the original image enters the subsequent feature extraction process, the size needs to be adjusted to the size set by the neural network. The specific adjustment method uses the bilinear interpolation method, and its specific expression is:
[0068]
[0069] Where: and are the coordinates in the adjusted image, and is the coordinate in the original image corresponding to the new coordinate and The coordinates of the four nearest pixels, The original image is The pixel value of the point, is the image after resizing After resizing the original image to the input size, in order to make the model easier to learn, the pixel value of the original image is converted from the range of [0,255] to [0,1]. The specific adjustment method is:
[0070]
[0071] Where: P is the pixel value of the original image, is the adjusted pixel value. After the original image has adjusted its size and pixel value, a digital feature map is formed. Subsequent feature extraction and analysis operations are performed on the basis of the digital feature map. In this embodiment, the spatial size of the original image can be any value, and the bilinear interpolation method is used to change its spatial size to a set value. The set value of the network can be adjusted according to the actual task, but its width and height must remain consistent. Preferably, in this embodiment, considering comprehensive factors such as high-resolution original images and computational complexity, its width and height can be set to 640 pixels.
[0072] S2. Using the digitized feature map as the initial input, multiple intermediate multidimensional feature maps and a final multidimensional feature map are output through composite feature extraction processing; the composite feature extraction processing includes several convolution processes and several C3 processes. During the composite feature extraction process, the output feature map retains the shallow features of the input feature map and combines its deep features, providing more dimensional information input for subsequent feature extraction and fusion operations, thereby enhancing the feature expression capability of the processing method.
[0073] S3. Using the final multi-dimensional feature map as the initial input, multiple intermediate global feature maps and the final global feature map are output through feature extraction processing embedded with the attention mechanism. The feature extraction processing embedded with the attention mechanism includes several convolution processing and several C3 processing (C3CBAM processing) embedded with channel attention and spatial attention. In the process of feature extraction, after the channel attention calculates and adjusts the channel weight information, the spatial attention calculates the weight of each spatial position to adjust the spatial information of the feature map. The feature map is output after the channel attention adjusts the channel weight and the spatial attention adjusts the spatial position weight. The above operations make the C3 processing embedded with channel attention and spatial attention have better representation capabilities. The feature extraction processing embedded with the attention mechanism can learn local detail features and global features, improve the meticulousness and accuracy of image processing, and make the processing method suitable for tasks that require high attention segmentation such as remote sensing target detection.
[0074] S4. Perform fast spatial pyramid pooling on the final global feature map to form a standard feature map; the spatial pyramid pooling is used to improve the receptive field of the processing method while maintaining computational efficiency, especially when processing the input feature map, it can capture a wider range of contextual information without significantly increasing the computational burden. The fast spatial pyramid pooling in this embodiment is optimized, and the fast spatial pyramid pooling in this embodiment uniformly uses a 5×5 pooling window and a smaller step size, and the pooling layer in the fast spatial pyramid pooling in this embodiment is connected in series with a residual structure. Compared with the traditional spatial pyramid pooling that uses multiple pooling windows (such as 5×5, 9×9, 13×13) and a pooling layer parallel residual structure, the fast spatial pyramid pooling in this embodiment can improve computational efficiency while fixing the receptive field.
[0075] S5. Using the standard feature map as the initial input, multiple intermediate deep feature maps and a final deep feature map are output through upsampling feature extraction processing; after the final global feature map is processed by the fast spatial pyramid pooling process in this embodiment, a standard feature map with a size of 20×20×512 is formed, and upsampling feature extraction processing is performed on the standard feature map to further extract the feature information in the input feature map. The feature extraction backbone network is extended through the upsampling feature extraction processing, and multiple feature maps of different scales are exposed to the feature extraction process. The intermediate deep feature map and the final deep feature map obtained by the upsampling feature extraction processing can be used as a good source of feature maps for subsequent fusion steps, which can enhance the feature expression ability of the processing method.
[0076] S6. Perform multi-scale fusion on the final multi-dimensional feature map, the intermediate multi-dimensional feature map, the intermediate global feature map, the intermediate deep feature map or the final deep feature map with the same spatial size to form a fused feature map; by setting the upsampling feature extraction processing step, the processing method adds more feature maps that can be fused with each other. The intermediate deep feature map and the final deep feature map obtained by the upsampling feature extraction processing are both good sources for generating fused feature maps, so that the processing method can capture a variety of features from low-level to high-level, which is particularly important for multi-scale and multi-texture target detection in the original image. Through multi-scale fusion, the processing method can more effectively capture these fine-grained features, thereby improving the detection capability of small targets.
[0077] S7. Extract detection results of different spatial sizes according to fused feature maps of different spatial sizes; wherein the extraction rules of the composite feature extraction processing, the feature extraction processing embedded with the attention mechanism and the upsampling feature extraction processing all include: based on the input feature map, outputting the extracted output feature map, the input feature map includes the initial input feature map and the intermediate feature map, and the intermediate feature map includes the output feature map obtained based on its last input image. In this embodiment, the specific processing methods adopted by the composite feature extraction processing, the feature extraction processing embedded with the attention mechanism, and the upsampling feature extraction processing are all different. The composite feature extraction processing includes several feature extraction processings performed in sequence, and the above feature extraction processing includes convolution processing and C3 processing. Among the above several feature extraction processings, the last feature extraction processing outputs a final multidimensional feature map, and the middle feature extraction processing outputs an intermediate multidimensional feature map; the feature extraction processing embedded with the attention mechanism also includes several feature extraction processings performed in sequence, and the above feature extraction processing includes convolution processing and C3 processing embedded with channel attention and spatial attention. Among the above several feature extraction processings, the last feature extraction processing outputs a final global feature map, and the middle feature extraction processing outputs an intermediate global feature map; the feature extraction processing of the upsampling feature extraction processing also includes several feature extraction processings performed in sequence, and the above feature extraction processing includes convolution processing and upsampling processing. Among the above several feature extraction processings, the last feature extraction processing outputs a final deep feature map, and the middle feature extraction processing outputs an intermediate deep feature map.
[0078] Specifically, the processing method in this embodiment, on the one hand, extends the feature extraction backbone network through upsampling feature extraction processing, and exposes multiple feature maps of different scales to the feature extraction process. On the other hand, the feature map source during multi-scale fusion is increased through upsampling feature extraction processing, wherein the feature map extracted through upsampling feature extraction processing can become a good input source for the feature fusion part, so that shallow features and deep features are integrated, and the feature expression ability of the processing method is enhanced; in addition, the feature extraction processing embedded with the attention mechanism enables the processing method to focus more on more important feature information and suppress the interference of complex irrelevant information on the detection process. This improves the accuracy and efficiency of target detection in the application scenario of UAV remote sensing.
[0079] Furthermore, the above S2 specifically includes the following steps: S21: performing convolution processing on the digitized feature map twice in a row, and then performing C3 processing once to form a first intermediate multidimensional feature map; S22: performing convolution processing on the first intermediate multidimensional feature map, and then performing C3 processing twice in a row to form a second intermediate multidimensional feature map; S23: performing convolution processing on the second intermediate multidimensional feature map to form a final multidimensional feature map. In the process of composite feature extraction, the output feature map retains the shallow features of the input feature map and combines its deep features, providing more dimensional information input for subsequent feature extraction and fusion operations, and enhancing the feature expression ability of the processing method. Each processing step in the above S21, S22 and S23 outputs an intermediate multidimensional feature map. In this embodiment, in order to avoid the description of the steps of the composite feature extraction process being too lengthy, multiple feature processing steps are written into one step, which should not be understood as violating the processing rules of the composite feature extraction process.
[0080] like Figure 3 As shown, further, the C3 processing in the above S21 and S22 includes the following steps:
[0081] S201, split the input feature map into a first input feature map and a second input feature map along the channel dimension; in this embodiment, the input feature map is evenly split into the first input feature map and the second input feature map on the channel. Of course, the split ratio can also be changed according to actual task requirements. The split operation in this embodiment is implemented by a simple index operation, and does not involve calculation or complex data processing;
[0082] S202, perform CBS processing on the first input feature map, and then perform multiple bottleneck processes connected in sequence to form a first branch feature map; the CBS processing in this embodiment is used to improve the expression of features and introduce nonlinear mapping. The CBS processing includes three steps: convolution processing, batch normalization, and function activation. In the convolution processing of the CBS processing in this embodiment, the size of the convolution kernel is preferably k=1, and the step size is preferably s=1. In the batch normalization of the CBS processing in this embodiment, BatchNorm2d is used to normalize the data, which prevents the performance from being unstable due to excessive data before function activation. The calculation formula of BatchNorm2d is:
[0083]
[0084] Where: It means that the mean is calculated along the channel dimension of the input feature map x. Indicates that the variance is calculated along the channel dimension of x, It is a fixed value added to prevent the calculation value from being unstable. , γ and β are affine parameters. The normalized data is scaled again to get new data. Both γ and β are coefficient matrices that can be learned. The mean and variance The expression is:
[0085]
[0086]
[0087] Where: represents the nth sample and the channel is the pixel value at position (i, j) of c, N represents the batch size, H and W represent the height and width in the spatial scale, and Represent the mean and variance of the cth channel respectively;
[0088] After convolution and batch normalization, the SiLU activation function needs to be introduced to further process the data. Its main purpose is to introduce nonlinearity to the processing method to make it better adapt to complex data and determine whether neurons should be activated, while helping the network to converge faster. The SiLU activation function provides a smooth activation function form with non-zero derivatives for all input values, which means that it remains active even when negative inputs are input, thus avoiding the problem of gradient vanishing. At the same time, the smoothness of SiLU helps to improve the efficiency of the gradient descent optimization algorithm because it can provide stable gradient values. In addition, the non-monotonicity of SiLU can help the module capture more complex information and provide better performance in remote sensing target detection. The calculation formula of the SiLU activation function is:
[0089]
[0090] Where: It is the standard Sigmoid activation function, and its calculation formula is:
[0091]
[0092] Where: The Sigmoid function performs nonlinear mapping on each input element and compresses it between (0, 1) to achieve normalization. The SiLU function has smoother gradients and stronger nonlinear characteristics, which is more helpful in improving the training speed and performance of the model.
[0093] The SiLU activation function is often used to replace the traditional ReLU activation function in dealing with deep learning problems, such as Figure 6As shown, the SiLU function has a smoother gradient and stronger nonlinear characteristics, which is more helpful to improve the training speed and performance of the processing method. The calculation formula of the ReLU activation function is:
[0094]
[0095] Where: b is the input tensor matrix. The ReLU function makes the negative gradient of the input 0 and retains the positive value of the input, which will cause the sparsity of the network and alleviate the gradient vanishing problem. However, the SiLU activation function is smoother and has derivatives in the entire definition domain, which is more conducive to optimizing the model than the ReLU function.
[0096] Bottleneck processing is Bottleneck processing. Bottleneck processing increases the depth of the network, allowing more complex and abstract feature expressions to be learned. Bottleneck processing specifically includes three processing steps: compression, bottleneck and expansion. First, the input feature map is subjected to 1×1 convolution processing to reduce the number of channels, reduce the computational cost and the number of parameters to complete the compression step. Secondly, the compressed feature map enters the bottleneck structure containing a 3×3 convolution kernel to extract higher-level feature representations. Finally, after passing through the bottleneck structure, the number of feature map channels is expanded back to the initial number of parameters through 1×1 convolution processing. S203, CBS processing is performed on the second input feature map to form a second branch feature map; S204, the first branch feature map and the second branch feature map are spliced to form a spliced feature map; the first branch feature map and the second branch feature map are connected through the concat splicing residual of the two channels.
[0097] S205, CBS processing is performed on the concatenated feature map to form an output feature map. The first branch feature map and the second branch feature map are concatenated by Concat at the end of the CBS processing. Concat concatenation is also merged in the channel dimension. This residual connection method enables the output feature map to retain its shallow features and combine its deep features. The merged feature map passes through a CBS unit again to further unify and optimize the concatenated feature map, providing more dimensional information input for subsequent feature extraction.
[0098] Furthermore, the above S3 specifically includes the following steps:
[0099] S31: The final multidimensional feature map is subjected to three consecutive C3 processings of embedding space and channel attention to form a first intermediate global feature map; the C3 processing of embedding space and channel attention has the same model structure as different C3 processing models, but after the expansion step of the bottleneck processing, channel attention processing and spatial attention processing are embedded. The channel attention calculates the weight of each channel of the feature map to adjust the channel information of the feature map. The channel weight is obtained by global average pooling operation and full connection calculation. The feature map will not change size after channel attention, but the weight of each channel will be fine-tuned according to its importance, where the calculation formula for channel weight adjustment is:
[0100]
[0101] Where: d is the input feature map, AvgPool is the global average pooling operation, which performs average pooling on each channel of the input feature map. For each channel, the values of all spatial dimensions are averaged, so each channel corresponds to a single scalar value. Global average pooling provides a form of feature compression, which captures the global statistical characteristics of the channel by compressing spatial information into a numerical value, which helps the model understand which channels contain more important information. ReLU is the activation function, W1 and W2 are the weights of the fully connected layer, and the calculation formula of AvgPool(d) is:
[0102]
[0103] Where: H, W represent the height and width of the feature map, AvgPool(d)c represents the average pooling result of feature map d on the cth channel, Represents the pixel value of the feature map d on the cth channel and at position (i, j);
[0104] After the feature map is adjusted through channel attention calculation, the spatial attention calculates the weight of each spatial position to adjust the spatial information of the feature map. The feature map is output after the channel attention adjusts the channel weight and the spatial attention adjusts the spatial position weight. It is then spliced with the second branch feature map of the C3 process embedded in space and channel attention in the channel dimension. The merged feature map is then processed by a CBS module to further integrate and optimize the features to obtain the final output feature map of the C3 process embedded in space and channel attention. Due to the embedding of channel and spatial attention, the representation ability of the C3 process embedded in space and channel attention is enhanced, allowing the processing method to learn local details and global information. The spatial weight is obtained by performing maximum pooling and average pooling operations on the feature map after the channel attention output and performing convolution processing. The feature map size will not change after the spatial attention output, but the weight of each spatial position will be adjusted according to its importance. The calculation formula for spatial attention is:
[0105]
[0106] Where: is the input feature map, MaxPool and AvgPool are the maximum pooling and average pooling operations, Concat is the concatenation operation, W3 and W4 are the weights of the convolution processing, where The calculation formula is:
[0107]
[0108] Where: represents the input feature map, represents the maximum pooling result on the cth channel, located in the i-th row and the j-th column, It represents the pixel value at the i×s+mth row, j×s+nth column, and cth channel on the input feature map d. m and n represent the row index and column index of the window respectively. Indicates selecting the maximum value along the row direction within the window. m is the row index within the window, ranging from 0 to k-1, and k is the size of the window. Indicates selecting the maximum value along the column direction in the window, n is the column index in the window, and the range is also from 0 to k-1. The two maximum values are calculated along the row and column directions of the window respectively. The maximum value of each row and column is selected in the window, and then these maximum values are combined to obtain the maximum value in the window, which realizes the maximum value pooling of each channel in the feature map in the window. In spatial attention, the MaxPool operation is used to calculate the maximum value of each channel in the feature map. This operation helps to capture the most significant features of each channel in the feature map, thereby highlighting important spatial location information. The calculation formula of the two-channel Concat operation that splices the calculation results of MaxPool and AvgPool is as follows:
[0109]
[0110] Where: It means that feature maps A and B are stacked along the dimension of channel C to form a new feature map with C channels. A +C B , the spatial dimension remains unchanged. S32: Convolution processing is performed on the first intermediate global feature map to form a second intermediate global feature map; S33: C3 processing of embedding spatial and channel attention is performed on the second intermediate global feature map to form the final global feature map. In the process of feature extraction, after the channel attention calculates and adjusts the channel weight information, the spatial attention calculates the weight of each spatial position to adjust the spatial information of the feature map. The feature map is output after the channel attention adjusts the channel weight and the spatial attention adjusts the spatial position weight. This makes the C3 processing embedded with channel attention and spatial attention have better representation capabilities, and the feature extraction processing embedded with the attention mechanism can learn local detail features and global features.
[0111] Further, the above S5 specifically includes the following steps: S51, convolution processing is performed on the standard feature map to form a first intermediate deep feature map; S52, upsampling processing is performed on the first intermediate deep feature map, and then convolution processing is performed to form a second intermediate deep feature map; Figure 4As shown in the figure, the upsampling process reconstructs the feature extraction process of the processing method, changing the original bottom-up feature extraction process into a bottom-up and then top-down feature extraction process, which enhances the processing method's ability to fuse and reuse information. The upsampling operation uses the nearest neighbor method to expand the spatial dimension of the data. The nearest neighbor method is a non-parametric method that increases the size of the image by directly copying the pixel values of the nearest neighbors. For example, when the upsampling factor is 2, the size of the output image is twice that of the input image. For each pixel in the input image, upsampling will create a 2×2 pixel block in the output image, and each pixel value in this block will be assigned the same value as the current pixel in the input image. S53, upsampling the second intermediate deep feature map, and then performing convolution processing to form a third intermediate deep feature map; S54, upsampling the third intermediate deep feature map, and then performing convolution processing to form a fourth intermediate deep feature map; S55, convolution processing is performed on the fourth intermediate deep feature map to form a fifth intermediate deep feature map; S56, convolution processing is performed twice on the fifth intermediate deep feature map to form a sixth intermediate deep feature map; S57, upsampling is performed on the sixth intermediate deep feature map to form a final deep feature map. By setting the upsampling feature extraction processing step, the processing method adds more feature maps that can be fused with each other. The intermediate deep feature map and the final deep feature map obtained by the upsampling feature extraction processing are both good sources for generating fused feature maps, so that the processing method can capture a variety of features from low-level to high-level, which is particularly important for multi-scale and multi-texture target detection in the original image. Through multi-scale fusion, the processing method can more effectively capture these fine-grained features, thereby improving the detection capability of small targets. Each processing step in the above S52, S53 and S54 outputs an intermediate multi-dimensional feature map. In this embodiment, in order to avoid overly lengthy description of the upsampling feature extraction processing steps, multiple feature processing steps are written into one step. This should not be understood as violating the processing rules of the upsampling feature extraction processing.
[0112] like Figure 5 As shown, further, the above S6 specifically includes the following steps:
[0113] S61, performing multi-scale fusion on the final deep feature map, the first intermediate global feature map, the second intermediate deep feature map, and the fifth intermediate deep feature map to form a first fused feature map;
[0114] S62, performing C3 processing of embedding space and channel attention on the first fused feature map to form a first auxiliary feature map;
[0115] S63, performing convolution processing on the first auxiliary feature map to form a second auxiliary feature map;
[0116] S64, performing upsampling processing on the second auxiliary feature map to form a third auxiliary feature map;
[0117] S65, performing multi-scale fusion on the third auxiliary feature map, the third intermediate deep feature map, the fourth intermediate deep feature map, and the second intermediate multi-dimensional feature map to form a second fused feature map;
[0118] S66, performing C3 processing of embedding space and channel attention on the second fused feature map to form a fourth auxiliary feature map;
[0119] S67, performing convolution processing on the fourth auxiliary feature map to form a fifth auxiliary feature map;
[0120] S68, performing multi-scale fusion on the fifth auxiliary feature map, the second auxiliary feature map, the final multi-dimensional feature map, and the second intermediate deep feature map to form a third fused feature map;
[0121] S69, performing C3 processing of embedding space and channel attention on the third fused feature map to form a sixth auxiliary feature map;
[0122] S610, performing convolution processing on the sixth auxiliary feature map to form a seventh auxiliary feature map;
[0123] S611, performing multi-scale fusion on the seventh auxiliary feature map, the sixth intermediate deep feature map, the second intermediate global feature map, and the first intermediate deep feature map to form a fourth fused feature map;
[0124] S612, convolution processing is performed on the fourth fused feature map to form an eighth auxiliary feature map. In this embodiment, the first auxiliary feature map can obtain a 160×160 detection result through convolution processing, the fourth auxiliary feature map can obtain an 80×80 detection result through convolution processing, the sixth auxiliary feature map can obtain a 40×40 detection result through convolution processing, and the eighth auxiliary feature map can obtain a 20×20 detection result through convolution processing. The addition of the 160×160 scale detection head enables the processing method to further enhance the recognition and detection capabilities of small targets, but correspondingly, it will also slightly increase the number of network parameters and the amount of calculation, causing the network detection speed to slightly decrease. The additional detection head can be flexibly selected according to the size distribution of the target in the application.
[0125] Furthermore, the multi-input Concat operation can better reuse the fusion of multi-layer feature information. Its input includes feature maps of different layers but the same spatial size. When the multi-input Concat operation involves multiple inputs, its mathematical expression can be extended to a combination of multiple feature maps. Specifically, the multi-scale fusion method in S6 above is:
[0126]
[0127] Where: Represents a splicing operation; Represents the feature map of n different inputs; Indicates that the feature map is transformed along the channel dimension The multi-input Concat operation can combine shallow and deep features to enhance the representation ability of the processing method. By fusing feature maps of different scales, the processing method can better handle objects of various scales and improve the small target detection performance. It can also effectively integrate multi-level features and enhance the feature expression ability of the processing method. In this embodiment, the fusion operation preferably uses a four-input Concat operation, and its calculation formula is:
[0128]
[0129] Where: Indicates that the feature map is transformed along the dimension of channel C The new feature map is concatenated, and its number of channels is the sum of the number of channels of the 4 feature maps.
[0130] In this embodiment, the convolution processing includes: passing a feature map of size N×N×C through a convolution kernel with a step size of s and a kernel size of k, and outputting a feature map of size N / s×N / s×C1. The information of each pixel of the feature map is transmitted and retained through the convolution processing, wherein the height and width of the feature map are consistent and are both represented by N, and C1 represents the number of output channels of the digitized feature map after the convolution operation. The convolution processing operation exists in many places of the processing method, and they can select different parameters according to different scenarios. For example, in the initial stage of the composite feature processing, the convolution kernel size k of the first convolution processing in S21 is 6×6, and the step size s=2. A larger convolution kernel can be used in the figure. Extracting features from a larger area of the image helps to capture more spatial information. The step size setting allows the convolution process to perform downsampling operations in multiples of the step size, which helps to reduce the amount of calculation and control overfitting. For example, in the upsampling feature extraction process in the network backbone, the convolution process in the middle of the three upsampling processes has a kernel size k set to 3×3 and a step size s=1. The 3×3 kernel size can effectively capture local features such as edges, corners, and textures in the input feature map. When the step size is 1, the spatial size of the input and output feature maps can be kept unchanged, which means that the spatial distribution of information is retained, which helps to continue detailed analysis in the deep layers of the network.
[0131] Embodiment 2
[0132] The same parts in this embodiment as those in the first embodiment are given the same figure marks, and the same text descriptions are omitted.
[0133] like Figure 2As shown, this embodiment discloses an image processing system for remote sensing target detection scenarios. The processing system 10 uses the above-mentioned processing method, including:
[0134] A digitizing module 110, configured to digitize the original image to form a digitized feature map;
[0135] The composite feature extraction module 120 is used to take the digitized feature map as the initial input, and output multiple intermediate multidimensional feature maps and a final multidimensional feature map through composite feature extraction processing; the feature extraction module 130 embedded with the attention mechanism is used to take the final multidimensional feature map as the initial input, and output multiple intermediate global feature maps and a final global feature map through feature extraction processing embedded with the attention mechanism; the spatial pyramid pooling module 140 is used to perform fast spatial pyramid pooling processing on the final global feature map to form a standard feature map; the upsampling feature extraction processing module 150 is used to take the standard feature map as the initial input, and output multiple intermediate deep feature maps and a final deep feature map through upsampling feature extraction processing; the fusion module 160 is used to perform multi-scale fusion on the final multidimensional feature map, the intermediate multidimensional feature map, the intermediate global feature map, the intermediate deep feature map or the final deep feature map with the same spatial size to form a fused feature map; the output module 170 is used to extract different detection results according to different fused feature maps.
[0136] Embodiment 3
[0137] The same parts as those in the first and second embodiments of the present embodiment are given the same reference numerals, and the same text descriptions are omitted.
[0138] This embodiment discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method in the first embodiment is implemented.
[0139] Embodiment 4
[0140] The same parts as those in the first, second and third embodiments of the present embodiment are given the same reference numerals, and the same textual descriptions are omitted.
[0141] This embodiment discloses a storage medium on which a computer program is stored. When the computer program is executed by a processor, the method in the first embodiment is implemented.
[0142] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. An image processing method for remote sensing target detection scenes, characterized in that: The processing method comprises the following steps: S1, digitally processing the original image to form a digital feature map; S2, using the digitized feature map as initial input, and outputting a plurality of intermediate multidimensional feature maps and a final multidimensional feature map through composite feature extraction processing; S3, using the final multidimensional feature map as the initial input, outputting multiple intermediate global feature maps and a final global feature map through feature extraction processing embedded with an attention mechanism; S4, performing fast spatial pyramid pooling processing on the final global feature map to form a standard feature map; S5, using the standard feature map as the initial input, and outputting a plurality of intermediate deep feature maps and a final deep feature map through upsampling feature extraction processing; S6, performing multi-scale fusion on the final multi-dimensional feature map, the intermediate multi-dimensional feature map, the intermediate global feature map, the intermediate deep feature map or the final deep feature map with the same spatial size to form a fused feature map; S7, extracting different detection results according to different fusion feature maps; Among them, the extraction rules of the composite feature extraction processing, the feature extraction processing embedded with the attention mechanism and the upsampling feature extraction processing all include: based on the input feature map, outputting the extracted output feature map, the input feature map includes the initial input feature map and the intermediate feature map, and the intermediate feature map includes the output feature map obtained based on its last input image.
2. The image processing method for remote sensing target detection scene according to claim 1, characterized in that: The above S2 specifically includes the following steps: S21: performing convolution processing on the digitized feature map twice in succession, and then performing C3 processing once to form a first intermediate multi-dimensional feature map; S22: performing convolution processing on the first intermediate multidimensional feature map, and then performing C3 processing twice in succession to form a second intermediate multidimensional feature map; S23: performing convolution processing on the second intermediate multi-dimensional feature map to form a final multi-dimensional feature map.
3. The image processing method for remote sensing target detection scene according to claim 2, characterized in that: The C3 process in the above S21 and S22 includes the following steps: S201, splitting the input feature map into a first input feature map and a second input feature map along the channel dimension; S202, performing CBS processing on the first input feature map, and then performing multiple sequentially connected bottleneck processing to form a first branch feature map; S203, performing CBS processing on the second input feature map to form a second branch feature map; S204, splicing the first branch feature map and the second branch feature map to form a spliced feature map; S205: Perform CBS processing on the concatenated feature map to form an output feature map.
4. The image processing method for remote sensing target detection scene according to claim 2, characterized in that: The above S3 specifically includes the following steps: S31: performing three consecutive C3 processes of embedding space and channel attention on the final multidimensional feature map to form a first intermediate global feature map; S32: performing convolution processing on the first intermediate global feature map to form a second intermediate global feature map; S33: Perform C3 processing of embedding space and channel attention on the second intermediate global feature map to form a final global feature map.
5. The image processing method for remote sensing target detection scene according to claim 4, characterized in that: The above S5 specifically includes the following steps: S51, performing convolution processing on the standard feature map to form a first intermediate deep feature map; S52, performing upsampling processing on the first intermediate deep feature map, and then performing convolution processing to form a second intermediate deep feature map; S53, performing upsampling processing on the second intermediate deep feature map, and then performing convolution processing to form a third intermediate deep feature map; S54, performing upsampling processing on the third intermediate deep feature map, and then performing convolution processing to form a fourth intermediate deep feature map; S55, performing convolution processing on the fourth intermediate deep feature map to form a fifth intermediate deep feature map; S56, performing convolution processing on the fifth intermediate deep feature map twice in succession to form a sixth intermediate deep feature map; S57. Upsample the sixth intermediate deep feature map to form a final deep feature map.
6. The image processing method for remote sensing target detection scene according to claim 5, characterized in that: The above S6 specifically includes the following steps: S61, performing multi-scale fusion on the final deep feature map, the first intermediate global feature map, the second intermediate deep feature map and the fifth intermediate deep feature map to form a first fused feature map; S62, performing C3 processing of embedding space and channel attention on the first fused feature map to form a first auxiliary feature map; S63, performing convolution processing on the first auxiliary feature map to form a second auxiliary feature map; S64, performing upsampling processing on the second auxiliary feature map to form a third auxiliary feature map; S65, performing multi-scale fusion on the third auxiliary feature map, the third intermediate deep feature map, the fourth intermediate deep feature map, and the second intermediate multi-dimensional feature map to form a second fused feature map; S66, performing C3 processing of embedding space and channel attention on the second fused feature map to form a fourth auxiliary feature map; S67, performing convolution processing on the fourth auxiliary feature map to form a fifth auxiliary feature map; S68, performing multi-scale fusion on the fifth auxiliary feature map, the second auxiliary feature map, the final multi-dimensional feature map, and the second intermediate deep feature map to form a third fused feature map; S69, performing C3 processing of embedding space and channel attention on the third fused feature map to form a sixth auxiliary feature map; S610, performing convolution processing on the sixth auxiliary feature map to form a seventh auxiliary feature map; S611, performing multi-scale fusion on the seventh auxiliary feature map, the sixth intermediate deep feature map, the second intermediate global feature map, and the first intermediate deep feature map to form a fourth fused feature map; S612: Perform convolution processing on the fourth fused feature map to form an eighth auxiliary feature map.
7. The image processing method for remote sensing target detection scene according to claim 1, characterized in that: The multi-scale fusion method in S6 above is: Where: Represents a splicing operation; Represents n different input feature maps; Indicates that the feature map is transformed along the channel dimension Concatenate into the output feature map.
8. An image processing system for remote sensing target detection scenes, the processing system using the processing method according to any one of claims 1 to 7, characterized in that: include: A digitization module, used for digitizing the original image to form a digitized feature map; A composite feature extraction module, used to take the digitized feature map as initial input, and output a plurality of intermediate multidimensional feature maps and a final multidimensional feature map through composite feature extraction processing; A feature extraction module embedded with an attention mechanism is used to take the final multidimensional feature map as an initial input, and output a plurality of intermediate global feature maps and a final global feature map through feature extraction processing embedded with the attention mechanism; A spatial pyramid pooling module, used for performing fast spatial pyramid pooling processing on the final global feature map to form a standard feature map; An upsampling feature extraction processing module, used to take the standard feature map as initial input, and output a plurality of intermediate deep feature maps and a final deep feature map through upsampling feature extraction processing; A fusion module, used for performing multi-scale fusion on the final multi-dimensional feature map, the intermediate multi-dimensional feature map, the intermediate global feature map, the intermediate deep feature map or the final deep feature map with the same spatial size to form a fused feature map; The output module is used to extract different detection results according to different fusion feature maps.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
