Remote sensing image target detection method and system based on ODC-YOLO network

Abstract

The application discloses a remote sensing image target detection method and system based on an ODC-YOLO network, and the method comprises the following steps: S1, acquiring a remote sensing dataset; S2, performing cutting processing on the images in the dataset, and dividing the images into a training set, a verification set and a test set; S3, setting the size of an input image, the number of training times and the category parameters of a detection object; S4, a three-dimensional attention mechanism is integrated into dynamic convolution to form a multi-dimensional dynamic convolution, and a residual network Res2Net and a C3 module are fused to form a new C3-Res2Net module; the module is combined with the multi-dimensional dynamic convolution network to obtain an OD-Res2Net backbone network; a receptive field enhancement module with a multi-scale is constructed; an up-sampling operator is designed to perform a sampling operation; and S5, the images are added to the ODC-YOLO network model, and the images marked by the training set and the verification set are used for training; during the training process, the images divided in the test set are tested to obtain the training effect of each stage.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to technologies such as target detection, remote sensing and aerial image analysis, and image classification. Specifically, it relates to a remote sensing image target detection method and system based on the ODC-YOLO network. Background Technology

[0002] With the rapid development of aerospace remote sensing technology, the resolution of remote sensing images can reach decimeter and centimeter levels. Early low-resolution images often only allowed for coarse classification of ground features, while today, high-resolution remote sensing images can automatically identify detailed ground targets using image processing and deep learning methods. The information features of remote sensing images can be divided into low-level features, mid-level features, and high-level features. Low-level features include spectral, texture, and structural information; mid-level features encode low-level features; and high-level features are abstract semantic information. Remote sensing image target detection is an automated target detection technology that utilizes remote sensing data, and it has broad application prospects in fields such as intelligent surveillance.

[0003] In recent years, the rapid development of deep learning has provided strong technical support for intelligent information extraction from remote sensing images. Most deep learning-based object detection methods use Convolutional Neural Networks (CNNs) as the backbone network because CNNs can automatically extract high-level semantic features, offering stronger feature representation capabilities compared to traditional manual feature extraction. Besides commonly used CNNs, Recurrent Neural Networks (RNNs), Autoencoders (AEs), and Generative Adversarial Networks (GANs) also have important applications in object detection. The rapid development of deep learning has solved many problems in the field of computer vision. Among them, the YOLOv5 network model, after several generations of updates, has the advantages of fast detection speed and high accuracy, and has made significant progress in the field of image object detection. Therefore, this invention uses YOLOv5 as a basic network model. However, many interfering factors exist in remote sensing images, such as small object detection and ground feature recognition in complex backgrounds, thus posing higher requirements and challenges to remote sensing image object detection technology. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this invention proposes a remote sensing image target detection method and system based on the ODC-YOLO network. The main contents of this invention are: (1) To solve the problem of low detection rate of small targets in remote sensing images, this invention introduces multidimensional dynamic convolution (ODC) into the backbone network, and simultaneously utilizes the residual network Res2Net and the C3 module to form a new C3-Res2Net module. The reconstructed backbone network OD-Res2Net can improve the network's fine-grained feature fusion capability and increase the detection accuracy of small targets. (2) To further improve the detection effect of small targets in complex backgrounds, this invention fully utilizes the advantages of different receptive fields in feature extraction, and introduces a receptive field enhancement module C3RFEM with multi-diffraction rate dilated convolution, which is embedded after the backbone network. The C3RFEM module mainly obtains rich contextual information through multi-scale dilated convolution, improving the recognition capability of small targets in complex backgrounds. (3) In order to extract the semantic information required in the dense prediction task, the present invention introduces an adaptive lightweight upsampling operator CARAFE in the neck. The CARAFE sampling operator can obtain more information such as texture, color and structure in the features.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The specific steps of the remote sensing image target detection method based on ODC-YOLO network are as follows:

[0007] S1. Data Acquisition:

[0008] Obtain remote sensing datasets;

[0009] S2. Data preprocessing:

[0010] The images in the dataset are segmented into training, validation, and test sets.

[0011] S3. Configure the network model:

[0012] Set the input image size, training iterations, and the category of the object to be detected;

[0013] S4. Modify the existing YOLOv5 network structure, i.e., set the YOLOv5 network structure:

[0014] The ODC-YOLO network model is obtained by improving the YOLOv5 structure. The specific improvements are as follows:

[0015] S4.1. Since dynamic convolution adapts to different targets in the input image compared to static convolution, it performs convolution operations with more suitable parameters. Based on this, this invention incorporates attention mechanisms in three other dimensions (convolution size, number of input channels, and number of output channels) into dynamic convolution, forming a multi-dimensional dynamic convolution. At the same time, it uses the residual network Res2Net and the C3 module to form a new C3-Res2Net module. This module is then combined with the multi-dimensional dynamic convolutional network (ODC) to obtain the OD-Res2Net backbone network.

[0016] S4.2. Using the idea of ​​dilated convolution with different dilation rates, a multi-scale receptive field enhancement module is constructed. In convolutional layers of different sizes, dilated convolutional layer branches with dilation rates of 1, 2 and 3 are used respectively. By combining ordinary convolution and dilated convolution, the receptive field of each branch is effectively enhanced. Finally, the branches are superimposed to express the multi-scale receptive field fusion effect.

[0017] S4.3 Design a lightweight upsampling operator (CARAFE) for sampling operations. This module is mainly divided into an upsampling prediction module and a feature reconstruction module. Given an input feature map of shape C*H*W (C represents the number of channels, H represents the height, and W represents the width), the upsampling prediction module is first used to predict the upsampling kernel, and then the feature reconstruction module is used to complete the upsampling.

[0018] S5. Training the network model:

[0019] The image obtained in step S2 is added to the ODC-YOLO network model obtained in step S4. The model is trained using the labeled images from the training and validation sets. During the training process, the images in the test set are used for testing to obtain the training effect at each stage.

[0020] Preferably, in step S1, the DOTA dataset is used, and the images are from different regions and different time periods.

[0021] In a further preferred embodiment, in step S2, since the DOTA dataset is much larger than general object detection data, with the largest image resolution reaching 20000*20000, the dataset is preprocessed using the official script before the experiment. The images in the DOTA dataset are first cut and processed before being input into the network for training.

[0022] Preferably, in step S2, the cutting is achieved through a sliding window, setting the size of the small images in each image to a fixed pixel size, and the cutting starts from the upper left corner of the image; the cut dataset and the corresponding labels are divided into training set, validation set and test set according to 8:1:1.

[0023] Preferably, in step S3, parameters such as the size of the input image, the number of training iterations, and the category of the detection object are set reasonably according to the computer's memory and graphics card size. In addition, the compatibility of the environment being built with the graphics card should also be considered.

[0024] Preferably, in step S4.1, multidimensional dynamic convolution (ODC) is introduced into the backbone network. ODC dynamically aggregates multiple parallel convolutional kernels based on an attention mechanism. These kernels aggregate biases for each individual input data x by inputting the same attention. The formula for multidimensional dynamic convolution (ODC) is shown below:

[0025] y=(α w1 ⊙α f1 ⊙α c1 ⊙α s1 ⊙W1+...+α wn ⊙α fn ⊙α cn ⊙α sn ⊙W n )*x

[0026] Where x represents the input and y represents the output; α wi ∈R is the convolution dimension W i Attention coefficient matrix, α si ∈R k*k , and α fi ∈R out These represent the paths along the convolution kernel W. i The dynamic convolutional attention coefficient matrix is ​​defined along the spatial dimension, input channel dimension, and output channel dimension of the kernel. ⊙ represents the multiplication operation along different dimensions of the kernel space, and the value of i ranges from 1 to n. The attention is multiplied by the convolution kernel W. i The process is as follows:

[0027] First, the input x is compressed into a length of C using global average pooling (GAP). in The feature vectors are then passed through a fully connected layer (FC) and four branches. After the FC, the linear unit is corrected by the ReLU activation function. The FC maps the compressed feature vectors to a low-dimensional space. For each of the four branches, there is an output size of k*k, C... in *1、C out *1 and n*1 (where C in Indicates the number of input channels, C out A fully connected layer (FC) representing the number of output channels is constructed, and the Sigmoid and SoftMax functions are used to generate a normalized attention layer. si a ci a fi awi ;

[0028] In multidimensional dynamic convolution (ODC), for the convolution kernel W... i :(1)a si Assign different attention scalars to the convolution parameters at k*k spatial locations; (2)a ci For each convolution filter W i m C in Different attention scalars are assigned to the channels; (3)a fi C out Convolutional filters assign different attention scalars; (4)a wi Assign an attention scalar to the entire convolution;

[0029] Based on Dynamic Multidimensional Convolution (ODC), the Res2Net network is fused with the C3 module to form a new C3-Res2Net module. Res2Net achieves feature fusion through a series of channel partitioning, group convolution, inter-block fusion, and channel concatenation operations, as shown in the following equation:

[0030]

[0031] The input features are divided into s feature maps by channels, x i Let K represent the i-th feature map. i y represents the convolutional layer that fuses the i-th feature map. i Indicates fusion x i The feature map obtained afterwards, y i-1 Indicates fusion x i-1 The feature map obtained afterward.

[0032] Preferably, in step S4.3, the lightweight upsampling operator is mainly divided into two modules: an upsampling kernel prediction module and a feature reconstruction module; the expressions for the two modules are as follows:

[0033]

[0034] χ′ l′ =φ(N(χ) l ,k up ),w l' )

[0035] Upsampling kernel prediction module Based on the input features χ centered at the target location l l For each neighboring location, predict a recombinant kernel w. l' , where N(χ l ,k encoder ) represents the input feature χ lk encoder *k encoder The sub-region, the feature reorganization module φ in the input feature χ l Take k from up *k up subregion N(χ) l ,k up ), where k up *k up This indicates the size of the subregion, which is then compared with the recombined kernel w from the previous step. l' Performing the dot product yields the recombined input feature χ′ centered at the target position l'. l' ;

[0036] The upsampling kernel prediction module is responsible for generating reconstructed kernels in a content-aware manner, given an input feature χ of size C*H*W, where each source position on χ corresponds to a δ value of χ'. 2 There are one target location; each target location requires one The recombined kernel, in which k up This indicates the size of the reconstructed kernel, therefore this module will output a size of [size missing]. The recombined kernel; for an input feature map of shape C*H*W, first use a 1*1 convolution to compress the number of channels to C. m This reduces computational load; assuming the upsampling kernel size is k. up *k up If a different upsampling kernel is used for each location of each output feature map, the shape of the upsampling kernel to be predicted needs to be δH*δW*k. up *k up For the compressed input feature map, use a k encoder *k encoder Convolutional layers are used to predict the upsampling kernel, with C input channels. m The number of output channels is Then the channel is unfolded in space, resulting in a size of The upsampling kernel is used in the feature recombination module;

[0037] For each recombined kernel w l' The feature reorganization module reorganizes the features within a local region using a function φ. In the upsampling kernel prediction module, for each position in the output feature map, it maps it back to the input feature map and extracts the features at k... up *k up The output value is obtained by taking the dot product of the sub-region centered on the predicted point and the upsampling kernel. Different channels at the same location share the same upsampling kernel.

[0038] Preferably, in step S5, after obtaining the improved ODC-YOLO network model, it is added to the configured computer environment, the previously configured parameter file is added to the network model, and the labeled images of the training set and validation set are used for training. During the training process, the images divided in the test set are put into the computer for testing to obtain the training effect at each stage.

[0039] After step S5, proceed to step S6 and analyze the results:

[0040] After the model is trained, it will generate a corresponding weight file. Import the trained weights, the image to be detected and the corresponding label, run the program to get the detected data and images, and compare the recognition effect and detection accuracy to see if they meet the expected requirements.

[0041] This invention also discloses a remote sensing image target detection system based on an ODC-YOLO network, which, based on the above method, includes the following modules:

[0042] Data acquisition module: used to acquire remote sensing datasets;

[0043] Data preprocessing module: This module segments the images in the dataset into training, validation, and test sets.

[0044] Network model configuration module: Sets the input image size, training iterations, and object category parameters;

[0045] YOLOv5 network structure settings module:

[0046] The dynamic convolution incorporates a three-dimensional attention mechanism, including the size of the convolution, the number of input channels, and the number of output channels, forming a multi-dimensional dynamic convolution. At the same time, the residual network Res2Net is fused with the C3 module to form a new C3-Res2Net module. This module is then combined with the multi-dimensional dynamic convolution network to obtain the OD-Res2Net backbone network.

[0047] A multi-scale receptive field enhancement module is constructed. In convolutional layers of different sizes, three scales of dilated convolutional layer branches are used respectively. By combining ordinary convolution and dilated convolution, the receptive field of each branch is enhanced. Then, the branches are stacked together to express the multi-scale receptive field fusion effect.

[0048] Design a lightweight upsampling operator for sampling operations, mainly divided into an upsampling prediction module and a feature reconstruction module; given an input feature map, first use the upsampling prediction module to predict the upsampling kernel, and then use the feature reconstruction module to complete the upsampling.

[0049] Network model training module: Add images to the ODC-YOLO network model, use labeled images from the training set and validation set for training, and test the images from the test set during the training process to obtain the training effect at each stage.

[0050] The beneficial effects of this invention are:

[0051] This invention discloses a remote sensing image target detection method and system based on the ODC-YOLO network. (1) To solve the problem of low detection rate of small targets, this invention proposes to fuse multidimensional dynamic convolution (ODC) with Res2Net to reconstruct the backbone network. By constructing hierarchical residual connections within a residual block, dynamic convolution operations are performed from four dimensions: number of convolution kernels, convolution size, number of convolution input channels, and number of convolution output channels, which provides a guarantee for extracting rich contextual information. (2) To further improve the detection of small targets in complex backgrounds, this invention proposes a receptive field enhancement module based on the idea of ​​receptive field and multi-branch shared weights. By setting different expansion rates in each branch, the network can extract richer features. (3) The lightweight upsampling operator CARAFE is used in the neck of YOLOv5 to perform upsampling operations. The sampling is divided into two steps: prediction and reconstruction. During the sampling process, more texture, color, and structural feature information of the lower layers are preserved. Attached Figure Description

[0052] Figure 1 This is a network model structure diagram based on ODC-YOLO.

[0053] Figure 2 This is a schematic diagram of multidimensional dynamic convolution.

[0054] Figure 3 This is a structural diagram of the C3RFEM receptive field enhancement module.

[0055] Figure 4 This is a schematic diagram of the CARAFE lightweight upsampling operator.

[0056] Figure 5 This is a comparison image of small target detection results from an aerial perspective.

[0057] Figure 6 This is a comparison chart of false positives in target detection under complex backgrounds.

[0058] Figure 7 This is a comparison chart of the overall effect of remote sensing images.

[0059] Figure 8 This is a block diagram of a remote sensing image target detection system based on an ODC-YOLO network, according to a preferred embodiment of the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments, but it should be noted that the present invention is not limited to the following embodiments.

[0061] like Figure 1 As shown in the figure, this embodiment discloses a remote sensing image target detection method based on ODC-YOLO network. The specific steps are as follows:

[0062] 1. Data Acquisition:

[0063] Firstly, regarding the acquisition of the remote sensing dataset, this embodiment uses the DOTA dataset, whose images originate from different urban areas and different time periods. The data samples exhibit diversity, and the proportion of small targets in the data is balanced, with a distribution similar to real-world scenes. The DOTA dataset contains 2806 images, including 15 categories: aircraft, ships, oil tanks, baseball fields, tennis courts, basketball courts, ground runways, ports, bridges, large vehicles, small vehicles, helicopters, roundabouts, football fields, and basketball courts, totaling 188,282 targets.

[0064] 2. Image preprocessing:

[0065] The DOTA dataset has images with a maximum resolution of 20,000*20,000 pixels, and the targets in the remote sensing images are small and densely packed. The existing YOLOv5 algorithm has a fixed input and a small input image size. When extracting features, the network uses a downsampling method, which makes the input image size of the prediction layer even smaller. As a result, the network will have difficulty detecting small and densely packed targets.

[0066] Therefore, in this embodiment, the images in the dataset are first segmented before being input into the network for training. Segmentation is achieved using a sliding window, where the size of each small image is set to a fixed pixel size. Segmentation starts from the top left corner of the image. To ensure the integrity of the target in the image, following the official recommendation, a window size of 1024*1024 and a sliding step of 200 are used to segment the original DOTA data. Then, the segmented dataset and corresponding labels are divided into training, validation, and test sets in an 8:1:1 ratio.

[0067] 3. Configuring YOLOv5 model parameters

[0068] After processing the dataset, the next step is to modify the configuration model YAML file. First, change the label category in the voc.yaml file under the date folder to match the label in this example. Then, replace the training paths of train and val with the paths under the current dataset. Next, modify the training times and batch-size parameters in tain.py according to the requirements of this example. The environment of this example is: cuda11.3, deep learning framework pytorch1.12.1, Intel Core i5-12450h CPU, 16GB RAM, and NVIDIA GeForce RTX 3060 GPU with 6GB VRAM.

[0069] 4. Improve the existing YOLOv5 structure. The main structural improvements are as follows:

[0070] (1) In this embodiment, multidimensional dynamic convolution is introduced into the backbone network. Multidimensional dynamic convolution dynamically aggregates multiple parallel convolutional kernels based on an attention mechanism. These kernels aggregate biases for each individual input data x by inputting the same attention. The formula for multidimensional dynamic convolution is shown below:

[0071] y=(α w1 ⊙α f1 ⊙α c1 ⊙α s1 ⊙W1+...+α wn ⊙α fn ⊙α cn ⊙α sn ⊙W n )*x

[0072] Where x represents the input and y represents the output. wi ∈R is the convolution dimension W i The attention coefficient matrix, a si ∈R k*k , and a fi ∈R out These represent the paths along the convolution kernel W. i The dynamic convolution attention coefficient matrix is ​​defined along the spatial dimension, input channel dimension, and output channel dimension of the kernel. ⊙ represents the multiplication operation along different dimensions of the kernel space (i ranges from 1 to n). Figure 2 This explains how to multiply these four types of attention by the convolution kernel W. i The process.

[0073] Specifically, the input x is first compressed into a length C using global average pooling (GAP). inThe feature vectors are then processed through a fully connected layer (FC) and four branches. The FC layer is followed by a ReLU activation function to correct the linear units. The FC layer maps the compressed feature vectors to a low-dimensional space. For each of the four branches, there is an output of size k*k, C... in *1、C out Fully connected layers of *1 and n*1 (FC) are used, and the Sigmoid and SoftMax functions are used to generate normalized attention a. si a ci a fi a wi .

[0074] In multidimensional dynamic convolution, for the convolution kernel W i :(1)a si Assign different attention scalars to the convolution parameters at k*k spatial locations; (2)a ci For each convolution filter W i m C in Different attention scalars are assigned to the channels; (3)a fi C out Convolutional filters assign different attention scalars; (4)a wi Assign an attention scalar to the entire convolution.

[0075] To extract richer feature information, this embodiment fuses the Res2Net network with the C3 module to form a new C3-Res2Net module based on multidimensional dynamic convolution. Res2Net achieves feature fusion through a series of operations such as channel partitioning, group convolution, inter-block fusion, and channel concatenation, thereby improving the detection effect of small objects. The specific process is shown in the following formula:

[0076]

[0077] The input features are divided into s feature maps by channels, x i Let K represent the i-th feature map. i y represents the convolutional layer that fuses the i-th feature map. i Indicates fusion x i The feature map obtained afterwards, y i-1 Indicates fusion x i-1 The feature map obtained afterward.

[0078] (2) In order to solve the problem of false detection of small targets in complex backgrounds, this embodiment introduces a receptive field enhancement module with multiple scales. Each branch is set with a different expansion rate, thereby generating different receptive fields. Finally, the different receptive fields are combined to form a receptive field enhancement module with multiple scales, which makes the features extracted by the network richer and solves the detection interference problem caused by complex backgrounds.

[0079] Details of the receptive field enhancement module, such as Figure 3 As shown. Figure 3 In the middle, on the left is the C3 module in YOLOv5. In this embodiment, the designed receptive field enhancement module is embedded into the C3 module, reconstructing it into a C3REFM module, which is then added after the SPPF layer of the backbone network. (See attached image) Figure 3 By using dilated convolutional layers of three scales with dilation rates of 1, 2, and 3 at convolutional kernels of different sizes in branches 1, 2, and 3, the receptive field of each branch is effectively enhanced through the combination of ordinary convolution and dilated convolution. Finally, by stacking the branches together, the multi-scale receptive field fusion effect is expressed.

[0080] Each branch uses 1x1 convolutions to compress the number of channels, reducing the amount of computational parameters. Different receptive field sizes help the backbone network obtain more detailed, discriminative, and robust features during feature extraction. The feature maps obtained through the receptive field enhancement module have rich receptive fields and also contain more abstract information, improving the accuracy of recognition in complex backgrounds.

[0081] (3) This embodiment designs a lightweight upsampling operator CARAFE, which mainly consists of two modules: an upsampling kernel prediction module and a feature reconstruction module. The expressions for the two modules are shown below:

[0082]

[0083] χ′ l′ =φ(N(χ) l ,k up ),w l' )

[0084] Upsampling kernel prediction module Based on the input features χ centered at the target location l l For each neighboring location, predict a recombinant kernel w. l' , where N(χ l ,k encoder ) represents the input feature χ l k encoder *k encoder The sub-region, the feature reorganization module φ in the input feature χ l Take k fromup *k up subregion N(χ) l ,k up ), where k up *k up This indicates the size of the subregion, which is then compared with the recombined kernel w from the previous step. l' Performing the dot product yields the recombined input feature χ′ centered at the target position l'. l′ Specific details combined Figure 4 Detailed description.

[0085] The upsampling kernel prediction module is responsible for generating reconstructed kernels in a content-aware manner, given an input feature χ of size C*H*W, where each source position on χ corresponds to a δ value of χ'. 2 There are several target locations. Each target location requires a... The recombined kernel, in which k up This indicates the size of the reassembled kernel, therefore this module will output the size. The recombined kernel. For an input feature map of shape C*H*W, a 1*1 convolution is first used to compress the number of channels to C. m This reduces computational load. Assume the upsampling kernel size is k. up *k up If a different upsampling kernel is used for each location of each output feature map, the shape of the upsampling kernel to be predicted needs to be δH*δW*k. up *k up For the compressed input feature map from the first step, a k is used. encoder *k encoder Convolutional layers are used to predict the upsampling kernel, with C input channels. m The number of output channels is Then the channel is unfolded in space, resulting in a size of The upsampling kernel is used in the feature reconstruction module.

[0086] For each recombined kernel w l' The feature reorganization module reorganizes the features within a local region using a function φ. In the upsampling kernel prediction module, for each position in the output feature map, it maps it back to the input feature map and extracts the features at k... up *k up The output value is obtained by taking the dot product of the predicted upsampling kernel with the sub-region centered on that point. Different channels at the same location share the same upsampling kernel. In the experiment, k... encoder The value is 3, k up The value is 5.

[0087] 5. Train using the improved network model:

[0088] In this embodiment, the dataset is divided into training, validation, and test sets in an 8:1:1 ratio. The training run is set to 100 rounds, with a batch size of 16 images per training iteration. The training process is observed using a tensor board, and the trained weights are saved after training. The effects achieved by this invention are described below with reference to the accompanying figures and data.

[0089] To further verify the beneficial effects of the improved network model, ablation experiments were conducted. In this embodiment, to obtain a more intuitive understanding, the COCO dataset evaluation metric was used, measuring the model's performance from four perspectives: Mean Accuracy (AP). Specific results are shown in Table 1. During the experiments, the input resolution was uniformly set to 640*640. The data shows that the proposed YOLOv5+CARAFE+OD-Res2Net+C3RFEM method exhibits the best performance. Compared to YOLOv5, the AP... 75 It rose by 2 percentage points, AP 50 The average accuracy (AP) for small targets improved by 6 percentage points. S This represents a 5 percentage point increase, demonstrating that the invention has effectively improved performance in addressing small-target problems.

[0090] Table 1

[0091]

[0092] like Figure 5 As shown, to demonstrate the effect achieved by this invention, the target to be detected is outlined with a dashed line and enlarged in the original image. It can be seen that the existing YOLOv5 failed to detect this small target from such a high-altitude perspective; the network model did not learn the features of the labeled object. Comparing this to the right image, after training with the ODC-YOLO network, it is clearly evident that small targets that were previously undetectable can now be successfully and correctly identified. Furthermore, from... Figure 6 As can be seen from the three magnified local areas indicated by the dashed boxes, false detections occurred in all three locations, incorrectly detecting cars as airplanes. However, in the improved network of this invention, the previously falsely detected targets have disappeared. The overall effect is as follows: Figure 7 As shown, in a dense parking lot scene, comparing the left and right images, it can be seen that the improved network recognition results also successfully identify small targets that are densely packed and mutually occluded.

[0093] like Figure 8 As shown, this embodiment discloses a remote sensing image target detection system based on an ODC-YOLO network, which, based on the above method embodiment, includes the following modules:

[0094] Data acquisition module: used to acquire remote sensing datasets;

[0095] Data preprocessing module: This module segments the images in the dataset into training, validation, and test sets.

[0096] Network model configuration module: Sets the input image size, training iterations, and object category parameters;

[0097] The YOLOv5 network architecture configuration module is as follows:

[0098] The dynamic convolution incorporates a three-dimensional attention mechanism, including the size of the convolution, the number of input channels, and the number of output channels, forming a multi-dimensional dynamic convolution. At the same time, the residual network Res2Net is fused with the C3 module to form a new C3-Res2Net module. This module is then combined with the multi-dimensional dynamic convolution network to obtain the OD-Res2Net backbone network.

[0099] A multi-scale receptive field enhancement module is constructed. In convolutional layers of different sizes, three scales of dilated convolutional layer branches are used respectively. By combining ordinary convolution and dilated convolution, the receptive field of each branch is enhanced. Then, the branches are stacked together to express the multi-scale receptive field fusion effect.

[0100] Design a lightweight upsampling operator for sampling operations, mainly divided into an upsampling prediction module and a feature reconstruction module; given an input feature map, first use the upsampling prediction module to predict the upsampling kernel, and then use the feature reconstruction module to complete the upsampling.

[0101] Network model training module: Add images to the ODC-YOLO network model, use labeled images from the training set and validation set for training, and test the images from the test set during the training process to obtain the training effect at each stage.

[0102] Other aspects of this embodiment can be found in the above method embodiments.

[0103] Those skilled in the art will recognize that various substitutions and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the scope of protection of the invention should not be limited to the content disclosed in the embodiments.

Claims

1. A remote sensing image target detection method based on ODC-YOLO network, characterized in that, The specific steps are as follows: S1. Obtain the remote sensing dataset; S2. Perform image segmentation on the dataset, dividing the images into training set, validation set and test set; S3. Set the input image size, training iterations, and object category parameters; S4 is as follows: S4.1 A three-dimensional attention mechanism is incorporated into the dynamic convolution, including the convolution size, the number of input channels, and the number of output channels, forming a multi-dimensional dynamic convolution. Simultaneously, a new C3-Res2Net module is formed by fusing the Res2Net residual network with the C3 module. This C3-Res2Net module is then combined with the multi-dimensional dynamic convolution network to obtain the OD-Res2Net backbone network. In this step, multi-dimensional dynamic convolution is introduced into the backbone network. The multi-dimensional dynamic convolution dynamically aggregates multiple parallel convolution kernels based on the attention mechanism. These kernels aggregate biases for each individual input data x by inputting the same attention. The formula for multi-dimensional dynamic convolution is shown below: x represents the input, and y represents the output; convolution dimension Attention coefficient matrix, These represent the paths along the convolution kernel. The dynamic convolutional attention coefficient matrix in the spatial dimension, input channel dimension, and output channel dimension. This represents a multiplication operation along different dimensions of the kernel space, where the value of i ranges from 1 to n; attention is multiplied by the convolution kernel. The process is as follows: First, the input x is compressed into a length of using global average pooling (GAP). The feature vectors are then passed through a fully connected layer (FC) and four branches. After the FC, a ReLU activation function is used to correct the linear unit. The FC maps the compressed feature vectors to a low-dimensional space. For each of the four branches, the output size is... , Fully connected layers (FC) are constructed, and normalized attention is generated using the Sigmoid and SoftMax functions. ;in, Indicates the number of input channels, C out Indicates the number of output channels; In multidimensional dynamic convolution, for the convolution kernel (1) Different attention scalars are assigned to the convolution parameters at k*k spatial locations; (2) For each convolution filter Different attention scalars are assigned to channels; (3) Convolutional filters are assigned different attention scalars; (4) Assign an attention scalar to the entire convolution; Based on multidimensional dynamic convolution, the Res2Net network is fused with the C3 module to form a new C3-Res2Net module. Res2Net achieves feature fusion through a series of channel partitioning, group convolution, inter-block fusion, and channel concatenation operations, as shown in the following equation: The input features are divided into s feature maps by channels. This represents the i-th feature map. This represents the convolutional layer that fuses the i-th feature map. Indicates fusion The feature maps obtained afterwards Indicates fusion The feature map obtained afterwards; S4.2 Construct a receptive field enhancement module with multiple scales. In convolutional layers of different sizes, three scales of dilated convolutional layer branches are used respectively. By combining ordinary convolution and dilated convolution, the receptive field of each branch is enhanced. Then, the branches are stacked together to express the multi-scale receptive field fusion effect. S4.3 Design a lightweight upsampling operator for sampling operations, which consists of an upsampling prediction module and a feature reconstruction module. Given an input feature map, the upsampling prediction module is used to predict the upsampling kernel first, and then the feature reconstruction module is used to complete the upsampling. S5. Add the image obtained in step S2 to the ODC-YOLO network model obtained in step S4, and train it using the labeled images from the training set and validation set. During the training process, test the images in the test set to obtain the training effect at each stage.

2. The remote sensing image target detection method based on ODC-YOLO network as described in claim 1, characterized in that, In step S1, the DOTA dataset is used, and the images come from different regions and different time periods.

3. The remote sensing image target detection method based on ODC-YOLO network as described in claim 1, characterized in that, In step S2, the cutting is achieved through a sliding window, setting the size of the small images in each image to a fixed pixel size, and the cutting starts from the upper left corner of the image; the cut dataset and the corresponding labels are divided into training set, validation set and test set according to 8:1:

1.

4. The remote sensing image target detection method based on ODC-YOLO network as described in claim 1, characterized in that, In step S4.3, the lightweight upsampling operator is divided into two modules: an upsampling kernel prediction module and a feature reconstruction module; the expressions for the two modules are shown below: Upsampling kernel prediction module Based on the input features centered at the target location l The nearest neighboring locations are used to predict a recombined kernel for each location. ,in Representing input features of Sub-regions, feature recombination module In input features Take out subregion ,in This indicates the size of the subregion, which is then compared with the recombined kernel from the previous step. Perform a dot product to obtain the recombined product with the target position. Centered input features ; The upsampling kernel prediction module is responsible for generating recombined kernels in a content-aware manner, given a size of Input features Each source location on corresponds to There are one target location; each target location requires one The recombined kernel, in which This indicates the size of the recombined kernel; therefore, the upsampling kernel prediction module will output a size of [size missing]. The recombined kernel; for the shape of The input feature map is first compressed to C using a 1*1 convolution. m Reduce computational load; assume the upsampling kernel size is... If a different upsampling kernel is used for each location of each output feature map, the shape of the upsampling kernel to be predicted is... For the compressed input feature map, use a Convolutional layers are used to predict the upsampling kernel, with C input channels. m The number of output channels is Then the channel is unfolded in space to obtain a size of The upsampling kernel is used in the feature recombination module; For each recombined kernel The feature reorganization module reorganizes features within a local region using a function. In the upsampling kernel prediction module, for each position in the output feature map, it maps back to the input feature map and extracts the relevant information. The output value is obtained by taking the dot product of the upsampling kernel at the center of the sub-region and the position in the predicted output feature map, where different channels at the same position share the same upsampling kernel.

5. The remote sensing image target detection method based on ODC-YOLO network as described in claim 1, characterized in that, Step S6 is performed after step S5: After the model is trained, it will generate a corresponding weight file. Import the trained weights, the image to be detected, and the corresponding label to obtain the detected data and images, and analyze and compare the recognition effect and detection accuracy.

6. A remote sensing image target detection system based on an ODC-YOLO network, wherein the method described in any one of claims 1-5 is characterized in that, Includes the following modules: Data acquisition module: used to acquire remote sensing datasets; Data preprocessing module: This module segments the images in the dataset into training, validation, and test sets. Network model configuration module: Sets the input image size, training iterations, and object category parameters; YOLOv5 network structure settings module: The dynamic convolution incorporates a three-dimensional attention mechanism, including the size of the convolution, the number of input channels, and the number of output channels, forming a multi-dimensional dynamic convolution. At the same time, the residual network Res2Net is fused with the C3 module to form a new C3-Res2Net module. The C3-Res2Net module is then combined with the multi-dimensional dynamic convolution network to obtain the OD-Res2Net backbone network. A multi-scale receptive field enhancement module is constructed. In convolutional layers of different sizes, three scales of dilated convolutional layer branches are used respectively. By combining ordinary convolution and dilated convolution, the receptive field of each branch is enhanced. Then, the branches are stacked together to express the multi-scale receptive field fusion effect. Design a lightweight upsampling operator for sampling operations, which consists of an upsampling prediction module and a feature recombination module; Given an input feature map, the upsampling prediction module is first used to predict the upsampling kernel, and then the feature recombination module is used to complete the upsampling. Network model training module: Add images to the ODC-YOLO network model, use labeled images from the training set and validation set for training, and test the images from the test set during the training process to obtain the training effect at each stage.

Images