Object detection system, object detection method and corresponding training method

Abstract

A target detection system, a target detection method, a corresponding training method, an electronic device, and a readable storage medium, the target detection system can include a backbone network, a neck network, and a head network, the backbone network is configured to extract different scale features of an input image to generate a multi-layer feature map; the neck network is configured to perform feature fusion on the multi-layer feature map to generate a multi-layer feature fusion map; the head network includes a multi-stage refinement network and a detection network, wherein the multi-stage refinement network is configured to generate an initial region proposal about a detection target based on each layer of the multi-layer feature fusion map, and iteratively refines each initial region proposal through the multi-stage refinement network in turn to obtain a final region proposal about the detection target in each layer of the multi-layer feature fusion map; the detection network is configured to perform position regression and classification on the detection target based on the final region proposal to obtain the position and category of the detection target in the input image.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a target detection system, target detection method, corresponding training method, electronic device, and readable storage medium. Background Technology

[0002] Object detection is a core task in computer vision, aiming to accurately locate and identify specific objects in static images or dynamic video frames. This task requires not only that the model accurately delineate the bounding boxes of objects, but also that it correctly classify the objects within those bounding boxes. In the field of remote sensing, the complexity of object detection is further increased. Remote sensing images are typically acquired by satellites or unmanned aerial vehicles (UAVs), covering a wide geographical area, and containing high resolution, multi-scale features, and diverse ground objects. These properties make object detection in remote sensing images more challenging and complex. For example, the scale differences of objects in remote sensing images can be very large, and even the resolution of the same object can vary significantly in different images. In addition, the complexity of ground object types often leads to blurred boundaries and severe occlusion. Coupled with complex geographical environments and variable lighting conditions, these factors further exacerbate the difficulty of object detection tasks. Therefore, achieving efficient and accurate object detection in remote sensing images has become an important problem that urgently needs to be solved.

[0003] Leveraging the powerful feature representation capabilities of deep learning methods, many researchers have made significant progress in target detection tasks in remote sensing images. These methods have been widely applied in fields such as disaster monitoring, environmental protection, and urban planning, greatly improving the ability to monitor surface changes, the ecological environment, and urban development. Currently, deep learning-based remote sensing image target detection methods, similar to those in computer vision, are mainly divided into single-stage detection models and two-stage detection models. Single-stage detection models typically integrate target localization and classification tasks within a single network, offering high detection speed suitable for real-time applications. Two-stage detection models separate region proposal generation from the classification process, providing higher detection accuracy and being more suitable for more detailed target detection tasks.

[0004] However, the aforementioned models all employ a one-step method to generate region proposals, resulting in limited accuracy. Therefore, obtaining more accurate region proposals to improve detection precision has become a research hotspot. Furthermore, the high performance of these deep learning methods often relies on large amounts of labeled data. In remote sensing image processing, especially in scenarios involving novel or rare objects, obtaining sufficient labeled data can be extremely difficult and time-consuming. This problem stems not only from the fact that remote sensing images typically cover vast geographical areas, requiring manual labeling of numerous objects frame by frame, but also from the fact that many remote sensing applications involve a wide variety of extremely scarce objects, making the labeling process even more complex and challenging. Due to these limitations, the generalization ability of existing deep learning methods in remote sensing target detection is somewhat restricted. Summary of the Invention

[0005] To overcome the multiple challenges of target detection in remote sensing images and to improve the generalization ability of existing methods under small sample conditions, this application provides a target detection system, detection method, training method, electronic device, and readable storage medium.

[0006] According to a first aspect of this application, a target detection system is provided, comprising: a backbone network, a neck network, and a head network implemented by a computer program; the backbone network is configured to extract features at different scales from an input image to generate a multi-layer feature map; the neck network is configured to perform feature fusion on the multi-layer feature maps to generate a multi-layer feature fusion map; the head network includes a multi-stage thinning network and a detection network, wherein the multi-stage thinning network is configured to generate initial region proposals for the detected target based on each layer of the multi-layer feature fusion map, and iteratively refines each initial region proposal through the multi-stage thinning network to obtain a final region proposal for the detected target in each layer of the feature fusion map; the detection network is configured to perform position regression and classification on the detected target based on the final region proposals to obtain the position and category of the detected target in the input image.

[0007] According to a second aspect of this application, a target detection method is provided, the target detection method comprising: extracting features of different scales of an input image using a backbone network to generate a multi-layer feature map; fusing features of the multi-layer feature map using a neck network to generate a multi-layer feature fusion map; generating initial region proposals for the detected target based on each layer of the multi-layer feature fusion map using a multi-stage thinning network, and iteratively refining each initial region proposal sequentially through the multi-stage thinning network to obtain a final region proposal for the detected target in each layer of the feature fusion map; and performing location regression and classification of the detected target using a detection network based on the final region proposals to obtain the location and category of the detected target in the input image.

[0008] According to a third aspect of this application, a training method for any of the target detection systems provided in the embodiments of this application is provided. The training method includes: a basic training step and a fine-tuning step. The basic training step includes: training the network parameters of the backbone network, neck network, and head network in the initial target detection system using basic class data to obtain a target detection system after basic training. The fine-tuning step includes: training the network parameters of the neck network and head network (excluding the backbone network) in the target detection system after basic training using the basic class data and small sample data containing at least one new class of objects to obtain a target detection system after fine-tuning training.

[0009] According to a fourth aspect of this application, an electronic device is provided, including a processor, a memory, and a program stored in the memory and capable of running on the processor. When executed by the processor, the program implements the steps of any of the target detection methods provided in the embodiments of this application, or implements the steps of any of the training methods for a target detection system provided in the embodiments of this application.

[0010] According to a fifth aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored on the computer-readable storage medium, and when executed by a processor, the instructions implement the steps of any of the target detection methods provided in the embodiments of this application, or implement the steps of any of the training methods for a target detection system provided in the embodiments of this application.

[0011] Advantageously, the target detection system, target detection method, corresponding training method, electronic device, and computer storage medium provided in this paper have at least the following beneficial effects:

[0012] The input image is processed by a backbone network to obtain multi-layer feature maps. These feature maps are then processed by a neck network to obtain a refined multi-layer feature fusion map. This ensures that each layer of the feature fusion map simultaneously contains high-level semantic information and fine-grained spatial details, thereby improving the system's ability to detect targets at different scales. The multi-layer feature fusion maps are then sequentially input into a multi-stage refinement network. This multi-stage optimization strategy improves the quality and effectiveness of region proposals, making them more stable and reliable, and enhancing the system's robustness. Furthermore, the detection network can accurately detect the location and category of the target in the input image using multiple final region proposals. By combining feature information at different scales, the target detection system can more accurately capture the features of the target, thus improving detection accuracy. Attached Figure Description

[0013] To more clearly illustrate the specific embodiments of this application, the specific embodiments will be briefly described below with reference to the accompanying drawings. The accompanying drawings described below are only some embodiments of the scheme of this application. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0014] Figure 1 This diagram illustrates the structure of a target detection system according to an embodiment of this application.

[0015] Figure 2 This invention provides a schematic diagram of the structure of another target detection system according to an embodiment of the present application;

[0016] Figure 3 This diagram illustrates a flowchart of a target detection method according to an embodiment of this application.

[0017] Figure 4 This diagram illustrates a flowchart of a training method for an object detection system according to an embodiment of this application.

[0018] Figure 5 This diagram illustrates the structure of an electronic device according to an embodiment of the present application. Detailed Implementation

[0019] To make the above and other features and advantages of this application clearer, the application is further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explanation to those skilled in the art, and are exemplary only, not restrictive.

[0020] In the following description, numerous specific details are set forth to provide a thorough understanding of this application. However, it will be apparent to those skilled in the art that the specific details are not required to practice this application. In other instances, well-known steps or operations have not been described in detail to avoid obscuring this application.

[0021] One embodiment of this application provides a target detection system. Figure 1 This illustration shows a schematic diagram of the structure of a target detection system provided in an embodiment of this application, such as... Figure 1 As shown, the target detection system 100 includes: a backbone network 10, a neck network 20, and a head network 30 implemented based on a computer program.

[0022] Backbone network 10 is configured to extract features at different scales from the input image to generate multi-layer feature maps.

[0023] The backbone network involved in the embodiments of this article can be composed of deep convolutional neural networks or other similar feature extraction networks. Different scale features of the input image can refer to the features of the input image at different resolutions and scales.

[0024] In some embodiments, the input image can be a remote sensing image acquired through remote sensing technology. The remote sensing image contains targets of varying scales and has a complex background.

[0025] In one embodiment, the backbone network can scale the input image at different scales to form a series of multi-layered images with gradually decreasing resolution. The bottom-level image is discarded, and features are extracted from each of the other layers to obtain multi-layered feature maps. For example, the number of multi-layered feature maps can be at least two, such as three, four, five, or more. In an embodiment with four multi-layered feature maps, the multi-layered feature maps may include a second-layer feature map C2, a third-layer feature map C3, a fourth-layer feature map C4, and a fifth-layer feature map C5.

[0026] In one embodiment, the backbone network can scale the input image at different scales. Specifically, the backbone network can perform downsampling operations of a preset factor on the input image multiple times in succession. Optionally, the preset factor can be 2, i.e., 2x downsampling. 2x downsampling can mean reducing both the height and width of the image by half.

[0027] The neck network 20 is configured to perform feature fusion on multi-layer feature maps to generate a multi-layer feature fusion map.

[0028] In one embodiment, the number of multi-layer feature fusion maps is equal to the number of multi-layer feature maps. For example, if there are four multi-layer feature maps, the number of multi-layer feature fusion maps is also four. The layers of the multi-layer feature fusion maps correspond to the layers of the multi-layer feature maps. For example, the multi-layer feature maps may include a second-layer feature map C2, a third-layer feature map C3, a fourth-layer feature map C4, and a fifth-layer feature map C5, and the multi-layer feature fusion map may include a second-layer feature fusion map P2, a third-layer feature fusion map P3, a fourth-layer feature fusion map P4, and a fifth-layer feature fusion map P5, respectively.

[0029] In one embodiment, the neck network 20 can use an attention mechanism to obtain a top-level feature fusion map corresponding to the top-level feature map, and can use intra-layer fusion mechanism and cross-layer fusion mechanism to obtain a feature fusion map corresponding to each other layer feature map.

[0030] The head network 30 can be configured to perform object detection on multi-layer feature fusion maps to obtain the location and category of the detected object in the input image.

[0031] The detection target in this embodiment can refer to a target object that the target detection system can identify in the input image. Location can refer to the position of the detected target in the input image. Category can refer to the category to which the detected target belongs.

[0032] like Figure 1 As shown, the head network 30 may include a multi-stage refinement network 31 and a detection network 32.

[0033] The multi-stage refinement network 31 is configured to generate initial region proposals for the detected target based on each layer of the feature fusion map in the multi-layer feature fusion map. Each initial region proposal is iteratively refined through the multi-stage refinement network to obtain the final region proposal for the detected target in each layer of the feature fusion map. The initial region proposals may include information about multiple initial bounding boxes that could potentially surround the detected target. The bounding box information may include parameters for multiple sets of bounding boxes and their corresponding classification scores. One set of bounding box parameters corresponds to one bounding box. The classification score can be the probability that the candidate region corresponding to the bounding box contains the detected target. The final region proposals may include information about multiple final bounding boxes that could potentially surround the detected target. The final bounding box information may include parameters for multiple sets of final bounding boxes and their corresponding class scores. The class score can refer to the probability that the detected target surrounded by the candidate region corresponding to the final bounding box belongs to a certain class.

[0034] In one embodiment, the multi-stage refinement network 31 can be called a multi-stage refinement region proposal network. It first generates an initial region proposal for each feature fusion map, and then refines each initial region proposal through multiple iterations to output the final region proposal corresponding to each initial region proposal.

[0035] For example, multiple feature fusion maps are represented by P2, P3, P4 and P5. The multi-stage refinement network 31 can output the final region proposal corresponding to P2, the final region proposal corresponding to P3, the final region proposal corresponding to P4 and the final region proposal corresponding to P3.

[0036] The detection network 32 is configured to perform location regression and classification of the detected target based on the final region proposal, so as to obtain the location and category of the detected target in the input image.

[0037] In one embodiment, the detection network can combine the feature fusion map of each layer with the corresponding final region proposal and integrate the multi-layered feature fusion map. It then performs location regression and classification on the integrated feature fusion map, outputting the location and category of the detected object in the input image. Here, the location of the detected object is the coordinates of the bounding box. Classification can refer to identifying the category of the detected object contained within the candidate region corresponding to each bounding box in the integrated region proposal. Location regression can refer to adjusting the position of each bounding box in the integrated region proposal.

[0038] Through the above embodiments, the input image is processed by the backbone network to obtain multi-layer feature maps. These multi-layer feature maps are then processed by the neck network to obtain a refined multi-layer feature fusion map. This ensures that each layer of the feature fusion map simultaneously contains high-level semantic information and fine-grained spatial details, thereby improving the system's ability to detect targets at different scales. The multi-layer feature fusion maps are sequentially input into a multi-stage refinement network, thereby improving the quality and effectiveness of region proposals through a multi-stage optimization strategy. This makes the region proposals more stable and reliable, enhancing the system's robustness. Furthermore, the detection network can accurately detect the location and category of the target in the input image using multiple final region proposals. By combining feature information at different scales, the target detection system can more accurately capture the features of the target, thus improving detection accuracy.

[0039] In addition, in some embodiments, since remote sensing images often contain complex geometric features, the inventors conceived of using dilated convolutional networks that can capture contextual features around the bounding box and variable convolutional networks that capture complex and detailed geometric features to construct multi-stage networks.

[0040] In one embodiment, the multi-stage refinement network can be constructed using at least five types of convolutional layers, multiple classification heads, and multiple regression heads. Each convolutional layer is composed of a convolutional neural network. The at least five types of convolutional layers may include, but are not limited to, dilated convolutional layers, standard convolutional layers, variable convolutional layers, classification convolutional layers, and regression convolutional layers. Dilated convolutional layers and standard convolutional layers can be used to enhance feature map features. Dilated convolutional layers can divide the input feature map into multiple fixed-size feature regions and perform convolution operations on each feature region, expanding the receptive field, aggregating more contextual information, and outputting a feature map containing contextual enhancement.

[0041] Standard convolutional layers can further refine local regions and emphasize key features in edge regions, building upon contextual enhancements of feature maps. Variable convolutional layers can adaptively adjust the receptive field to capture more complex and detailed geometric features in the final stage input feature map.

[0042] It should be noted that the feature map input in the first stage is the feature fusion map of each layer, and the feature map input in each stage other than the first stage can be the feature map output from the previous stage.

[0043] The classification head can be used to predict the classification score of the candidate region corresponding to the bounding box, that is, to predict the probability of the detected target existing within the candidate region corresponding to the bounding box. The regression head can be used to predict the regression offset of the bounding box.

[0044] It should be noted that in some embodiments, the classification head and regression head can be implemented using existing classification and regression heads. For example, the classification head may include fully connected layers and activation functions. The regression head may include multiple fully connected layers.

[0045] Regression convolutional layers, a type of convolutional neural network, focus on regression tasks, optimizing bounding boxes to generate final region proposals with precise localization. Classification convolutional layers, also a type of convolutional neural network, focus on classification tasks, determining the category of detected objects within candidate regions corresponding to proposed bounding boxes.

[0046] Figure 2 This application provides a schematic diagram of the structure of another target detection system according to an embodiment of the present application. Figure 2 As shown, the multi-stage refinement network 31 in the target detection system may include M cascaded stage networks 311, where M is greater than or equal to 2.

[0047] A cascaded stage network of M stages includes at least a first-stage network and a final-stage network.

[0048] It should be noted that in the M cascaded stage networks, except for the final stage network, each of the other stage networks 311 includes at least one dilated convolutional layer, one standard convolutional layer, one classification head, and one regression head. The expanded receptive field provided by the dilated convolutional layer is crucial for accurately capturing object boundaries, especially in scenarios with limited training data, such as target detection in remote sensing with small sample sizes. Cascading dilated convolutional layers with standard convolutional layers can more effectively aggregate global and local information, compensating for information loss due to insufficient samples, thereby enhancing the system's robustness and significantly improving detection accuracy under limited data conditions.

[0049] In one embodiment, the first-stage network may include a first dilated convolutional layer, a first standard convolutional layer, a first classification head, and a first regression head. The first dilated convolutional layer is configured to extract contextual features from the Nth layer feature fusion map in the multi-layer feature fusion map to obtain a first-stage enhanced feature map. Here, N is greater than or equal to 2.

[0050] The Nth layer feature fusion map can be any layer of the multi-layer feature fusion map. In other words, the input to the first-stage network is the Nth layer feature fusion map.

[0051] In one embodiment, the first dilated convolutional layer can be represented by the following formula.

[0052] DilConv(P n )=∑ k P n [x+r·k x，y +r·k y ]·w[kx k y (1)

[0053] Where (x, y) are the coordinates of a specific point on the Nth layer feature fusion map Pn, r is the dilation rate that determines the spacing between kernel elements, and (k x k y () is the index of the convolution kernel, identifying a specific element in the kernel.

[0054] The first standard convolutional layer is configured to refine the local features of the first-stage enhanced feature map, resulting in the first-stage optimized feature map.

[0055] It should be noted that the input to the first dilated convolutional layer also includes preset anchor box information. Since the first-stage network does not have region suggestions from the previous stage network, the anchor boxes are used as the initial bounding box input.

[0056] Compared to the Nth layer feature fusion map, the first-stage enhanced feature map integrates the contextual features surrounding the anchor box region into the feature representation. Compared to the first-stage enhanced feature map, the first-stage optimized feature map adds refined features to local regions, emphasizing key features of the anchor box region. These key features can be edge features and texture features.

[0057] The first regression head is configured to obtain initial bounding boxes for the detected target based on the first-stage optimized feature map, and the first classification head is configured to predict classification scores for candidate regions corresponding to the initial bounding boxes based on the first-stage optimized feature map. Thus, the first-stage initial region proposals for the detected target are obtained from the first-stage network output. It should be noted that the initial region proposals include multiple initial bounding box positions and sizes, as well as their corresponding classification scores.

[0058] In one embodiment, the final-stage network may include variable convolutional layers, regression convolutional layers, and classification convolutional layers. The variable convolutional layers are configured to extract features from the optimized feature map of the previous stage, resulting in the final-stage feature map. It should be noted that by adaptively adjusting the receptive field, the variable convolutional layers can capture more complex and detailed geometric features from the optimized feature map of the previous stage. This is crucial for accurately identifying targets with different shapes and scales. The regression convolutional layers are configured to regress the location of the bounding boxes of the detected targets based on the region proposals from the previous stage using the final-stage feature map, and the classification convolutional layers are configured to classify and predict the candidate regions corresponding to the bounding boxes of the detected targets after location regression based on the final-stage feature map. Thus, the final region proposals for the detected targets are obtained from the Nth layer feature fusion map output by the final-stage network.

[0059] The classification prediction in this paper can predict the category to which an object belongs within a candidate region. The final region proposal can include the location, size, and category of the bounding box.

[0060] In one embodiment, the M cascaded stage networks further include at least one intermediate stage network, which includes, for example, a second stage network, a third stage network, and so on, up to the (M-1)th stage network.

[0061] Furthermore, each stage of the intermediate-stage network can include intermediate dilated convolutional layers, intermediate standard convolutional layers, intermediate classification heads, and intermediate regression heads. It should be noted that the input to the intermediate-stage network is the optimized feature map output from the previous stage network and the region proposal from the previous stage. The intermediate dilated convolutional layer is configured to extract contextual features from the previous stage's optimized feature map, resulting in the intermediate-stage enhanced feature map. The intermediate standard convolutional layer is configured to refine the candidate region features of the intermediate-stage enhanced feature map, resulting in the intermediate-stage optimized feature map. It should be noted that the candidate region features are the regions corresponding to each bounding box in the previous stage's region proposal.

[0062] Intermediate classification and intermediate regression heads are configured to update the previous stage region proposal based on the intermediate stage optimized feature map, resulting in intermediate stage region proposals. In one embodiment, the intermediate regression head is specifically configured to perform position regression on the bounding boxes in the previous stage region proposal based on the intermediate stage optimized feature map, i.e., update the bounding box positions in the previous stage region proposal. And the intermediate classification head is specifically used to predict the classification scores of the bounding boxes in the previous stage region proposal based on the intermediate stage optimized feature map, i.e., update the classification scores of the candidate regions corresponding to the bounding boxes in the previous stage region proposal.

[0063] In the above embodiments, the multi-stage refinement network improves the quality of region proposals by performing iterative refinement across multiple stages using M cascaded stage networks. Unlike traditional single-stage networks that typically generate region proposals in a single step, the multi-stage refinement network progressively improves the accuracy and quality of proposals through a series of gradual refinement processes. Furthermore, the dilated and standard convolutional layers in each stage network, along with the variable convolutional layers in the final stage network, better integrate features at different resolution levels and enable adaptive refinement strategies. This allows the system to generalize more effectively to objects of different scales and appearances, significantly improving the overall generalization ability of the enhancement system. This is particularly advantageous when addressing the challenges of target detection tasks with small sample sizes in remote sensing.

[0064] In some embodiments, such as Figure 2As shown, the neck network 20 may include a dual attention module 21. The dual attention module 21 can be configured to obtain a highest-level channel attention map based on a channel attention mechanism for the highest-level feature map, and to generate a highest-level spatial attention map based on a spatial attention mechanism; and to fuse the highest-level channel attention map and the highest-level spatial attention map to generate a highest-level feature fusion map, wherein the highest-level feature map is the feature map corresponding to N as the maximum value.

[0065] In one embodiment, the dual attention module 21 can be a convolutional block attention module constructed by combining channel attention and spatial attention mechanisms. The dual attention module 21 performs a combination of global average pooling and global max pooling operations on the highest-level feature map, and then generates the highest-level channel attention map through a shared multilayer perceptron.

[0066] The dual attention module 21 then multiplies the highest-level channel attention map element-wise with the highest-level feature map to obtain the highest-level optimized feature map. The dual attention module 21 performs a combination of max pooling and global average pooling on the highest-level optimized feature map to obtain a connected feature map, and performs a convolution operation on the connected feature map to obtain the highest-level spatial attention map.

[0067] Finally, the dual attention module 21 uses element-wise multiplication to fuse the highest-level channel attention map and the highest-level spatial attention map to generate the highest-level feature fusion map.

[0068] It should be noted that the highest-level channel attention map is a single-channel attention map, which emphasizes the most information-rich channel while suppressing less relevant channels, thus enhancing feature representation. The highest-level spatial attention map can optimize target region detection by focusing on the most salient spatial regions in the feature map.

[0069] Taking the highest layer feature map C5 as an example, the calculation formula of the highest layer channel attention map Mc(C5) can be expressed as follows.

[0070] Mc(C5)=σ(MLP(AvgPool(C5))+MLP(MaxPool(C5))) (2)

[0071] Where σ is the sigmoid activation function, MLP is a shared multilayer perceptron, AvgPool represents the global average pooling operation, and MaxPool represents the global max pooling operation.

[0072] The calculation formula for the highest-level optimized feature map C5′ can be expressed as follows.

[0073]

[0074] in, This indicates element-wise multiplication.

[0075] Highest-level spatial attention graph M s The formula for calculating (C5′) can be expressed as follows.

[0076] M s (C5′)=σ(Conv 7×7 ([AvgPool(C5′);MaxPool(C5′)])) (4)

[0077] Among them, Conv 7×7 This represents a 7×7 convolution kernel, σ represents the sigmoid activation function, AvgPool represents the global average pooling operation, and MaxPool represents the global max pooling operation.

[0078] The calculation formula for the highest-level feature fusion map P5 can be expressed as follows.

[0079]

[0080] In the above embodiments, the dual attention module can ensure that more attention is paid to the regions containing key information in the highest-level feature map by applying a dual attention mechanism to the highest-level feature map, thereby improving the overall accuracy of target detection.

[0081] In some embodiments, such as Figure 2 As shown, the neck network 20 also includes multiple intra-layer feature fusion modules 22, multiple cross-layer feature fusion modules 23, and multiple upsampling modules 24, which are respectively distributed in the lowest layer, the highest layer, and the middle layer.

[0082] In one embodiment, the number of intra-layer feature fusion modules 22 is equal to the number of multi-layer feature maps, and the number of cross-layer feature fusion modules and the number of sampling modules are one less than the number of intra-layer feature fusion modules.

[0083] It should be noted that although each feature map corresponds to an intra-layer feature fusion module 22, Figure 2 The highest-level intra-layer feature fusion module 22 is configured to multiply the highest-level channel attention map element-wise with the highest-level feature map to obtain the highest-level optimized feature map, which is then used to obtain the highest-level spatial attention map. Each feature map, except for the lowest-level feature map, corresponds to a cross-layer feature fusion module 23 and an upsampling module 24.

[0084] For each layer except the highest layer, the intra-layer feature fusion module 22 of each layer is configured to perform intra-layer feature fusion on the feature fusion map of each layer and the feature map after convolution operation based on the first spatial weight and the second spatial weight, so as to obtain the intra-layer feature fusion map of each layer.

[0085] In other words, the intra-layer feature fusion module 22 can perform convolution operations on the feature maps of each layer, and then perform weighted fusion with the corresponding feature fusion maps.

[0086] For each layer except the lowest layer, the sampling module 24 of each layer is configured to upsample the intra-layer feature fusion map of each layer to obtain a sampled feature map, so that the resolution of the sampled feature map of each layer is equal to the resolution of the feature map of the next layer.

[0087] It should be noted that the sampling factor of the upsampling module 24 is the same as the downsampling factor of the backbone network 10. That is, if the backbone network uses 2x downsampling, then the upsampling module 24 uses 2x upsampling.

[0088] For each layer except the lowest layer, the cross-layer feature fusion module 32 of each layer is configured to perform cross-layer feature fusion on the sampled feature map of each layer and the feature map of the next layer based on the third spatial weight, so as to obtain the feature fusion map of the next layer.

[0089] The neck network involved in the embodiments of this application corresponds to the layers of the multi-layer feature maps. That is, if the feature map has 2 to 5 layers, then the neck network also includes 2 to 5 layers. The next layer can be any feature map other than the highest feature map.

[0090] In one embodiment, the next layer feature fusion map, excluding the highest layer feature fusion map, is obtained by performing a convolution operation on the previous layer feature map of the next layer and then fusing it with the feature fusion map. The resulting fusion map is then upsampled to match the resolution of the next layer feature map, and then fused with the convolutional feature map of the next layer.

[0091] Specifically, the fusion of the feature map from the previous layer and the feature fusion map is achieved through weighted fusion, which is obtained by multiplying the first spatial weight by the feature fusion map from the previous layer and then adding the second spatial weight multiplied by the feature map from the previous layer. In this way, this weighted fusion effectively captures and emphasizes features from P. n+1 High-level semantic features and C n+1 The most relevant information in the detailed spatial features.

[0092] The fusion of the sampled feature map and the feature map of the current layer after the operation can also be achieved through weighted fusion, that is, the sampled feature map is multiplied by the third spatial weight and the feature map of the next layer.

[0093] It should be noted that each layer has its own first-space weight, second-space weight, and third-space weight. Each weight needs to be set according to the importance of the task at each layer.

[0094] Thus, the adaptive spatial weighting mechanism enables the neck network to dynamically balance the contributions of these different feature sources according to the importance of the task at the current level.

[0095] Feature fusion map P at layer n n The formula for calculating can be expressed as follows.

[0096] P n =U(α) n ·P n+1 +β n ·Conv(C n+1 ))+γ n ·Conv(C n (6)

[0097] Where U represents upsampling, α n Represents the first spatial weight, β n Represents the second spatial weight, γ n C represents the third space weights. n Let represent the feature map of the nth layer, and Conv represent convolution.

[0098] In other words, after obtaining the highest-level feature fusion map, the neck network generates lower-level feature fusion maps sequentially from top to bottom. For example, the neck network first generates P5, then P4, P3, and finally P2.

[0099] In the above embodiments, by utilizing a cross-layer adaptive fusion strategy, the neck network effectively enhances the richness and accuracy of the generated feature maps. Furthermore, this robust multi-scale representation significantly enhances the system's generalization ability in remote sensing target detection tasks with limited sample sizes, enabling it to accurately detect and classify targets of various scales and complexities even with limited training data.

[0100] Furthermore, the neck network enhances the system's generalization ability by introducing an attention mechanism on the top-level feature map and integrating multi-scale features through a cross-layer fusion process. This neck network is crucial for improving the system's ability to accurately detect targets across various scales, especially in the complex and diverse environments presented by remote sensing imagery. By effectively capturing and integrating multi-scale information, the neck network ensures that the system can robustly handle common target size and appearance variations in remote sensing data.

[0101] In some embodiments, the sum of the first spatial weight, the second spatial weight, and the third spatial weight is 1, which can be expressed as the following formula.

[0102] α n +β n +γ n =1 (7)

[0103] In the above embodiments, the constraint of the sum of the three weights ensures that the combination of features from different sources remains balanced, and no single source dominates the fusion process. The adaptability of the three spatial weights allows the neck network to dynamically emphasize the most relevant features based on the context of each spatial location.

[0104] In some embodiments, such as Figure 2 As shown, the detection network 32 further includes a pooling layer 321, a fully connected layer 322, a final classification layer 323, and a final regression layer 324.

[0105] Pooling layer 321 is configured to perform pooling operations based on the final region proposal and the feature fusion map of each layer to obtain multiple normalized feature maps.

[0106] In other words, pooling layer 321 is used to map the bounding boxes of the final region proposals of each layer to the feature fusion map of each layer, and to perform pooling operations on the mapped feature fusion maps so that the output size of each layer of feature fusion maps after pooling is consistent.

[0107] Fully connected layer 322 is configured to fully connect each normalized feature map to obtain a fixed-dimensional feature vector corresponding to each normalized feature map.

[0108] In other words, the fully connected layer 322 maps each normalized feature map to a fixed-dimensional feature vector, where each fixed-dimensional feature vector contains high-level features of the target object in the input image. It should be noted that the fixed dimension can be set according to requirements. For example, the fixed dimension can be 512.

[0109] Finally, the classification layer 323 is configured to perform classification based on feature vectors of multiple fixed dimensions to determine the category of the detected target in the input image.

[0110] In other words, the final classification layer is used to predict the probability that the target enclosed by the bounding box in each final region proposal belongs to each category by analyzing the feature vector of each fixed dimension, and then merging the classification results to determine the category of each detected target in the input image.

[0111] Finally, regression layer 324 is configured to perform location regression based on multiple fixed-dimensional feature vectors to determine the location of the detected target in the input image.

[0112] In other words, the final regression layer 324 adjusts the position and size of the bounding box in each final region proposal by analyzing the feature vector of each fixed dimension, and merges the adjustment results to determine the location of the detected target in the input image.

[0113] In some embodiments, the final classification layer 323 is also provided with placeholder nodes.

[0114] Specifically, a key challenge in few-shot object detection is that the system is prone to overfitting to basic classes during the initial training phase, which affects its ability to adapt to new classes in subsequent phases. To address this issue, a dedicated node called a placeholder node is introduced in the final classification layer of the system. Placeholder nodes are specifically reserved for new classes not encountered by the system during the initial training phase.

[0115] In some embodiments, the object detection system 100 further includes a loss component. The loss component is used to train the object detection system. The loss component includes a total base loss and a total fine-tuning loss in the embodiments described below.

[0116] This application also provides a target detection method, which can be applied to the target detection system 100 described in any of the foregoing embodiments. Figure 3 This document illustrates a flowchart of a target detection method provided in an embodiment of this application. Figure 3 As shown, the target detection method may include the following steps.

[0117] S41 uses a backbone network to extract features at different scales from the input image to generate multi-layer feature maps.

[0118] S42, use the neck network to perform feature fusion on the multi-layer feature map to generate a multi-layer feature fusion map.

[0119] S43, using a multi-stage refinement network to generate initial region proposals for the detected target based on each layer of the feature fusion map in the multi-layer feature fusion map, and iteratively refines each initial region proposal through the multi-stage refinement network to obtain the final region proposals for the detected target in each layer of the feature fusion map.

[0120] S44, the detection network is used to perform location regression and classification of the detected target based on the final region proposal, so as to obtain the location and category of the detected target in the input image.

[0121] In the above embodiments, the input image is processed by a backbone network to obtain multi-layer feature maps. These multi-layer feature maps are then processed by a neck network to obtain a refined multi-layer feature fusion map. This ensures that each layer of the feature fusion map simultaneously contains high-level semantic information and fine-grained spatial details, thereby improving the system's ability to detect targets at different scales. The multi-layer feature fusion map is input into a multi-stage refinement network, which improves the quality and effectiveness of region proposals through a multi-stage optimization strategy. Furthermore, the detection network can accurately detect the location and category of the target in the input image using multiple final region proposals.

[0122] In some embodiments, the multi-stage refinement network includes M cascaded stage networks, where M is greater than or equal to 2. The M cascaded stage networks include at least a first-stage network and a final-stage network. The first-stage network includes a first dilated convolutional layer, a first standard convolutional layer, a first classification head, and a first regression head. The final-stage network includes a variable convolutional layer, a regression convolutional layer, and a classification convolutional layer. S43 may include the following steps.

[0123] The first dilated convolution of the first-stage network is used to extract contextual features from the Nth layer feature fusion map of the multi-layer feature fusion map to obtain the first-stage enhanced feature map. The local features of the first-stage enhanced feature map are refined using the first standard convolutional layer to obtain the first-stage optimized feature map. The first regression head is used to obtain the initial bounding box of the detected target based on the first-stage optimized feature map, and the first classification head is used to predict the classification score of the candidate region corresponding to the initial bounding box based on the first-stage optimized feature map, thereby obtaining the first-stage initial region proposal of the detected target, where N is greater than or equal to 2.

[0124] The final stage network uses variable convolutional layers to extract features from the previous stage optimized feature map to obtain the final stage feature map. Then, regression convolutional layers are used to regress the location of the bounding box of the detected target based on the previous stage region proposal using the final stage feature map. Finally, classification convolutional layers are used to classify and predict the candidate regions corresponding to the bounding boxes of the detected targets after location regression based on the final stage feature map, thereby obtaining the final region proposal of the detected target in the Nth layer feature fusion map.

[0125] In some embodiments, the M cascaded stage networks further include at least one intermediate stage network, wherein the intermediate stage network includes an intermediate dilated convolutional layer, an intermediate standard convolutional layer, an intermediate classification head, and an intermediate regression head. S43 may further include: using the intermediate dilated convolutional layer to extract contextual features from the previous stage optimized feature map to obtain an intermediate stage enhanced feature map; using the intermediate standard convolutional layer to refine the candidate region features of the intermediate stage enhanced feature map to obtain an intermediate stage optimized feature map; and using the intermediate classification head and intermediate regression head to update the previous stage region proposal based on the intermediate stage optimized feature map to obtain an intermediate stage region proposal.

[0126] In some embodiments, the neck network includes a dual attention module. S42 may include: using the dual attention module to obtain a highest-level channel attention map based on a channel attention mechanism for the highest-level feature map, and generating a highest-level spatial attention map based on a spatial attention mechanism; fusing the highest-level channel attention map and the highest-level spatial attention map to generate a highest-level feature fusion map, wherein the highest-level feature map is the feature map corresponding to N being the maximum value.

[0127] In some embodiments, the neck network further includes multiple intra-layer feature fusion modules, multiple cross-layer feature fusion modules, and multiple upsampling modules distributed on the lowest layer, the highest layer, and the middle layer, respectively. S42 may also include the following steps.

[0128] For each layer except the highest layer, the intra-layer feature fusion module of each layer performs intra-layer feature fusion on the feature fusion map of each layer and the feature map after convolution operation based on the first spatial weight and the second spatial weight, to obtain the intra-layer feature fusion map of each layer.

[0129] For each layer except the lowest layer, the intra-layer feature fusion map of each layer is upsampled using the upsampling module of each layer to obtain a sampled feature map, so that the resolution of the sampled feature map of each layer is equal to the resolution of the feature map of the corresponding next layer.

[0130] For each layer except the lowest layer, the cross-layer feature fusion module of each layer is used to perform cross-layer feature fusion with the feature map of the corresponding next layer based on the third spatial weight, so as to obtain the feature fusion map of the next layer.

[0131] In some embodiments, the sum of the first spatial weight, the second spatial weight, and the third spatial weight is 1.

[0132] In some embodiments, S44 may include the following steps:

[0133] Pooling layers are used to perform pooling operations based on the final region proposal and the feature fusion map of each layer to obtain multiple standardized feature maps.

[0134] A fully connected layer is used to fully connect each normalized feature map to obtain a fixed-dimensional feature vector corresponding to each normalized feature map.

[0135] The final classification layer is used to classify based on feature vectors of multiple fixed dimensions to determine the category of the detected target in the input image.

[0136] The final regression layer is configured to perform location regression based on feature vectors of multiple fixed dimensions to determine the location of the detected target in the input image.

[0137] In practical testing, airplanes, baseball fields, and tennis courts were selected as new categories, while the remaining categories served as base categories. The performance of the object detection system provided in this application was evaluated on 3-shot, 5-shot, 10-shot, and 20-shot detection tasks. For baseline comparison, we selected Meta-RCNN, Fs-DetView, TFA, P-CNN, FSOD, FSCE, ICPE, VFA, and SAE-FSDet as comparison methods. Test results show that the object detection system provided in this application consistently achieves the highest object detection performance for small sample sizes across all settings, outperforming the best baseline model by 2.47, 2.55, and 3.44 percentage points in 3-shot, 5-shot, and 10-shot settings, respectively.

[0138] This application also provides a training method for an object detection system, which can be applied to train any of the object detection systems described above. Figure 4 This diagram illustrates a flowchart of a training method for an object detection system according to an embodiment of this application. Figure 4 As shown, the training method of this target detection system can include basic training steps and fine-tuning steps.

[0139] S51, the basic training steps include: using basic class data to train the network parameters of the backbone network, neck network and head network in the initial object detection system to obtain the object detection system after basic training.

[0140] For example, base class data can include a large number of images labeled with base class objects. Base class data can be selected from the DIOR dataset, a large-scale benchmark dataset used for object detection in optical remote sensing images.

[0141] In one embodiment, the network parameters of the backbone network, neck network, and head network in the initial target detection system are set to initial values. After basic training, the target detection system learns a general feature representation, which can be used to identify the location and class of a base category target in the input image.

[0142] S52, the fine-tuning step includes: using basic class data and small sample data containing at least one new class of objects to train the network parameters of the neck network and head network in the object detection system after basic training, excluding the backbone network, to obtain the fine-tuned object detection system.

[0143] The small sample data in this paper consists of multiple images labeled with new category objects. The amount of data in the small sample data is much smaller than the amount of data in the base class data (e.g., less than one-third or less). For example, the base class data has 100 or more images, such as 120 or 150 images or more, while the small sample data has, for example, 20, 15, 10 images or fewer, as long as the amount of data in the small sample data is much smaller than the amount of data in the base class data. The new category objects are objects whose category differs from the base category. For example, the new category objects include baseball fields, basketball courts, bridges, chimneys, and ships, while the remaining categories are the base categories.

[0144] In one embodiment, the fine-tuned target detection system, compared to the basic trained target detection system, features fine-tuned network parameters for both the neck and head networks. The fine-tuned target detection system is better able to detect new categories.

[0145] In some embodiments, S51, the basic training step may further include applying a sparse activation mechanism to placeholder nodes in the final classification layer of the initial object detection system to limit the activity of the placeholder nodes.

[0146] In one embodiment, a sparse activation mechanism can be applied to all placeholder nodes using L1 regularization. Applying sparse activation ensures that placeholder nodes maintain minimal activation, meaning their outputs are 0 or close to 0, thus preventing them from inadvertently learning features related to the base category. In this way, by maintaining the inactivity of all placeholder nodes, their flexibility is preserved, allowing for effective activation when new categories are introduced in subsequent fine-tuning phases.

[0147] In some embodiments, the network parameters of the backbone network, neck network, and head network in the initial target detection system are trained using basic class data to obtain a target detection system after basic training. This further includes: inputting basic class data into the initial target detection system and iteratively updating the network parameters of the backbone network, neck network, and head network in the initial target detection system to obtain a minimized total basic loss, thereby obtaining a target detection system after basic training.

[0148] The total basic loss includes basic classification loss and basic regression loss. Basic classification loss includes the generalized classification loss of the basic categories generated by the detection network and the classification loss of the basic categories generated by the final stage network of the multi-stage refinement network.

[0149] The generalized classification loss of the base class generated by the detection network includes the standard cross-entropy loss of the base class and the sparse regularization loss of the placeholder nodes in the classification layer of the initial object detection system.

[0150] Generalized classification loss of the basic categories generated by the detection network It can be expressed as the following formula.

[0151]

[0152] Among them, L base L represents the standard cross-entropy loss of the base class. placeholder λ represents the sparse regularization loss applied to the placeholder nodes. placeholder It is a regularization coefficient that controls the degree of this sparsity constraint.

[0153] In one embodiment, the classification loss of the base categories generated by the final stage network of the multi-stage refinement network can be used to evaluate the accuracy of bounding box predictions, and can be represented by cross-entropy loss. Cross-entropy loss measures the class probability distribution predicted by the final stage network compared to the class probability distribution of the labeled data.

[0154] The basic regression loss includes the regression loss generated by the detection network and the regression loss generated by each stage of the multi-stage refinement network.

[0155] In one embodiment, the regression loss generated by each stage network can be represented by the Cross-Union Ratio (CIRR) loss. The CIRR loss can be used to measure the degree of overlap between the predicted bounding boxes and the ground truth bounding boxes of the stage network.

[0156] The regression loss generated by the multi-stage refinement network can be expressed by the following formula.

[0157]

[0158] Among them, L τ MRRPN_reg Let α represent the regression loss in stage τ. τ λ represents the weight of the regression loss of the network in stage τ. λ is the weight that measures the classification loss and regression loss of the multi-stage refined network.

[0159] In one embodiment, the regression loss generated by the detection network can also be represented by the cross-union loss.

[0160] In the above embodiments, the system trained with a total base loss that includes a generalized classification loss can further improve the performance of few-shot classification tasks.

[0161] In some embodiments, the network parameters of the neck network and head network (excluding the backbone network) in the base class data and small sample data containing at least one new class object are trained using the base-trained object detection system to obtain a fine-tuned object detection system. This includes: freezing the parameters of the backbone network in the base-trained object detection system; activating the placeholder nodes in the classification layer of the base-trained object detection system; inputting the base class data and small sample data containing at least one new class object into the activated base-trained object detection system; iteratively updating the parameters of the unfrozen networks in the activated base-trained object detection system to obtain a minimized total combined loss, thereby obtaining the fine-tuned object detection system.

[0162] The total fine-tuning loss includes classification loss and regression loss for the base category and the new category.

[0163] The classification loss for the base category and the new category includes the generalized classification loss generated by the detection network for the base category and the new category, as well as the classification loss generated by the final stage network of the multi-stage refinement network for the base category and the new category.

[0164] Generalized classification loss for classification loss of base category and new category This includes the classification loss of the basic category generated by the detection network, the classification loss of the new category, and the sparse regularization loss of the placeholder nodes.

[0165]

[0166] Among them, L base L represents the classification loss of the basic categories. novel L represents the classification loss for the new category. regularization Let λ represent the regularization loss. regularization This represents the regularization coefficient.

[0167] The classification loss of the base class and the new class generated by the final stage of the multi-stage refinement network can be represented by the cross-entropy loss.

[0168] In one embodiment, both the classification loss for the base category and the classification loss for the new category can be represented by cross-entropy loss. The classification loss for the base category ensures the system retains its ability to classify the base category, thus preserving previously learned knowledge. The classification loss for the new category allows for fine-tuning of placeholder nodes, enabling the system to adapt to the classification requirements of the new category. The sparsity regularization loss of the placeholder nodes prevents the system from overfitting to the new category.

[0169] The regression loss for the base category and the new category includes the regression loss for the base category and the new category generated by the detection network and the regression loss for the base category and the new category generated by each stage of the multi-stage refinement network.

[0170] The regression loss of the base class and the new class generated by each stage of the multi-stage refinement network can be expressed as Equation 9. The regression loss of the base class and the new class generated by the detection network can be expressed as the crossover ratio loss.

[0171] In the above embodiments, by introducing placeholder nodes, regularization terms, and generalized classification loss to train the system, not only is the model's basic classification ability preserved, but its generalization ability is also significantly enhanced. The trained target detection system can effectively adapt to new categories in scenarios with a small number of samples, thereby improving its detection accuracy and generalization performance in remote sensing target detection tasks with small sample sizes.

[0172] It should be understood that the specific features, operations, and details described herein with respect to the methods of this application can also be similarly applied to the apparatus and system of this application, or vice versa. Furthermore, each step of the methods of this application described above can be performed by a corresponding component or unit of the apparatus or system of this application.

[0173] It should be understood that the various modules / units of the device of this application can be implemented wholly or partially through software, hardware, firmware, or a combination thereof. Each module / unit can be embedded in the processor of the electronic device in hardware or firmware form or independent of the processor, or it can be stored in the memory of the electronic device in software form for the processor to call to execute the operation of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0174] In another aspect, this application provides an electronic device. Figure 5 This diagram illustrates the structure of an electronic device according to an embodiment of the present application, as shown below. Figure 5 As shown, the electronic device 60 includes a processor 61, a memory 62, and a program stored in the memory and capable of running on the processor. When the program is executed by the processor, it implements the steps of the target detection method provided in any of the above embodiments, or the steps of the training method of the target detection system provided in any of the above embodiments.

[0175] The electronic device 60 can be broadly defined as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities.

[0176] In one embodiment, the electronic device 60 may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the electronic device 60 can be used to provide necessary computing, processing, and / or control capabilities. The memory of the electronic device 60 may include non-volatile storage media and internal memory. The non-volatile storage media may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface and communication interface of the electronic device 60 can be used to connect and communicate with external devices via a network.

[0177] This application also provides a computer-readable storage medium storing instructions, wherein when executed by a processor, the instructions implement the steps of the target detection method provided in any of the above embodiments, or the steps of the training method of the target detection system provided in any of the above embodiments.

[0178] Those skilled in the art will understand that the method steps of this application can be performed by a computer program instructing related hardware, such as electronic devices or processors. The computer program can be stored in a non-transitory computer-readable storage medium, and its execution causes the steps of this application to be performed. Depending on the context, any reference herein to memory, storage, or other media may include non-volatile or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0179] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A target detection system, characterized by, The target detection system comprises a backbone network, a neck network and a head network implemented based on a computer program, The backbone network is configured to extract different scale features of an input image to generate multi-layer feature maps; The neck network is configured to perform feature fusion on the multi-layer feature maps to generate multi-layer feature fusion maps; The neck network comprises a double attention module, wherein the double attention module is configured to obtain a highest layer channel attention map based on a channel attention mechanism and generate a highest layer spatial attention map based on a spatial attention mechanism for a highest layer feature map corresponding to a feature map with N being the maximum value; and fuse the highest layer channel attention map and the highest layer spatial attention map to generate a highest layer feature fusion map; The neck network further comprises a plurality of intra-layer feature fusion modules, a plurality of cross-layer feature fusion modules and a plurality of up-sampling modules distributed on the lowest layer, the highest layer and the intermediate layers respectively; For each layer except the highest layer, the intra-layer feature fusion module of each layer is configured to perform intra-layer feature fusion on a feature fusion map of each layer and a feature map subjected to a convolution operation based on first and second spatial weights to obtain an intra-layer feature fusion map of each layer; for each layer except the lowest layer, the up-sampling module of each layer is configured to perform up-sampling on the intra-layer feature fusion map of each layer to obtain a sampling feature map, so that the resolution of the sampling feature map of each layer is equal to the resolution of a feature map of a corresponding next layer; for each layer except the lowest layer, the cross-layer feature fusion module of each layer is configured to perform cross-layer feature fusion on the sampling feature map of each layer and the feature map of the corresponding next layer based on a third spatial weight to obtain a feature fusion map of the next layer; The head network comprises a multi-stage refinement network and a detection network, wherein, The multi-stage refinement network is configured to generate initial region proposals about a detection target based on each of the multi-layer feature fusion maps, and iteratively refine each initial region proposal through the multi-stage refinement network in sequence to obtain final region proposals about the detection target in each of the multi-layer feature fusion maps; The detection network is configured to perform position regression and classification on the detection target based on the final region proposals to obtain the position and category of the detection target in the input image.

2. The object detection system of claim 1, wherein, The multi-stage refinement network comprises M cascaded stage networks, M is greater than or equal to 2, and the M cascaded stage networks at least comprise a first stage network and a final stage network, wherein, The first-stage network comprises a first dilated convolution layer, a first standard convolution layer, a first classification head, and a first regression head, the first dilated convolution layer is configured to perform context feature extraction on an Nth layer feature fusion map in the plurality of layer feature fusion maps to obtain a first-stage enhanced feature map, the first standard convolution layer is configured to refine local features of the first-stage enhanced feature map to obtain a first-stage optimized feature map, the first regression head is configured to obtain an initial bounding box of a detection target based on the first-stage optimized feature map, and the first classification head is configured to perform classification score prediction on a candidate region corresponding to the initial bounding box based on the first-stage optimized feature map, thereby obtaining an initial region proposal of the detection target at the first stage, wherein N is greater than or equal to 2; and / or The final-stage network comprises a variable convolution layer, a regression convolution layer, and a classification convolution layer, the variable convolution layer is configured to perform feature extraction on a previous-stage optimized feature map to obtain a final-stage feature map, the regression convolution layer is configured to perform position regression on the previous-stage region proposal based on the final-stage feature map to obtain a bounding box of the detection target, and the classification convolution layer is configured to perform classification prediction on a candidate region corresponding to the bounding box of the detection target after the position regression based on the final-stage feature map, thereby obtaining a final region proposal of the detection target of the Nth layer feature fusion map.

3. The target detection system of claim 2, wherein, The M cascaded stage networks further comprise at least one intermediate stage network, wherein The intermediate stage network comprises an intermediate dilated convolution layer, an intermediate standard convolution layer, an intermediate classification head, and an intermediate regression head, the intermediate dilated convolution layer is configured to perform context feature extraction on a previous-stage optimized feature map to obtain an intermediate-stage enhanced feature map, the intermediate standard convolution layer is configured to refine candidate region features of the intermediate-stage enhanced feature map to obtain an intermediate-stage optimized feature map, and the intermediate classification head and the intermediate regression head are configured to update the previous-stage region proposal based on the intermediate-stage optimized feature map to obtain an intermediate-stage region proposal.

4. The object detection system of claim 1, wherein, The sum of the first spatial weight, the second spatial weight, and the third spatial weight is 1.

5. The object detection system of claim 1, wherein, The detection network further comprises a pooling layer, a fully connected layer, a final classification layer, and a final regression layer; wherein The pooling layer is configured to perform a pooling operation based on the final region proposal and each layer feature fusion map to obtain a plurality of normalized feature maps; The fully connected layer is configured to perform full connection on each normalized feature map to obtain a fixed-dimension feature vector corresponding to each normalized feature map; The final classification layer is configured to perform classification based on a plurality of the fixed-dimension feature vectors to determine the category of the detection target in the input image; The final regression layer is configured to perform position regression based on a plurality of the fixed-dimension feature vectors to determine the position of the detection target in the input image.

6. A target detection method characterized by, The target detection method comprises: extracting different scale features of an input image using a backbone network to generate a plurality of layer feature fusion maps; perform feature fusion on the multi-layer feature maps using a neck network to generate multi-layer feature fusion maps; perform initial region proposal on the detection target based on each of the multi-layer feature fusion maps using a multi-stage refinement network, and iteratively refine each initial region proposal through the multi-stage refinement network in sequence to obtain final region proposal on the detection target in each of the multi-layer feature fusion maps; perform position regression and classification on the detection target based on the final region proposal using a detection network to obtain the position and category of the detection target in the input image; the neck network comprises a dual attention module, and the performing feature fusion on the multi-layer feature maps using the neck network to generate multi-layer feature fusion maps comprises: performing channel attention mechanism based on the dual attention module to obtain a highest layer channel attention map and performing spatial attention mechanism to generate a highest layer spatial attention map for a highest layer feature map, which corresponds to a feature map with N being the maximum value; and fusing the highest layer channel attention map and the highest layer spatial attention map to generate a highest layer feature fusion map. The neck network further comprises a plurality of intra-layer feature fusion modules, a plurality of cross-layer feature fusion modules and a plurality of up-sampling modules distributed on the lowest layer, the highest layer and the intermediate layers respectively, and the performing feature fusion on the multi-layer feature maps using the neck network to generate multi-layer feature fusion maps further comprises: for each layer except the highest layer, performing intra-layer feature fusion on the feature fusion map of each layer and the feature map after convolution operation based on the first spatial weight and the second spatial weight using the intra-layer feature fusion module of each layer to obtain an intra-layer feature fusion map of each layer; for each layer except the lowest layer, performing up-sampling on the intra-layer feature fusion map of each layer using the up-sampling module of each layer to obtain a sampling feature map, so that the resolution of the sampling feature map of each layer is equal to the resolution of the feature map of the corresponding next layer; for each layer except the lowest layer, performing cross-layer feature fusion on the sampling feature map of each layer and the feature map of the corresponding next layer based on the third spatial weight using the cross-layer feature fusion module of each layer to obtain a feature fusion map of the next layer.

7. The object detection method of claim 6, wherein, The multi-stage refinement network comprises M cascaded stage networks, M is greater than or equal to 2, and the M cascaded stage networks at least comprise a first stage network and a final stage network, wherein the first stage network comprises a first dilated convolution layer, a first standard convolution layer, a first classification head and a first regression head, and the final stage network comprises a variable convolution layer, a regression convolution layer and a classification convolution layer. The performing initial region proposal on the detection target based on each of the multi-layer feature fusion maps using the multi-stage refinement network, and iteratively refining each initial region proposal through the multi-stage refinement network in sequence to obtain final region proposal on the detection target in each of the multi-layer feature fusion maps comprises: The first dilated convolution of the first stage network is used for context feature extraction on the Nth feature fusion graph in the multi-layer feature fusion graph, to obtain a first stage enhanced feature graph; a first standard convolution layer is used for local feature refinement on the first stage enhanced feature graph, to obtain a first stage optimized feature graph; a first regression head is used to obtain an initial bounding box of a detection target based on the first stage optimized feature graph; and a first classification head is used to perform classification score prediction on a candidate region corresponding to the initial bounding box based on the first stage optimized feature graph, to obtain an initial region proposal of the detection target in the first stage, wherein N is greater than or equal to 2; A variable convolution layer of the final stage network is used for feature extraction on a previous stage optimized feature graph, to obtain a final stage feature graph; a regression convolution layer is used to perform position regression on the previous stage region proposal based on the final stage feature graph, to obtain a bounding box of the detection target; and a classification convolution layer is used to perform classification prediction on a candidate region corresponding to the bounding box of the detection target after the position regression, to obtain a final region proposal of the detection target in the Nth feature fusion graph.

8. The object detection method of claim 7, wherein, The M cascaded stage networks further comprise at least one intermediate stage network, wherein The intermediate stage network comprises an intermediate dilated convolution layer, an intermediate standard convolution layer, an intermediate classification head, and an intermediate regression head, The method further comprises: The intermediate dilated convolution layer is used for context feature extraction on a previous stage optimized feature graph, to obtain an intermediate stage enhanced feature graph; the intermediate standard convolution layer is used for candidate region feature refinement on the intermediate stage enhanced feature graph, to obtain an intermediate stage optimized feature graph; and the intermediate classification head and the intermediate regression head are used to update a previous stage region proposal based on the intermediate stage optimized feature graph, to obtain an intermediate stage region proposal.

9. The object detection method of claim 6, wherein, The sum of the first spatial weight, the second spatial weight, and the third spatial weight is 1.

10. A training method for a target detection system according to any one of claims 1-5, characterized by, The training method comprises a basic training step and a fine-tuning step. The basic training step comprises training network parameters of a backbone network, a neck network, and a head network in an initial target detection system using basic class data, to obtain a basic trained target detection system. The fine-tuning step comprises training network parameters of the neck network and the head network in the basic trained target detection system, except for the backbone network, using the basic class data and small sample data containing at least one new class object, to obtain a fine-tuned target detection system.

11. The training method of claim 10, wherein, The basic training step further comprises applying a sparse activation mechanism to a placeholder node in a final classification layer in the initial target detection system, to limit the activity of the placeholder node.

12. The training method of claim 10, wherein, The method comprises the following steps of: training network parameters of a backbone network, a neck network and a head network in an initial target detection system by using basic class data, to obtain a basic trained target detection system; inputting the basic class data into the initial target detection system, and iteratively updating the network parameters of the backbone network, the neck network and the head network in the initial target detection system, to obtain a minimized total basic loss, so as to obtain the basic trained target detection system. The total basic loss comprises a basic classification loss and a basic regression loss, the basic classification loss comprises a generalized classification loss of the basic class generated by the detection network and a classification loss of the basic class generated by a final stage network of the multi-stage refinement network, and the generalized classification loss of the basic class comprises a standard cross-entropy loss of the basic class and a sparse regularization loss applied to a placeholder node in a classification layer in the initial target detection system; and the basic regression loss comprises a regression loss generated by the detection network and a regression loss generated by each stage network of the multi-stage refinement network. The method comprises the following steps of: training network parameters of a backbone network, a neck network and a head network in an initial target detection system by using basic class data, to obtain a basic trained target detection system; inputting the basic class data into the initial target detection system, and iteratively updating the network parameters of the backbone network, the neck network and the head network in the initial target detection system, to obtain a minimized total basic loss, so as to obtain the basic trained target detection system.

13. The training method according to any one of claims 10-12, characterized in that, The method comprises the following steps of: freezing the parameters of the backbone network in the basic trained target detection system; activating the placeholder node in the classification layer of the basic trained target detection system; inputting the basic class data and the small sample data containing at least one new class object into the activated basic trained target detection system, and iteratively updating the parameters of the unfrozen network in the activated basic trained target detection system, to obtain a minimized total fine-tuning loss, so as to obtain a fine-tuned target detection system. The total fine-tuning loss comprises classification loss and regression loss of the basic class and the new class, the classification loss of the basic class and the new class comprises a generalized classification loss of the classification loss of the basic class and the new class generated by the detection network and a classification loss of the basic class and the new class generated by the final stage network of the multi-stage refinement network, and the generalized classification loss of the classification loss of the basic class and the new class comprises the classification loss of the basic class, the classification loss of the new class and the sparse regularization loss of the placeholder node generated by the detection network; and the regression loss of the basic class and the new class comprises the regression loss of the basic class and the new class generated by the detection network and the regression loss of the basic class and the new class generated by each stage network of the multi-stage refinement network. The program is executed by the processor to implement the steps of the target detection method according to any one of claims 6-9, or the steps of the training method for the target detection system according to any one of claims 10-14. ​ 14. The training method of claim 13, wherein, ​ 15. An electronic device, comprising: ​ 16. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, which, when executed by a processor, implement the steps of the target detection method according to any one of claims 6-9, or implement the steps of the training method of the target detection system according to any one of claims 10-14.

Images