Remote sensing target detection method and device, computing device, and medium

Abstract

The present disclosure provides a remote sensing target detection method and device, and relates to the field of computer vision. The method comprises the following steps: obtaining sample prediction information of a sample by an initial target detection network; obtaining network loss of the initial target detection network by using the sample prediction information; obtaining a target detection network by network training using the network loss; and inputting an image of a detection target into the target detection network to obtain a detection result of the detection target. The present disclosure effectively improves the network's missed detection phenomenon for small targets, accelerates the convergence speed of the network, and improves the detection accuracy by using a branch communication mode based on an anchor-free mechanism.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and remote sensing target detection, and in particular to a remote sensing target detection method, apparatus, computing device, and medium. Background Technology

[0002] Remote sensing image target detection, a fundamental and challenging task in computer vision, has been widely applied in both military and civilian fields, such as UAV reconnaissance, maritime rescue, and material defect detection. Small targets in remote sensing images have a small number of pixels and occupy a very small proportion of the overall image, making it difficult for networks to accurately detect them in complex backgrounds.

[0003] Currently, deep learning-based object detectors can be broadly categorized into anchored detectors and anchorless detectors. Anchored detectors can be further divided into horizontal anchor-box detectors and rotating anchor-box detectors. Horizontal anchor-box detectors offer high detection accuracy but are slow. Small objects of different shapes have very few pixels, resulting in a small pixel proportion within the horizontal ground truth box. This leads to less effective information during feature extraction, hindering the network's ability to distinguish foreground from background. Rotating anchor-box detectors face greater learning difficulty and convergence challenges. Furthermore, anchor-box-based detectors rely heavily on pre-defined anchor box sizes, making it easy to miss small objects. Anchorless detectors, on the other hand, eliminate the reliance on prior anchor boxes, transforming object detection into keypoint estimation. This allows for more granular image detection, mitigating the problem of missed small objects. However, the simultaneous prediction of positional and category information at the network output can distract the network, leading to convergence difficulties and increasing the complexity of the model's application. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of this, the purpose of this disclosure is to provide a remote sensing target detection method, apparatus, computing device and medium to at least partially solve the above-mentioned technical problems.

[0006] (II) Technical Solution

[0007] According to one aspect of this disclosure, a remote sensing target detection method is provided, comprising:

[0008] Based on a two-dimensional Gaussian distribution model of the target sample, sample prediction information of the sample is obtained through an initial target detection network; the network loss of the initial target detection network is obtained using the sample prediction information; the network loss is used to train the initial target detection network to obtain a target detection network; and the image of the target is input into the target detection network to obtain the detection result of the target.

[0009] Furthermore, the initial object detection network includes a feature extraction network, a localization branch network, a category branch network, and a bounding box size regression network. Sample prediction information includes a sample localization heatmap, sample category prediction information, and sample bounding box prediction information. Based on a two-dimensional Gaussian distribution model of the detected target samples, the sample prediction information obtained through the initial object detection network includes: using the feature extraction network to extract features based on the two-dimensional Gaussian distribution model to obtain a shallow feature map of the sample; using the localization branch network to obtain a sample localization heatmap with decreasing receptive field based on the shallow feature map; using the category branch network to obtain sample category prediction information based on the shallow feature map and the sample localization heatmap; and using the bounding box size regression network to perform feature fusion based on the shallow feature map and the sample localization heatmap to obtain sample bounding box prediction information.

[0010] Furthermore, obtaining sample category prediction information using the class branch network based on the shallow feature map and sample localization heatmap includes: inputting the shallow feature map into the deconvolution network in the class branch network to obtain a deep feature map with a reduced receptive field; and fusing the deep feature map, sample localization heatmap, and shallow feature map using the class branch network to obtain sample category prediction information.

[0011] Furthermore, the localization branch network includes a CBL nonlinear layer and a nearest neighbor interpolation upsampling unit. Obtaining a sample localization heatmap with decreasing receptive field based on the shallow feature map using the localization branch network includes: Step 1, the shallow feature map passes through the CBL nonlinear layer and the nearest neighbor interpolation upsampling unit to obtain a feature map with increased resolution; Step 2, the feature map with increased resolution is fused with the shallow feature map to obtain a heatmap with reduced receptive field; Step 3, using the heatmap obtained in Step 2 as input, Step 1 and Step 2 are repeated step by step to obtain a heatmap with decreasing receptive field.

[0012] Furthermore, shallow feature maps contain detailed information about the samples; deep feature maps contain semantic information about the samples.

[0013] Furthermore, the network loss includes center point localization loss, category loss, and bounding box size regression loss. The network loss of the initial object detection network obtained using sample prediction information includes: calculating the MSE loss using the sample localization heatmap and the heatmap of the preset two-dimensional Gaussian distribution to obtain the center point localization loss; calculating the binary cross-entropy loss using the sample category prediction information and the preset label information to obtain the category loss; and calculating the L1 loss using the sample bounding box prediction information and the preset bounding box size label information to obtain the bounding box size regression loss.

[0014] Furthermore, training the initial object detection network using network loss includes: training the localization branch network and the category branch network according to the network loss to obtain the convolutional kernel weights of the localization branch network and the category branch network; fixing the convolutional kernel weights of the localization branch network and the category branch network, training the bounding box size regression network according to the network loss to obtain the convolutional kernel weights of the bounding box size regression network; and attaching the convolutional kernel weights of the localization branch network, the category branch network, and the bounding box size regression network to the initial object network to obtain the object detection network.

[0015] Furthermore, inputting the image of the target to the target detection network to obtain the detection result includes: passing the image of the target through the target detection network to obtain a localization heatmap, category prediction information, and bounding box prediction information; normalizing the localization heatmap to obtain a normalized localization heatmap; obtaining the center point coordinates of the target from the normalized localization heatmap using a maximum value filter; obtaining the width and height values ​​of the bounding box of the target from the bounding box prediction information using the center point coordinates; obtaining the confidence scores of each category under the center point coordinates from the category prediction information using the center point coordinates; taking the maximum value among the confidence scores of each category as the confidence score of the target, and the category to which the confidence score belongs is the category to which the target belongs.

[0016] According to another aspect of this disclosure, a remote sensing target detection device is provided, comprising: a first acquisition module for obtaining sample prediction information of samples based on a two-dimensional Gaussian distribution model of samples of the target to be detected; a second acquisition module for obtaining network loss of a target detection network module using the sample prediction information; a third acquisition module for training the target detection network module using the network loss to obtain a target detection network; and a fourth acquisition module for inputting an image of the target to be detected into the target detection network to obtain the detection result of the target.

[0017] According to another aspect of this disclosure, a computing device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0018] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores executable instructions thereon, which, when executed by a processor, cause the processor to implement the method described above.

[0019] (III) Beneficial Effects

[0020] (1) This disclosure improves the modeling of sample labels by transforming the bounding box modeling in traditional technology into two-dimensional Gaussian distribution modeling. This eliminates the dependence on anchor boxes in traditional technology and avoids the situation where small target objects are judged as negative samples due to slight positional deviations of the prediction box when the target size is different. This increases the coverage of the labels on the samples and reduces the misguidance of the network by background features.

[0021] (2) This disclosure shares the foreground information predicted by the positioning branch, i.e. the generated heat map, with the category branch. At the same time, the network loss of each branch is weighted to achieve rapid convergence of the network.

[0022] (3) The present invention integrates detailed features in the positioning branch to achieve pixel-level prediction of remote sensing images, thereby improving the network’s ability to miss small targets.

[0023] (4) The category branch network disclosed herein uses deconvolution to reduce the receptive field of deep features, focuses on the semantic information of small targets, and improves detection accuracy. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a remote sensing target detection method according to an embodiment of the present disclosure is shown schematically.

[0025] Figure 2 A flowchart illustrating the process of obtaining sample prediction information according to an embodiment of the present disclosure is shown.

[0026] Figure 3 A flowchart illustrating the process of obtaining detection results of a target according to an embodiment of the present disclosure is shown.

[0027] Figure 4 A schematic diagram of the network structure of an object detection network according to an embodiment of the present disclosure is shown.

[0028] Figure 5 A block diagram of a remote sensing target detection apparatus according to an embodiment of the present disclosure is shown schematically.

[0029] Figure 6 A block diagram of a computer system suitable for implementing the methods described above, according to embodiments of the present disclosure, is illustrated schematically. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0031] However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of embodiments of this disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this disclosure.

[0032] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The term "comprising" as used herein indicates the presence of features, steps, or operations, but does not exclude the presence or addition of one or more other features.

[0033] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0034] The accompanying drawings illustrate several block diagrams and / or flowcharts. It should be understood that some blocks, or combinations thereof, in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that, when executed by the processor, these instructions can create means for implementing the functions / operations described in these block diagrams and / or flowcharts. The technology of this disclosure can be implemented in hardware and / or software (including firmware, microcode, etc.). Alternatively, the technology of this disclosure can take the form of a computer program product stored on a computer-readable storage medium, which is available for use by or in conjunction with an instruction execution system.

[0035] The following is an illustrative example illustrating a remote sensing target detection method, apparatus, computing device, and medium. It should be noted that this illustration is merely a specific embodiment of this disclosure and does not limit the scope of protection of this disclosure.

[0036] Figure 1 A flowchart illustrating a remote sensing target detection method according to an embodiment of the present disclosure is shown schematically.

[0037] like Figure 1 As shown, the method includes steps S1 to S4.

[0038] Step S1: Based on the two-dimensional Gaussian distribution model of the sample of the detected target, obtain the sample prediction information of the sample through the initial target detection network.

[0039] According to the embodiments of this disclosure, the sample label modeling method is changed from the traditional box modeling to a two-dimensional Gaussian distribution model. The feature maps obtained by the initial target detection network are all generated based on the two-dimensional Gaussian distribution model of the detected target or the detected target sample. Each branch network in the initial target detection network, such as the feature extraction network, the localization branch network, the category branch network, and the bounding box size regression network, also contains the corresponding two-dimensional Gaussian distribution feature maps.

[0040] According to embodiments of this disclosure, the Gaussian kernel function expression for a two-dimensional Gaussian distribution is as follows:

[0041]

[0042]

[0043] Where α1 is the normalization coefficient; x0 and y0 are the center coordinates of the Gaussian kernel; a, b, and c are the control coefficients for the distribution magnitude and direction, and their calculation methods are as follows:

[0044]

[0045]

[0046]

[0047] Where θ is the rotation angle of the Gaussian kernel (bounding box); σ x and σ y The maximum value of the Gaussian kernel function is related to the bounding box size (w, h) of the target object.

[0048] According to embodiments of this disclosure, θ = 0, σ x Set as σ y Set as max(G(x, y)) is 1.

[0049] According to embodiments of this disclosure, the initial object detection network may include, for example, a feature extraction network, a localization branch network, a category branch network, and a bounding box size regression network.

[0050] According to embodiments of this disclosure, sample prediction information may include, for example, sample location heatmaps, sample category prediction information, and sample bounding box prediction information.

[0051] Based on this, step S1 may include obtaining sample location heatmaps from the location branch network, obtaining sample category prediction information from the category branch network, and obtaining sample bounding box prediction information from the bounding box size regression network.

[0052] Step S2: Use the sample prediction information to obtain the network loss of the initial target detection network.

[0053] According to embodiments of this disclosure, network loss may include, for example, center point localization loss, corresponding to the loss function Loss. center Category loss, corresponding to the loss function Loss class ; and the loss function corresponding to the bounding box size regression loss. box Among them, the center point localization loss is the error information between the sample localization heatmap obtained by the localization branch network and the preset localization label information (the preset heatmap containing the two-dimensional Gaussian distribution of each sample), the category loss is the error information between the sample category prediction information obtained by the category branch network and the preset category label information, and the bounding box size regression loss is the error information between the sample bounding box prediction information obtained by the bounding box size regression network and the preset bounding box size label information.

[0054] Based on this, the network loss obtained in step S2 specifically includes: calculating the MSE loss using the sample location heatmap and the preset two-dimensional Gaussian distribution heatmap to obtain the center point location loss; calculating the binary cross-entropy loss using the sample category prediction information and the preset category label information to obtain the category loss; and calculating the L1 loss using the sample bounding box prediction information and the preset bounding box size label information to obtain the bounding box size regression loss.

[0055] Step S3: Train the initial object detection network using network loss to obtain the object detection network.

[0056] According to embodiments of this disclosure, training the initial object detection network specifically includes:

[0057] After determining the weights of the loss function corresponding to the center point localization loss and the weights of the loss function corresponding to the class loss, the total loss of the network is as follows:

[0058] Loss = α2Loss center +βLoss class

[0059] Where α2 is the Loss class The weights, β is the loss box The weights, α2 and β, are selected based on the magnitude of the network loss and the sample prediction information.

[0060] The localization branch network and the category branch network are trained based on the network loss and loss function to obtain the convolutional kernel weights of the localization branch network and the convolutional kernel weights of the category branch network.

[0061] With α2 and β values ​​fixed, and the convolutional kernel weights of the localization branch network and the class branch network also fixed, the total loss of the network is as follows:

[0062] Loss = α2Loss center +βLoss class +γLoss box

[0063] Where γ is the loss box The weights γ are selected based on the magnitude of the network loss and the sample prediction information.

[0064] The bounding box size regression network is trained based on the network loss and loss function to obtain the convolutional kernel weights of the bounding box size regression network.

[0065] The convolutional kernel weights of the localization branch network, the convolutional kernel weights of the category branch network, and the convolutional kernel weights of the bounding box size regression network are added to the initial target network to obtain the target detection network. Specifically, the convolutional kernel weights of the localization branch network are added to the localization branch network, the convolutional kernel weights of the category branch network are added to the category branch network, and the convolutional kernels of the bounding box size regression network are added to the bounding box size regression network.

[0066] Step S4: Input the image of the target to be detected into the target detection network to obtain the detection result of the target.

[0067] According to embodiments of this disclosure, after inputting an image of the target to be detected, the output of the target detection network includes, for example, a localization heatmap, category prediction information, and bounding box prediction information.

[0068] According to embodiments of this disclosure, the detection results include, for example, the coordinates of the center point of the detected target, the width and height values ​​of the bounding box of the detected target, the confidence level of the detected target, and the category to which the detected target belongs.

[0069] Based on this, step S4 specifically includes passing the image of the detected target through a target detection network to obtain a localization heatmap, category prediction information, and bounding box prediction information; and performing post-processing on the localization heatmap, category prediction information, and bounding box prediction information to obtain the detection result of the detected target.

[0070] The following is for reference. Figures 2-3 In conjunction with specific embodiments, Figure 1 The method shown will be further explained.

[0071] Figure 2 A flowchart illustrating the process of obtaining sample prediction information according to an embodiment of the present disclosure is shown.

[0072] According to embodiments of this disclosure, obtaining sample prediction information of samples through an initial target detection network based on a two-dimensional Gaussian distribution model of the sample of the detected target includes steps S101 to S104.

[0073] Step S210: Based on the two-dimensional Gaussian distribution model, feature extraction is performed using a feature extraction network to obtain a shallow feature map of the sample.

[0074] According to an embodiment of this disclosure, the input tensor is O0, the output tensor of the first layer is O1, and then the output of the previous layer is used as the input of the next layer. For example, the input of the second layer is O1 and the output is O2, the input of the third layer is O2 and the output is O3, and so on, to obtain the output tensors O1 to O2. 10 This refers to the shallow feature map.

[0075] According to embodiments of this disclosure, the correspondence between modules in the feature extraction network, the number of repetitions of feature extraction by the modules, the input tensor, the number of output channels, and the size of the output feature map is shown in Table 1.

[0076] Table 1 Feature Extraction Network

[0077]

[0078] Step S220: Based on the shallow feature map, use the localization branch network to obtain the sample localization heatmap with decreasing receptive field.

[0079] According to embodiments of this disclosure, the input image has a size of W×H, and the output of the feature extraction network is... After passing through a CBL nonlinear layer, the feature map resolution is increased by a nearest neighbor interpolation upsampling unit to obtain the feature map. Next, compare with the feature map containing detailed information. The features are concatenated along the channel dimension, and then fused using the C3 module to obtain a feature map with a larger receptive field. And so on, O 15 After passing through a CBL nonlinear layer, the feature map resolution is increased by adjacent interpolation upsampling units to obtain the feature map. Next, compare with the feature map containing detailed information. The features are concatenated along the channel dimension, and then fused using the C3 module to obtain a feature map with a larger receptive field. O 20 After passing through a CBL nonlinear layer, the feature map resolution is increased by adjacent interpolation upsampling units to obtain the feature map. Next, compare with the feature map containing detailed information. The features are concatenated along the channel dimension, and then fused using the C3 module to obtain a feature map with a larger receptive field. This yields a feature map O showing a progressively decreasing receptive field. 15 O 20 and O 25 That is, the sample location heatmap.

[0080] According to embodiments of this disclosure, the correspondence between modules in the localization branch network, the number of repetitions of feature extraction by the modules, the input tensor, the number of output channels, and the size of the output feature map is shown in Table 2.

[0081] Table 2 Location Branch Network

[0082]

[0083] Step S230: Obtain sample category prediction information using a category branching network based on the shallow feature map and sample localization heatmap.

[0084] According to embodiments of this disclosure, the category branching network utilizes deconvolution to reduce the receptive field (receptive field: O) of deep feature maps. 26 >O 27 >O 28 With the assistance of the shallow feature map O3 containing a lot of detailed information, local category information related to small targets is further obtained, so as to achieve category prediction of small targets.

[0085] According to embodiments of this disclosure, inter-branch communication is designed, utilizing the Concat operation to transmit the prediction information of the foreground and background of the image from the localization branch network to the category branch network, that is, the heatmap O output by the localization branch network. 25 Deep feature maps O in class branch networks 28 Feature fusion is performed using the shallow feature map O3 to accelerate the convergence speed of the network.

[0086] According to embodiments of this disclosure, the output tensor of the class branching network is That is, the sample category prediction information, where Cls represents the number of categories.

[0087] According to embodiments of this disclosure, the correspondence between modules in the category branch network, the number of repetitions of feature extraction by the modules, the input tensor, the number of output channels, and the size of the output feature map is shown in Table 3.

[0088] Table 3. Category Branch Networks

[0089]

[0090] Step S240: Based on the shallow feature map and the sample localization heatmap, feature fusion is performed using a bounding box size regression network to obtain sample bounding box prediction information.

[0091] According to embodiments of this disclosure, a bounding box size regression network is cascaded with a localization branch network and a feature extraction network.

[0092] According to embodiments of this disclosure, the heatmap O output by the positioning branch network is... 25 As input, the shallow feature map O3, which retains detailed information about the target object, is fused to achieve accurate prediction of the bounding box size, further reducing the positional deviation of the predicted box and improving the problem of missed detection caused by positive samples being judged as negative samples.

[0093] According to embodiments of this disclosure, the output of the bounding box size regression network is: That is, the sample bounding box prediction information.

[0094] According to embodiments of this disclosure, the correspondence between modules in the bounding box size regression network, the number of repetitions of feature extraction by the modules, the input tensor, the number of output channels, and the output feature map size is shown in Table 4.

[0095] Table 4 Bounding Box Regression Network Structure

[0096]

[0097] Figure 3 A flowchart illustrating the process of obtaining detection results of a target according to an embodiment of the present disclosure is shown.

[0098] Step S310: The image of the detected target is processed through the target detection network to obtain the localization heatmap, category prediction information and bounding box prediction information.

[0099] According to embodiments of this disclosure, the localization heatmap is obtained using a localization branch network trained by the aforementioned network, the category prediction information is obtained using a category branch network trained by the aforementioned network, and the bounding box prediction information is obtained using a bounding box size regression network trained by the aforementioned network. The process of obtaining the localization heatmap, category prediction information, and bounding box prediction information from the image of the detected target through the target detection network is the same as the process of obtaining the sample prediction information of the sample by the target detection network.

[0100] Step S320: Normalize the location heatmap to obtain the normalized location heatmap.

[0101] According to embodiments of this disclosure, the heatmap O output by the positioning branch network is... 25 Perform normalization; set the confidence level c threshold T = 0.0001, and set the predicted values ​​with confidence levels less than T to zero.

[0102] Step S330: Use the maximum value filter to obtain the center point coordinates of the detected target from the normalized positioning heatmap.

[0103] According to embodiments of this disclosure, a maximum value filter (maxmum) is used to detect the maximum value of pixels within a local region of an image, and the peak point is the center point P of the predicted target. center The coordinates (x, y).

[0104] Step S340: Use the center point coordinates to obtain the confidence scores of each category under the center point coordinates from the category prediction information.

[0105] According to embodiments of this disclosure, the feature map O output from the category branch network is obtained using the center point coordinates (x, y). 31 Obtain the confidence scores (Conf) for each category under this coordinate.

[0106] Step S350: Take the maximum value among the confidence scores of each category as the confidence score of the target being detected, and the category to which the confidence score belongs is the category to which the target being detected belongs.

[0107] According to embodiments of this disclosure, Conf is taken as its maximum value to obtain the class to which the target object belongs. index .

[0108] Step S360: Use the center point coordinates to obtain the width and height values ​​of the bounding box of the detected target from the bounding box prediction information.

[0109] According to embodiments of this disclosure, the width and height values ​​B corresponding to the bounding box prediction output tensor are obtained using the center point coordinates (x, y) of the target object. (w，h) .

[0110] The pseudocode for steps S320 to S360 (i.e., post-processing of the output of the target detection network) is shown in Table 5:

[0111] Table 5 Post-processing

[0112]

[0113] Figure 4 A schematic diagram of the network structure of a target detection network according to an embodiment of the present disclosure is shown.

[0114] like Figure 4 As shown, the initial object detection network before network training and the object detection network after network training both include the same network branches, including: feature extraction network, localization branch network, category branch network, and bounding box size regression network.

[0115] According to embodiments of this disclosure, the feature extraction network includes the first 10 layers of the YOLOv5s network; the first 10 layers of the YOLOv5s network include the Focus module, the CBL module, the C3_res module, and the SPP module.

[0116] According to embodiments of this disclosure, there are two types of C3 structures: the first structure consists of a CBL module and a bottleneck structure with residuals, referred to as C3_res, and is used only for feature extraction networks; the second C3 structure includes a CBL module and a bottleneck structure without residuals, referred to as C3_norm (hereinafter referred to as C3 in the accompanying drawings).

[0117] According to embodiments of this disclosure, the category branch network includes a CBL module and a deconvolution (DeConv) module.

[0118] According to embodiments of this disclosure, the CBL module includes a two-dimensional convolution operation, a BatchNorm layer, and a Leaky_Relu activation function.

[0119] According to embodiments of this disclosure, the Concat operation has the functions of information transmission and feature fusion.

[0120] Table 6 illustrates the performance comparison of different schemes, including this one, on the dataset.

[0121] Table 6 Performance comparison of different schemes on the dataset

[0122]

[0123] In Table 6, AP represents the percentage of accuracy, and mAP represents the average percentage of accuracy.

[0124] According to the embodiments of this disclosure, each scheme in Table 6 has undergone 30 iterations. The technical solution of this disclosure is significantly more accurate than other schemes in detecting small targets (cars).

[0125] Figure 5 A block diagram of a product recommendation device according to an embodiment of the present disclosure is shown schematically.

[0126] like Figure 5 As shown, the remote sensing target detection device 500 includes a first acquisition module 510, a second acquisition module 520, a third acquisition module 530, and a fourth acquisition module 540. This remote sensing target detection device 500 can perform the operations described above. Figures 1-3 The method described.

[0127] Specifically, the first acquisition module 510 is used to obtain sample prediction information of the samples based on the two-dimensional Gaussian distribution model of the samples of the detected target; the second acquisition module 520 is used to obtain the network loss of the target detection network module using the sample prediction information; the third acquisition module 530 is used to train the target detection network module using the network loss to obtain the target detection network; and the fourth acquisition module 540 is used to input the image of the detected target into the target detection network to obtain the detection result of the detected target.

[0128] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), Systems-on-Chip, Systems-on-Substrate, Systems-on-Package, Application-Specific Integrated Circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0129] Figure 6 A block diagram of a computer system suitable for implementing the methods described above, according to embodiments of the present disclosure, is illustrated schematically. Figure 6 The computer system shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0130] like Figure 6 As shown, the computer system 600 includes a processor 610 and a computer-readable storage medium 620. The computer system 600 can perform methods according to embodiments of the present disclosure.

[0131] Specifically, processor 610 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 610 may also include onboard memory for caching purposes. Processor 610 may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this disclosure.

[0132] Computer-readable storage medium 620 may be a non-volatile computer-readable storage medium, including but not limited to: magnetic storage devices such as magnetic tape or hard disk (HDD); optical storage devices such as optical disc (CD-ROM); memory such as random access memory (RAM) or flash memory; and so on.

[0133] Computer-readable storage medium 620 may include computer program 621, which may include code / computer-executable instructions that, when executed by processor 610, cause processor 610 to perform a method or any variation thereof according to embodiments of the present disclosure.

[0134] Computer program 621 may be configured to have computer program code, for example, including computer program modules. For example, in an exemplary embodiment, the code in computer program 621 may include one or more program modules, such as 621A, module 621B, ... It should be noted that the division and number of modules are not fixed. Those skilled in the art can use appropriate program modules or combinations of program modules according to actual circumstances. When these combinations of program modules are executed by processor 610, processor 610 can perform the methods according to embodiments of this disclosure or any variations thereof.

[0135] According to embodiments of this disclosure, at least one of the first acquisition module 510, the second acquisition module 520, the third acquisition module 530, and the fourth acquisition module 540 can be implemented as follows: Figure 6 The described computer program module, when executed by processor 610, can perform the corresponding operations described above.

[0136] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0137] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0138] The embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. It should be noted that implementations not illustrated or described in the drawings or the main text of the specification are forms known to those skilled in the art and have not been described in detail. Furthermore, the definitions of the various components described above are not limited to the specific structures, shapes, or methods mentioned in the embodiments, and those skilled in the art can easily modify or substitute them.

[0139] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0140] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this disclosure. It should be understood that the above descriptions are merely specific embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for detecting a remote sensing target, characterized in that, The method comprises the following steps: obtaining sample prediction information of a sample of a detection target based on a two-dimensional Gaussian distribution model of the sample of the detection target through an initial target detection network; obtaining a network loss of the initial target detection network by using the sample prediction information; obtaining a target detection network by network training of the initial target detection network by using the network loss; obtaining a detection result of the detection target by inputting an image of the detection target into the target detection network; the initial target detection network comprises a feature extraction network, a positioning branch network, a category branch network and a bounding box size regression network, the sample prediction information comprises sample positioning heat map, sample category prediction information and sample bounding box prediction information, and the two-dimensional Gaussian distribution model of the sample of the detection target based on the detection target comprises the following steps: obtaining shallow feature maps of the sample by feature extraction by using the feature extraction network based on the two-dimensional Gaussian distribution model; obtaining sample positioning heat maps with decreasing receptive fields by using the positioning branch network according to the shallow feature maps; obtaining sample category prediction information by using the category branch network according to the shallow feature maps and the sample positioning heat maps; obtaining sample bounding box prediction information by feature fusion by using the bounding box size regression network according to the shallow feature maps and the sample positioning heat maps.

2. The method of claim 1, wherein, The method for obtaining sample category prediction information by using the category branch network according to the shallow feature maps and the sample positioning heat maps comprises the following steps: inputting the shallow feature maps into a deconvolution network in the category branch network to obtain deep feature maps with reduced receptive fields; obtaining the sample category prediction information by feature fusion of the deep feature maps, the sample positioning heat maps and the shallow feature maps by using the category branch network.

3. The method of claim 1, wherein, The positioning branch network comprises a CBL nonlinear layer and a nearest neighbor interpolation up-sampling unit, and the method for obtaining sample positioning heat maps with decreasing receptive fields by using the positioning branch network according to the shallow feature maps comprises the following steps: S1, inputting feature maps into the CBL nonlinear layer and the nearest neighbor interpolation up-sampling unit to obtain feature maps with increased resolution; S2, performing feature fusion of the feature maps with increased resolution and the shallow feature maps to obtain heat maps with reduced receptive fields; S3, taking the heat maps with reduced receptive fields as the feature maps, and repeatedly performing S1 and S2 to obtain heat maps with decreasing receptive fields.

4. The remote sensing target detection method according to claim 2, wherein: the shallow feature maps contain detail information of the sample; the deep feature maps contain semantic information of the sample.

5. The method of claim 1, wherein, The network loss comprises a center point positioning loss, a category loss and a bounding box size regression loss, and the method for obtaining the network loss of the initial target detection network by using the sample prediction information comprises the following steps: obtaining the center point positioning loss by performing MSE loss calculation of the sample positioning heat maps and a preset heat map of a two-dimensional Gaussian distribution; obtaining the category loss by performing binary cross-entropy loss calculation of the sample category prediction information and a preset category label information; The sample bounding box prediction information is used for L1 loss calculation with preset bounding box size label information, to obtain a bounding box size regression loss.

6. The method of claim 5, wherein, The network loss is used for network training of the initial target detection network, to obtain the target detection network. The network loss is used for network training of the positioning branch network and the category branch network, to obtain the convolution kernel weight of the positioning branch network and the convolution kernel weight of the category branch network. The convolution kernel weight of the positioning branch network and the convolution kernel weight of the category branch network are fixed, and the network loss is used for network training of the bounding box size regression network, to obtain the convolution kernel weight of the bounding box size regression network. The convolution kernel weight of the positioning branch network, the convolution kernel weight of the category branch network and the convolution kernel weight of the bounding box size regression network are added to the initial target network, to obtain the target detection network.

7. The method of claim 1, wherein, The image of the detection target is input into the target detection network to obtain the detection result of the detection target, including: The image of the detection target is input into the target detection network to obtain a positioning heat map, category prediction information and bounding box prediction information; The positioning heat map is normalized to obtain a normalized positioning heat map; A maximum value filter is used to obtain the center point coordinates of the detection target from the normalized positioning heat map; The center point coordinates are used to obtain the confidence of each category at the center point coordinates from the category prediction information; The maximum value of the confidence of each category is taken as the confidence of the detection target, and the category corresponding to the confidence is the category of the detection target; The center point coordinates are used to obtain the width and height of the bounding box of the detection target from the bounding box prediction information.

8. A remote sensing target detection apparatus, characterized by comprising: The method comprises: A first obtaining module is configured to obtain sample prediction information of a sample based on a two-dimensional Gaussian distribution model of the sample by using an initial target detection network. A second obtaining module is configured to obtain a network loss of the target detection network by using the sample prediction information. A third obtaining module is configured to obtain a target detection network by using the network loss to perform network training on the target detection network. A fourth obtaining module is configured to input an image of a detection target into the target detection network to obtain a detection result of the detection target. The initial target detection network comprises a feature extraction network, a positioning branch network, a category branch network and a bounding box size regression network. The sample prediction information comprises a sample positioning heat map, sample category prediction information and sample bounding box prediction information. The two-dimensional Gaussian distribution model is used to perform feature extraction by using the feature extraction network, to obtain a shallow feature map of the sample. The shallow feature map is used to obtain a sample positioning heat map with decreasing receptive fields by using the positioning branch network. According to the shallow feature map and the sample positioning heat map, the category branch network is used to obtain sample category prediction information; According to the shallow feature map and the sample positioning heat map, the bounding box size regression network is used for feature fusion to obtain sample bounding box prediction information.

9. A computing device, comprising: Comprise: One or more processors; Memory for storing one or more computer programs, When the one or more computer programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Executable instructions are stored thereon, which are executed by the processor to enable the processor to implement the method of any one of claims 1 to 7.

Images