Target detection method and target detection device

Abstract

The invention discloses a target detection method and a target detection device, and relates to the field of image processing. The accuracy of target detection can be improved. The target detection method comprises the following steps: acquiring a to-be-detected remote sensing image, inputting the remote sensing image into a target detection network, and performing feature extraction on the remote sensing image by a feature pyramid to obtain first features of different scales; the candidate region extraction network extracts candidate features corresponding to candidate anchor frames from the first features of each scale; the classification branches extract feature amplitudes of the candidate features, determine dynamic margins corresponding to the feature amplitudes based on a dynamic angle margin penalty function, and output target categories of the candidate features in combination with the dynamic margins; the regression branch obtains the predicted offset of the candidate anchor frame through the full connection layer, adjusts the candidate anchor frame based on the predicted offset, and outputs an adjusted target detection frame; and determining a detection result of the remote sensing image based on the target category and the target detection frame corresponding to each scale.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a target detection method and a target detection device. BACKGROUND

[0002] Remote sensing target detection refers to identifying and locating specific targets such as ships from remote sensing images, and outputting target categories and spatial positions. Existing remote sensing target detection methods mainly fall into two categories: one is a rotated bounding box positioning method (such as DRBox, S2A-Net): by rotating the rectangular box to reduce the interference of background pixels, but the classification loss function is not optimized for feature space distribution. The other is a context enhancement method (such as FarSeg, SRAF-Net): using scene context relationship to distinguish targets from background, but the discrimination ability for blurred targets and shape-like interference is limited. SUMMARY

[0003] The present application provides a target detection method and a target detection device, which can simultaneously adjust and optimize target categories and target positions through a double-branch, thereby improving the accuracy of target detection.

[0004] In a first aspect, the present application provides a target detection method, comprising:

[0005] obtaining a remote sensing image to be detected, and inputting the remote sensing image into a target detection network, wherein the target detection network comprises a feature pyramid, a candidate region extraction network, and a target detection head;

[0006] the feature pyramid extracts features of the remote sensing image to obtain first features of different scales;

[0007] the candidate region extraction network extracts candidate features corresponding to candidate anchor boxes from the first features of each scale;

[0008] the target detection head comprises a classification branch and a regression branch, the classification branch extracts a feature amplitude of the candidate features, determines a dynamic margin corresponding to the feature amplitude based on a dynamic angle margin penalty function, and outputs a target category of the candidate features in combination with the dynamic margin;

[0009] the regression branch obtains a predicted offset of the candidate anchor boxes through a fully connected layer, adjusts the candidate anchor boxes based on the predicted offset, and outputs an adjusted target detection box;

[0010] based on the target category and the target detection box corresponding to each scale, a detection result of the remote sensing image is determined.

[0011] According to the target detection method provided in the embodiment, on one hand, the feature distribution of the candidate feature is optimized by using a dynamic angle margin penalty function, and the classification is adjusted in combination with the feature distribution, so that the target and the background or similar interference objects can be better distinguished, and the accuracy of the classification category is improved; on the other hand, the classification category and the detection frame position are simultaneously optimized and adjusted by using the double branches, so that the overall accuracy of the detection result is further improved.

[0012] In a second aspect, the present application provides a target detection device, comprising:

[0013] an image input module, configured to acquire a remote sensing image to be detected, and input the remote sensing image into a target detection network, wherein the target detection network comprises a feature pyramid, a candidate region extraction network and a target detection head;

[0014] The feature pyramid is configured to perform feature extraction on the remote sensing image to obtain first features of different scales;

[0015] The candidate region extraction network is configured to extract candidate features corresponding to candidate anchor frames from the first features of each scale;

[0016] The target detection head comprises a classification branch and a regression branch, the classification branch is configured to extract a feature amplitude of the candidate feature, and determine a dynamic margin corresponding to the feature amplitude based on a dynamic angle margin penalty function, and output a target category of the candidate feature in combination with the dynamic margin;

[0017] The regression branch is configured to obtain a predicted offset of the candidate anchor frame through a full connection layer, adjust the candidate anchor frame based on the predicted offset, and output an adjusted target detection frame;

[0018] a detection output module, configured to determine a detection result of the remote sensing image based on the target category and the target detection frame corresponding to each scale.

[0019] In a third aspect, the present application provides an electronic device, comprising a memory and one or more processors. Wherein the memory stores one or more computer programs, and the computer program comprises instructions, when the instructions are executed by the processor, the electronic device can execute the target detection method in the first aspect.

[0020] In a fourth aspect, the present application provides a computer readable storage medium, and the computer readable storage medium stores instructions, when the instructions are run on the electronic device, the electronic device executes the target detection method in the first aspect.

[0021] In a fifth aspect, the present application provides a computer program product, when the computer program product is run on the electronic device, the electronic device executes the target detection method in the first aspect.

[0022] It can be understood that the beneficial effects that can be achieved by the target detection device, the electronic device, the computer readable storage medium, and the computer program product provided above can refer to the beneficial effects in the first aspect, and will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flowchart of a target detection method provided by an embodiment of the present application is shown in the figure.

[0024] Figure 2 A structure diagram of a target detection device provided by an embodiment of the present application is shown in the figure.

[0025] Figure 3 A structure diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0026] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using the words “first”, “second”, etc. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit the sequence. Those skilled in the art can understand that the words “first”, “second”, etc. do not limit the quantity and execution sequence, and the words “first”, “second”, etc. also do not necessarily mean different. It should be noted that in the embodiments of the present application, the words “exemplary” or “for example” are used to represent an example, illustration or description. Any embodiment or design scheme described as “exemplary” or “for example” in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words “exemplary” or “for example” are used to present the relevant concepts in a specific manner. In the embodiments of the present application, “at least one” means one or more, and “multiple” means two or more.

[0027] It should be noted that “at the time of” in the embodiments of the present application can be at the moment when a certain condition occurs, or can be within a period of time after a certain condition occurs, which is not specifically limited in the embodiments of the present application.

[0028] The implementation of the embodiments will be described in detail below with reference to the accompanying drawings.

[0029] The embodiments provide a target detection method. Exemplarily, the target detection method can be applied to various electronic devices such as a computer (PC), a tablet computer, a virtual reality / augmented reality device, a wearable device, an industrial computer, and a car machine. The target detection method can also be applied to a server, a cloud, a server cluster, etc., which is not specially limited in the embodiments.

[0030] Figure 1A flowchart of a target detection method provided by an embodiment of the present application is shown.

[0031] As shown in the figure, the target detection method can include the following steps: Figure 1

[0032] Step 101: Obtain a remote sensing image to be detected, and input the remote sensing image into a target detection network, wherein the target detection network includes a feature pyramid, a candidate region extraction network, and a target detection head.

[0033] Step 102: The feature pyramid extracts features from the remote sensing image, and obtains first features of different scales.

[0034] Step 103: The candidate region extraction network extracts candidate features corresponding to candidate anchor boxes from the first features of each scale.

[0035] Step 104: The target detection head includes a classification branch and a regression branch. The classification branch extracts a feature amplitude of the candidate features, determines a dynamic margin corresponding to the feature amplitude based on a dynamic angle margin penalty function, and outputs a target category of the candidate features in combination with the dynamic margin.

[0036] Step 105: The regression branch obtains a predicted offset of the candidate anchor box through a fully connected layer, adjusts the candidate anchor box based on the predicted offset, and outputs an adjusted target detection box.

[0037] Step 106: Determine a detection result of the remote sensing image based on the target category and the target detection box corresponding to each scale.

[0038] In this embodiment, the target detection network is a deep learning model including a feature pyramid, a candidate region extraction network, and a target detection head. The feature pyramid (Feature Pyramid Network, FPN) refers to a network in a pyramid structure. The feature pyramid network in this embodiment can be ResNet50. The network can output feature maps at different levels of the remote sensing image. These feature maps correspond to different scales, i.e., resolutions. For example, four layers of features {P2, P3, P4, P5} are extracted. The resolutions of these four layers of feature maps decrease. The highest resolution feature map P2 is a shallow feature from the image, which contains more details. The lowest resolution feature map P5 is a deep feature from the image, which contains higher-level semantic information.

[0039] ​The candidate region extraction network can employ an Oriented Region Proposal Network (Oriented RPN), which has stronger detection capabilities for rotated targets. The candidate RPN can detect features at different scales, generating candidate anchor boxes that may contain foreground targets, and then extracting candidate features at the locations of these anchor boxes from the feature maps.

[0040] In this embodiment, the target detection network further includes a foreground target enhancement module. This module fuses the first features at each scale to obtain fused features; weights the fused features using a channel attention mechanism to obtain enhanced features; scales the enhanced features to obtain second features corresponding to each scale; merges the first and second features at each scale to obtain foreground enhanced multi-scale features; and inputs the foreground enhanced multi-scale features at each scale into a candidate region extraction network. The candidate region extraction network detects targets based on these foreground enhanced multi-scale features, determines candidate anchor boxes, and thus extracts candidate features including foreground targets.

[0041] The foreground target enhancement module first scales features from different scales to a uniform size; for example, it scales the four-layer features {P2, P3, P4, P5} to the size of P2. Then calculate the fusion features. F′2, F′3, F′4, and F′5 are the original feature layers (C2, C3, C4, C5) extracted from the backbone network, i.e., the first features, after scale adjustment. Features are then weighted by ECA channel attention. The enhanced features are obtained, then scaled to different scales and added to the first feature to obtain foreground enhancement multi-scale features for each scale. Previously, the multi-scale features {P2, P3, P4, P5} extracted by FPN were uniformly scaled to the resolution of the baseline layer P2. Multi-scale feature fusion was then performed at the P2 resolution to obtain the normalized feature F, and semantic enhancement features were generated based on F. (At this point, it's still P2 resolution), so here we will... Restore to the original resolution of each layer, i.e., the spatial dimensions of P2, P3, P4, and P5 respectively. Rescale back to the original resolution, add element-wise to {P2,P3,P4,P5}, and output the enhanced features {P′2,P′3,P′4,P′5}.

[0042] This embodiment further includes: generating a pseudo-mask of the remote sensing image based on the ground truth bounding boxes of the remote sensing image; segmenting the enhanced features to obtain a segmentation prediction mask; calculating a segmentation loss based on the segmentation prediction mask and the pseudo-mask; and optimizing the channel attention mechanism based on the segmentation loss.

[0043] The ground truth bounding boxes are manually annotated to indicate the location of the target to be detected in the remote sensing image. Pixels within the ground truth bounding boxes are set to 1, and pixels outside the boxes are set to 0, resulting in the pseudo-mask M. gt The pseudomask M gt This provides ground truth for semantic segmentation tasks, thereby supervising the network to distinguish foreground objects from background noise. The foreground object enhancement module in this embodiment can process the segmentation task by adding convolutional layers, segmenting the enhanced features, and generating a segmentation prediction mask, as shown below:

[0044]

[0045] The segmentation loss L is calculated using binary cross-entropy. seg =BCE(M pred M gt BCE represents binary cross-entropy. The segmentation loss enhances the output features of the channel attention mechanism. Learning the weight distribution of the foreground target and strengthening the foreground target can help improve detection accuracy.

[0046] Specifically, the candidate region extraction network transforms the foreground enhancement multi-scale features at each scale into classification feature maps through a 3×3 convolutional layer; generates preset anchor boxes in the classification feature maps at each scale according to the anchor box size corresponding to each scale; outputs the category score of each preset anchor box through a 1×1 convolutional layer, the category score being used to represent the probability that the target at the preset anchor box position is a foreground target or background; filters candidate anchor boxes for foreground targets from the preset anchor boxes based on the category scores, and extracts corresponding candidate features from the enhancement features based on the candidate anchor boxes.

[0047] Pre-defined rotating anchor boxes are used on each feature layer. The angles of the anchor boxes can be 0°, 30°, 60°, 90°, 120°, and 150°. The size of the anchor boxes in each feature map layer is related to the layer level; for example, the anchor boxes in layer P2 are smaller, and those in layer P5 are larger. A class score is predicted for each pre-defined anchor box. This score can include a binary classification score, specifically representing the probability that the anchor box is a target or background; it can also include the anchor box's positional offset (Δx, Δy, Δw, Δh, Δθ), where the parameters represent: the horizontal offset coefficient of the anchor box center point, the vertical offset coefficient of the anchor box center point, the logarithmic offset of the width scaling ratio, the logarithmic offset of the height scaling ratio, and the angle offset, respectively. The foreground enhancement multi-scale feature P... kA classification feature map is generated through a 3×3 convolutional layer. The classification feature map is then passed through a 1×1 convolutional layer to output a binary classification score for each preset anchor box. This binary classification score represents the two score values ​​s corresponding to each anchor box. 目标 and s 背景 This represents the probability that the anchor box is a foreground target and the probability that it is background. For example, [0.8, 0.2] indicates that the probability of a foreground target is 80%. The number of channels in this 1×1 convolutional layer is equal to twice the number of anchor boxes. Anchor boxes that may contain targets are initially filtered based on their classification scores, and the top N candidate anchor boxes with the highest scores are selected. Candidate anchor box set. N represents the number of candidate anchor boxes, and each candidate anchor box contains information such as coordinates, angle, and score. In this embodiment, ROIAlign can extract the corresponding features from the foreground enhancement multi-scale features based on the coordinates of each candidate anchor box, and use them as candidate features. The size of the candidate features is fixed, for example, 7×7×C, where C is the number of channels.

[0048] After obtaining candidate anchor boxes and candidate features, the object detection network in this embodiment begins to execute the two-branch task of the object detection head, which includes a classification branch and a regression branch. The operations performed by the classification branch specifically include: converting the candidate features into feature vectors through a fully connected layer; normalizing the feature vectors and the weight parameters of the classification branch; calculating the cosine similarity between the normalized feature vectors and the normalized weight parameters; and determining the target category of the candidate features based on the cosine similarity and dynamic margin.

[0049] Candidate features are compressed into a classification feature vector x by a fully connected layer. i Then, feature vector normalization is performed: Classification weight normalization: W j The weights for category j are calculated using cosine similarity. With all The cosine of the included angle:

[0050]

[0051] The feature magnitude of a candidate feature is represented as: l = ||x i ‖2 refers to the feature vector x i The L2 norm and feature magnitude can characterize sample quality, such as clarity and significance.

[0052] The Dynamic Angular Margin Penalty Function is as follows:

[0053] Δ(l)=a·(ul)+b

[0054] Where l is the feature amplitude of the candidate feature, Δ(l) is the dynamic margin; a>0, u=max(l), decreasing with l.

[0055] Then, the probability distribution of the target category is determined based on the dynamic margin, expressed as:

[0056]

[0057] The function is designed as a strictly decreasing convex function with respect to l, which causes high-quality samples (samples with large l) to be subject to a larger angle penalty to enhance intra-class compactness, and causes low-quality samples (samples with small l) to be subject to a smaller penalty to avoid over-suppression.

[0058] The above formula is used to determine the true category y. i Apply dynamic margins to amplify the confidence level using a scaling factor s. Wherein, Representing candidate feature x i Its true category y i The included angle between them, where i is the feature index, and Δ(l) represents the dynamic margin calculated based on the feature magnitude l. Indicates the angle The cosine value after superimposing the dynamic margin Δ(l); the denominator θ represents the sum of the exponential terms of all non-real categories, where j is the feature index of the non-real category, and θ j Indicates feature x i The angle between the error category j and the error type j, where s is the scaling factor.

[0059] The calculated P cls (y i The probability that the joystick image to be detected belongs to the foreground category is denoted as . For example, in a vehicle detection task, this mainly includes:

[0060] Vehicles (positive category): The category of foreground targets to be detected (such as cars, trucks, etc.).

[0061] Background (negative class): Non-target areas (such as buildings, vegetation, shadows, and other distractions).

[0062] The loss function for this classification branch uses the probability distribution P. cls (y i The negative log-likelihood (NLL) of ) is expressed as:

[0063] L cls =-log(P cls (y i ))

[0064] The first classification loss L of the target category is calculated using this loss function. cls .

[0065] The input to the regression branch is shared with the classification branch. The regression branch calculates the second classification loss using the smooth L1 loss function. The second classification loss is used to optimize the selection box parameter offset, which together with the first classification loss constitutes the complete target of the detection head optimization.

[0066] The regression branch processing includes: predicting the geometric offset ΔB = (Δx, Δy, Δw, Δh, Δθ) through a fully connected layer. Based on the predicted offset ΔB... pred With the actual offset ΔB gt Calculate the Smooth L1 loss based on the difference:

[0067]

[0068] The true offset is the difference between the candidate anchor box and the true labeled box. Based on this second classification loss, the rotated detection box can be finely adjusted, outputting the adjusted target detection box B. det .

[0069] The target category P output by the classification branch cls Fine detection box B of the regression branch output det Candidate boxes are associated by index, and boxes with low confidence are removed. The best boxes are then sorted in descending order of confidence and retained as the final detection result. The detection result can include the detection box, the category label of the target category, and the confidence score. The detection box is specifically represented by its center coordinates, width, height, and rotation angle.

[0070] In this embodiment, the object detection network also needs to be trained. Specifically, the first classification loss of the object category is calculated using a classification loss function; the second classification loss of the predicted offset and the true offset between the object detection box and the ground truth box is calculated using a regression loss function; and the object detection network is optimized by combining the first classification loss, the second classification loss, and the segmentation loss.

[0071] It also includes: calculating the third classification loss of the category score of the preset anchor box through cross-entropy loss; determining the positional offset loss between the preset anchor box and the ground truth bounding box of the remote sensing image; and optimizing the preset anchor box by combining the third classification loss and the positional offset loss.

[0072] The overall loss function is expressed as follows:

[0073] L total =L RPN +L cls +L reg+λL seg (λ=1)

[0074] Among them, L cls For the first category loss, L reg For the second category loss, L seg For segmentation loss;

[0075] L RPN =L rpn_cls +L rpn_reg

[0076] L rpn_cls The cross-entropy loss is used to calculate the loss for the third classification. rpn_reg The position offset loss is calculated using Smooth L1 loss. The preset anchor boxes are optimized using third-class classification loss and position offset loss to ensure high recall and accurate positioning of the generated preset anchor boxes.

[0077] The overall loss of the object detection model is calculated using the total loss function, and backpropagation of the loss is performed. A gradient step-descent strategy is adopted to balance convergence speed and accuracy, avoiding the network loss optimization from getting trapped in local optima. In this implementation, backpropagation using the segmentation loss from mask prediction can supervise the weights of regions that enhance foreground targets in the feature pyramid, suppressing background noise and thus improving the model's robustness against interference. The second loss function of the regression branch optimizes the localization of the detection boxes, improving the fineness of the localization. Through classification loss, the intra-class penalty is increased for high-quality samples, causing features to move closer to the cluster center, while the penalty is decreased for low-quality samples, improving robustness to blurred targets, achieving adaptability to sample quality, and enhancing the stability of the model.

[0078] Furthermore, this embodiment also provides a target detection device that can be used to perform the above-described target detection method.

[0079] like Figure 2As shown, the target detection device 200 may include: an image input module 201, used to acquire a remote sensing image to be detected, and input the remote sensing image into a target detection network, the target detection network including a feature pyramid, a candidate region extraction network, and a target detection head; the feature pyramid extracts features from the remote sensing image to obtain first features at different scales; the candidate region extraction network extracts candidate features corresponding to candidate anchor boxes from the first features at each scale; the target detection head includes a classification branch and a regression branch, the classification branch extracts the feature amplitude of the candidate features, and determines the dynamic margin corresponding to the feature amplitude based on a dynamic angle margin penalty function, and outputs the target category of the candidate features in combination with the dynamic margin; the regression branch obtains the predicted offset of the candidate anchor box through a fully connected layer, and adjusts the candidate anchor box based on the predicted offset, and outputs the adjusted target detection box; and a detection output module 202, used to determine the detection result of the remote sensing image based on the target category corresponding to each scale and the target detection box.

[0080] In one embodiment, the target detection network further includes a foreground target enhancement module, which fuses the first features at each scale to obtain fused features; weights the fused features using a channel attention mechanism to obtain enhanced features; scales the enhanced features to obtain a second feature corresponding to each scale; merges the first features and the second features at each scale to obtain foreground enhancement multi-scale features; and inputs the foreground enhancement multi-scale features at each scale into a candidate region extraction network.

[0081] In one embodiment, the candidate region extraction network transforms the foreground enhancement multi-scale features at each scale into classification feature maps using 3×3 convolutional layers; generates preset anchor boxes in the classification feature maps at each scale according to the anchor box size corresponding to each scale; outputs the category score of each preset anchor box through 1×1 convolutional layers, the category score representing the probability that the target at the preset anchor box position is a foreground target or background; filters candidate anchor boxes for foreground targets from the preset anchor boxes based on the classification scores, and extracts corresponding candidate features from the foreground enhancement multi-scale features based on the candidate anchor boxes.

[0082] In one implementation, the classification branch transforms the candidate features into feature vectors through a fully connected layer; the feature vectors are normalized, and the weight parameters of the classification branch are also normalized; the cosine similarity between the normalized feature vectors and the normalized weight parameters is calculated; and the target category of the candidate features is determined based on the cosine similarity and dynamic margin.

[0083] In one embodiment, the target detection device further includes a first supervision module, configured to generate a pseudo mask of the remote sensing image based on the ground truth bounding boxes of the remote sensing image; segment the enhanced features to obtain a segmentation prediction mask; calculate a segmentation loss based on the segmentation prediction mask and the pseudo mask; and optimize the channel attention mechanism based on the segmentation loss.

[0084] In one embodiment, the target detection device 200 further includes a second supervision module, configured to calculate a first classification loss for the target category using a classification loss function; calculate a second classification loss for the predicted offset and the true offset between the target detection box and the ground truth box using a regression loss function; and optimize the target detection network by combining the first classification loss, the second classification loss, and the segmentation loss.

[0085] In one embodiment, the target detection device 200 further includes a third supervision module, which is used to calculate a third classification loss of the category score of the preset anchor box through cross-entropy loss; determine the position offset loss between the preset anchor box and the ground truth bounding box of the remote sensing image; and optimize the preset anchor box by combining the third classification loss and the position offset loss.

[0086] In one embodiment, the dynamic angular margin penalty function is:

[0087] Δ(l)=a·(ul)+b

[0088] Where a>0, u=max(l), and l is the feature amplitude of the candidate feature.

[0089] In one implementation, the target category represents the probability distribution of the candidate feature being a foreground target, expressed as:

[0090]

[0091] in, Representing candidate feature x i Its true category y i The included angle between them, where i is the feature index, and Δ(l) represents the dynamic margin calculated based on the feature magnitude l. Indicates the angle The cosine value after superimposing the dynamic margin Δ(l); the denominator θ represents the sum of the exponential terms of all non-real categories, where j is the feature index of the non-real category, and θ j Indicates feature x i The angle between the error category j and the error type j, where s is the scaling factor.

[0092] The specific details of each module or unit in the above-mentioned target detection device have been described in detail in the corresponding target detection method, so they will not be repeated here.

[0093] This application also provides an electronic device. Figure 3 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 3 The electronic device 600 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0094] like Figure 3 As shown, the electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0095] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0096] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined in the embodiments of this application.

[0097] For example, when the computer program is executed by the central processing unit (CPU) 601, it can perform the following: acquire a remote sensing image to be detected; input the remote sensing image into a target detection network, the target detection network including a feature pyramid, a candidate region extraction network, and a target detection head; the feature pyramid extracts features from the remote sensing image to obtain first features at different scales; the candidate region extraction network extracts candidate features corresponding to preset anchor boxes from the first features at each scale; the target detection head includes a classification branch and a regression branch; the classification branch extracts the feature amplitude of the candidate features and determines the dynamic margin corresponding to the feature amplitude based on a dynamic angle margin penalty function, and outputs the target category of the candidate features in combination with the dynamic margin; the regression branch obtains the predicted offset of the candidate anchor boxes through a fully connected layer, adjusts the candidate anchor boxes based on the predicted offset, and outputs the adjusted target detection boxes; and determines the detection result of the remote sensing image based on the target category corresponding to each scale and the target detection boxes.

[0098] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0100] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0101] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which include instructions that, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0102] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A target detection method, characterized in that, include: A remote sensing image to be detected is acquired, and the remote sensing image is input into a target detection network, which includes a feature pyramid, a candidate region extraction network, and a target detection head. The feature pyramid extracts features from the remote sensing image to obtain first features at different scales; The candidate region extraction network extracts candidate features corresponding to candidate anchor boxes from the first feature at each scale. The target detection head includes a classification branch and a regression branch. The classification branch extracts the feature amplitude of the candidate features and determines the dynamic margin corresponding to the feature amplitude based on the dynamic angle margin penalty function. The target category of the candidate features is output in combination with the dynamic margin. The regression branch obtains the predicted offset of the candidate anchor box through the fully connected layer, and adjusts the candidate anchor box based on the predicted offset, outputting the adjusted target detection box. The detection result of the remote sensing image is determined based on the target category and target detection box corresponding to each scale.

2. The target detection method according to claim 1, characterized in that, The target detection network further includes a foreground target enhancement module, and the method further includes: The foreground target enhancement module fuses the first features at each scale to obtain fused features; and The fused features are weighted using a channel attention mechanism to obtain enhanced features; The enhanced features are scaled to obtain the second feature corresponding to each scale; The first feature and the second feature at each scale are merged to obtain foreground enhancement multi-scale features; The foreground enhancement multi-scale features at each scale are input into the candidate region extraction network.

3. The target detection method according to claim 2, characterized in that, The candidate region extraction network extracts candidate features corresponding to candidate anchor boxes from the first feature at each scale, including: The candidate region extraction network transforms the foreground enhancement multi-scale features at each scale into classification feature maps through 3×3 convolutional layers; Preset anchor boxes are generated in the classification feature map at each scale according to the anchor box size corresponding to each scale. Each preset anchor box is output through a 1×1 convolutional layer, and the category score is used to represent the probability that the target at the preset anchor box position is a foreground target or a background target. Based on the category score, candidate anchor boxes for the foreground target are selected from the preset anchor boxes, and corresponding candidate features are extracted from the foreground enhancement multi-scale features based on the candidate anchor boxes.

4. The target detection method according to claim 1, characterized in that, The target category that combines the dynamic margin output candidate features includes: The classification branch transforms the candidate features into feature vectors through a fully connected layer; The feature vector is normalized, and the weight parameters of the classification branch are normalized. The cosine similarity between the normalized feature vector and the normalized weight parameters is calculated. The target category of the candidate features is determined based on the cosine similarity and dynamic margin.

5. The target detection method according to claim 2, characterized in that, Also includes: Based on the actual bounding boxes of the remote sensing image, generate a pseudo mask for the remote sensing image; The enhanced features are segmented to obtain a segmentation prediction mask; The segmentation loss is calculated based on the segmentation prediction mask and the pseudo-mask, and the channel attention mechanism is optimized based on the segmentation loss.

6. The target detection method according to claim 5, characterized in that, Also includes: The first classification loss for the target category is calculated using a classification loss function; The second classification loss is calculated by using a regression loss function to determine the true offset between the predicted offset and the target detection box and the ground truth labeled box. The target detection network is optimized by combining the first classification loss, the second classification loss, and the segmentation loss.

7. The target detection method according to claim 3, characterized in that, Also includes: The third classification loss of the category score of the preset anchor box is calculated by cross-entropy loss; Determine the positional offset loss between the preset anchor frame and the actual annotation frame in the remote sensing image; The preset anchor frame is optimized by combining the third classification loss and the position offset loss.

8. The target detection method according to claim 1, characterized in that, The dynamic angle margin penalty function is: Δ(l)=a·(ul)+b Where a>0, u=max(l), and l is the feature amplitude of the candidate feature.

9. The target detection method according to claim 1, characterized in that, The target category represents the probability distribution of the candidate feature being a foreground target, expressed as: in, Representing candidate feature x i Its true category y i The included angle between them, where i is the feature index, and Δ(l) represents the dynamic margin calculated based on the feature magnitude l. Indicates the angle The cosine value after superimposing the dynamic margin Δ(l); the denominator θ represents the sum of the exponential terms of all non-real categories, where j is the feature index of the non-real category, and θ j Indicates feature x i The angle between the error category j and the error type j, where s is the scaling factor.

10. A target detection device, characterized in that, include: An image input module is used to acquire a remote sensing image to be detected and input the remote sensing image into a target detection network, the target detection network including a feature pyramid, a candidate region extraction network and a target detection head; The feature pyramid extracts features from the remote sensing image to obtain first features at different scales; The candidate region extraction network extracts candidate features corresponding to candidate anchor boxes from the first feature at each scale. The target detection head includes a classification branch and a regression branch. The classification branch extracts the feature amplitude of the candidate features and determines the dynamic margin corresponding to the feature amplitude based on the dynamic angle margin penalty function. The target category of the candidate features is output in combination with the dynamic margin. The regression branch obtains the predicted offset of the candidate anchor box through the fully connected layer, and adjusts the candidate anchor box based on the predicted offset, outputting the adjusted target detection box. The detection output module is used to determine the detection result of the remote sensing image based on the target category and target detection box corresponding to each scale.

Images