Target detection method and device, storage medium and electronic device

Abstract

The application discloses a target detection method and device, a storage medium and an electronic device, and the method comprises the following steps: processing a target image through an equal-variant main trunk network to obtain a rotation equal-variant feature of the target image; generating a rotation robust prototype through the rotation equal-variant feature, and performing enhancement processing on the rotation equal-variant feature according to the rotation robust prototype to obtain an enhanced rotation equal-variant feature, wherein the enhancement processing comprises stable enhancement processing and semantic enhancement processing; and performing target detection on the target image according to the enhanced rotation equal-variant feature. By using the technical scheme, the problem that the rotation equal-variant feature of the image determined by directly using the equal-variant main trunk network to detect the target of the image leads to poor target detection effect is solved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more specifically, to a target detection method and apparatus, a storage medium and an electronic device. Background Technology

[0002] As one of the fundamental tasks of computer vision, visible light remote sensing target detection is a technique that uses visible light remote sensing data (such as satellite imagery and aerial imagery) to identify and locate specific targets or objects on the Earth's surface. These targets may include buildings, vehicles, roads, farmland, and water bodies. Visible light remote sensing target detection has wide applications in many fields, such as urban planning, environmental monitoring, disaster management, agriculture, and national defense.

[0003] Remote sensing target detection typically uses a rotated bounding box to enclose the target, which involves predicting the target's orientation. Traditional detection algorithms are based on Convolutional Neural Networks (CNNs). However, CNNs lack robustness to rotational variations, limiting their ability to encode object orientation. Recent methods have used equivariant CNNs to encode object orientation features, but as network depth increases, the computational instability leads to unstable and noisy orientation encodings in these networks, impacting detection performance.

[0004] In the field of visible light remote sensing, traditional detection methods are essentially the same as those used for traditional target detection in natural images. Early studies first used candidate region extraction algorithms to extract high-confidence image sub-regions, then applied hand-designed algorithms to extract features from the corresponding regions, and finally fed the obtained features into the detector for final classification and regression.

[0005] In recent years, with the development of machine learning, especially deep learning, object detection performance has significantly improved. Deep detectors can be divided into two categories: two-stage and single-stage. Two-stage detectors first generate a set of regions of interest (RoIs) in the first stage, and then perform regression of the bounding boxes of the RoIs and object classification in the second stage. Single-stage detectors, such as YOLO and SSD, directly perform position regression and classification of the object, without the need for RoI extraction. Single-stage detectors often lag behind two-stage detectors due to the extreme foreground-background class imbalance problem. Focal Loss is used to address this issue. Some algorithms are based on the idea of ​​anchor-free detection or directly describe object detection as the detection problem of key points of an object to avoid complex calculations related to anchor boxes, and this approach usually makes the algorithm run faster.

[0006] Objects in remote sensing images often appear in arbitrary directions, and horizontal bounding boxes are commonly used in general object detection methods to locate targets. However, this often results in severe misalignment: a single horizontal anchor box may contain multiple target instances. Some methods use rotated anchor boxes with different angles, scales, and aspect ratios to alleviate this problem, but this also involves a large amount of computation related to the anchor boxes (e.g., bounding box changes, matching with ground truth values, etc.). A large number of pre-defined anchor boxes can easily lead to memory redundancy. Some researchers have proposed the RoI Transformer structure to transform horizontal RoIs into rotated RoIs. This structure reduces the number of pre-defined anchor boxes and alleviates the mismatch problem. However, this structure requires predicting additional parameters for converting horizontal RoIs to rotated RoIs, introducing additional RoI computation operations and adding computational complexity. Others have proposed using horizontal sliding vertices to represent rotated boxes, which can accurately locate rotated targets. However, the RoI features under this representation are still horizontal, and the mismatch problem persists. The recently proposed R3Det samples and sums features from five positions (e.g., center point and corner points) of the corresponding anchor box to re-encode positional information.

[0007] The ReDet network is a two-stage network, an improvement on the RoI Transformer. It rewrites ResNet50 using the E2CNN library, resulting in the equivariant ResNet50 (Re-ResNet50). The equivariant backbone extracts rotationally equivariant features. In the second stage, channel alignment is used to rearrange the channels to obtain rotationally invariant features, which are then used for final localization and classification. However, in ReDet, Re-ResNet50 cannot extract sufficiently accurate rotationally equivariant features, and due to its network structure, it introduces noise into some channels, severely affecting the detector's ability to recognize targets. For example... Figure 1 As shown, (a) is the result of traditional CNN detection, and (b) is the result of detection using Re-ResNet50. Some channels have huge background noise or suppress the target feature response.

[0008] There is currently no effective solution to the problem that directly using the rotational equivariance features of the image determined by the equivariant backbone network to perform target detection in related technologies results in poor target detection performance. Summary of the Invention

[0009] This invention provides a target detection method, apparatus, storage medium, and electronic device to at least solve the problem that the target detection effect is poor when the rotational equivariance features of the image determined by the equivariant backbone network are directly used to perform target detection on the image.

[0010] According to an embodiment of the present invention, a target detection method is provided, comprising: processing a target image through an equivariant backbone network to obtain rotational equivariant features of the target image; generating a rotationally robust prototype through the rotationally robust features, and enhancing the rotationally equivariant features according to the rotationally robust prototype to obtain enhanced rotationally equivariant features, wherein the enhancement processing includes stabilization enhancement processing and semantic enhancement processing; and performing target detection on the target image according to the enhanced rotationally equivariant features.

[0011] In an exemplary embodiment, generating a rotationally robust prototype using the rotationally equivariant features includes: dividing the rotationally equivariant features according to the number of channels of the rotationally equivariant features of the target image to obtain K sub-rotationally equivariant features, wherein K is a positive integer and K is divisible by the number of channels of the rotationally equivariant features of the target image; and performing an aggregation operation on the K sub-rotationally equivariant features to obtain the rotationally robust prototype.

[0012] In an exemplary embodiment, the K sub-rotational isomorphic features are aggregated to obtain the rotationally robust prototype, which includes: adding the K sub-rotational isomorphic features to obtain the rotationally robust prototype.

[0013] In an exemplary embodiment, enhancing the rotational isomorphic features according to the rotational robust prototype to obtain enhanced rotational isomorphic features includes: enhancing each of the K sub-rotational isomorphic features according to the rotational robust prototype to obtain K enhanced sub-rotational isomorphic features; and concatenating the K enhanced sub-rotational isomorphic features to obtain the enhanced rotational isomorphic features.

[0014] In an exemplary embodiment, each of the K sub-rotational isomorphic features is enhanced according to the rotational robust prototype, including: enhancing the i-th sub-rotational isomorphic feature among the K sub-rotational isomorphic features in the following manner to enhance each sub-rotational isomorphic feature, where i is a positive integer greater than or equal to 1 and less than or equal to K: convolving the rotational robust prototype and concatenating the convolved rotational robust prototype with the i-th sub-rotational isomorphic feature to obtain a first feature; convolving the first feature and adding the convolved first feature with the i-th sub-rotational isomorphic feature to obtain a second feature; and processing the second feature through an activation function to obtain the i-th enhanced sub-rotational isomorphic feature.

[0015] In an exemplary embodiment, target detection of the target image based on the enhanced rotational isotropic features includes: when using a one-stage detector to perform target detection on the target image, directly performing object classification and position regression based on the enhanced rotational isotropic features to obtain the target detection result.

[0016] In an exemplary embodiment, target detection of the target image based on the enhanced rotational isotropic features includes: when using a two-stage detector to detect the target image, generating a set of regions of interest using a region generation network with the enhanced rotational isotropic features; performing object classification and position regression on the set of regions of interest based on the rotational isotropic features to obtain a target detection result; or performing object classification and position regression on the set of regions of interest based on the enhanced rotational isotropic features to obtain a target detection result.

[0017] According to another embodiment of the present invention, a target detection device is also provided, comprising: a first processing module, configured to process a target image through an equivariant backbone network to obtain rotational equivariant features of the target image; a second processing module, configured to generate a rotationally robust prototype through the rotationally robust features, and enhance the rotationally equivariant features according to the rotationally robust prototype to obtain enhanced rotationally equivariant features, wherein the enhancement processing includes stability enhancement processing and semantic enhancement processing; and a detection module, configured to perform target detection on the target image according to the enhanced rotationally equivariant features.

[0018] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0019] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0020] This invention generates a rotationally robust prototype from the rotational equivariance features of a target image determined by an equivariant backbone network. The rotational equivariance features are then enhanced using this prototype (including stability enhancement and semantic enhancement). Finally, the enhanced rotational equivariance features are used to perform target detection on the target image. By improving the stability of the rotational equivariance features and adding semantic enhancements, this invention solves the problem of poor target detection performance caused by directly using the rotational equivariance features determined by the equivariant backbone network for target detection, thus improving the overall target image detection performance. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the effect of existing technologies for object detection in images;

[0022] Figure 2 This is a block diagram of the mobile terminal hardware structure of the target detection method according to an embodiment of the present invention;

[0023] Figure 3 This is a flowchart (I) of a target detection method according to an embodiment of the present invention;

[0024] Figure 4 This is a flowchart (II) of the target detection method according to an embodiment of the present invention;

[0025] Figure 5 This is an overall schematic diagram of the target detection method according to an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram illustrating the effect of target detection in an image according to an embodiment of the present invention;

[0027] Figure 7 This is a structural block diagram of a target detection device according to an embodiment of the present invention. Detailed Implementation

[0028] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0030] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 2 This is a block diagram of the mobile terminal hardware structure of the target detection method according to an embodiment of the present invention. Figure 2 As shown, a mobile terminal may include one or more ( Figure 2 Only one is shown in the diagram. A processor 202 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 204 for storing data are also shown. The mobile terminal may further include a transmission device 206 for communication functions and an input / output device 208. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 2 The more or fewer components shown, or having the same Figure 2The different configurations shown.

[0031] The memory 204 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the target detection method in this embodiment of the invention. The processor 202 executes various functional applications and data processing by running the computer program stored in the memory 204, thereby implementing the aforementioned method. The memory 204 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 204 may further include memory remotely located relative to the processor 202, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0032] The transmission device 206 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 206 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 206 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0033] This embodiment provides a target detection method. Figure 3 This is a flowchart (I) of a target detection method according to an embodiment of the present invention, as follows: Figure 3 As shown, the process includes the following steps S302-S306:

[0034] Step S302: Process the target image using an equivariant backbone network to obtain the rotational equivariant features of the target image;

[0035] As an alternative example, the equivariant backbone network is ReResNet50-ReFPN.

[0036] Step S304: Generate a rotationally robust prototype using the rotationally robust features, and enhance the rotationally robust features based on the rotationally robust prototype to obtain enhanced rotationally robust features. The enhancement process includes stability enhancement and semantic enhancement.

[0037] In an exemplary embodiment, the generation of a rotationally robust prototype through the rotationally equivariant features described above can be achieved through the following steps S11-S12:

[0038] Step S11: Divide the rotational isomorphic features of the target image into K sub-rotational isomorphic features according to the number of channels of the rotational isomorphic features of the target image, wherein K is a positive integer and K is divisible by the number of channels of the rotational isomorphic features of the target image.

[0039] It should be noted that the rotational equivariance features of the target image include 256 channels; for example, K equals 8.

[0040] Step S12: Perform an aggregation operation on the K sub-rotational isomorphic features to obtain the rotationally robust prototype.

[0041] In other words, the process of generating a rotationally robust prototype can be briefly described as follows:

[0042] x pro =φ(ψ(x,K));

[0043] Here, ψ(x,K) represents dividing x into K parts according to channels. φ represents the aggregation operation, which aggregates the results of ψ to finally obtain the rotationally robust prototype x. pro , where x is a rotationally isotropic feature.

[0044] It should be noted that the aggregation operation can be diverse. In an exemplary embodiment, a basic addition method can be used, that is, step S12 above includes: adding the K sub-rotational isomorphic features to obtain the rotational robust prototype.

[0045] In other words, Where, x k Let ψ represent the k-th feature generated by ψ.

[0046] To better understand, such as Figure 4 As shown, channel group features are considered in one dimension. Each channel group feature has a high response to the position of the object of interest and also contains random outliers. By involving rotation robust prototypes, misactivated regions (i.e. regions that do not belong to the original object but are misactivated when the orientation changes) can be suppressed, thus making the model more stable to rotation.

[0047] In an exemplary embodiment, the above-described enhancement process of the rotationally isotropic feature based on the rotationally robust prototype to obtain the enhanced rotationally isotropic feature includes the following steps S21-S22:

[0048] Step S21: Enhance each of the K sub-rotational isomorphic features according to the rotational robust prototype to obtain K enhanced sub-rotational isomorphic features;

[0049] It should be noted that the rotationally robust prototype contains stable cyclic displacements, preserving significant semantic information in each channel group. Therefore, the prototype features can be used to enhance the rotationally equivariant features in each channel group.

[0050] First, a convolutional layer can be applied to the rotation-robust prototype to reduce the numerical gap between it and the slice features (i.e., the i-th sub-rotational isovariant feature). Then, convolution is used to extract complementary semantic features by concatenating the prototype with each slice feature individually. Finally, the acquired semantic features are added to the slice features to achieve enhancement.

[0051] In other words, in an exemplary embodiment, step S21 above includes: enhancing the i-th sub-rotational isomorphic feature among the K sub-rotational isomorphic features through the following steps S211-S213, so as to enhance each sub-rotational isomorphic feature, wherein i is a positive integer greater than or equal to 1 and less than or equal to K:

[0052] Step S211: Convolve the rotation robust prototype and concatenate the convolved rotation robust prototype with the i-th sub-rotation equivariant feature to obtain the first feature;

[0053] Step S212: Convolve the first feature, and add the convolved first feature to the i-th sub-rotational equivariant feature to obtain the second feature;

[0054] Step S213: Process the second feature using an activation function to obtain the i-th enhanced sub-rotational equivariant feature.

[0055] To better understand steps S211-S213 above, the following explanation is provided in conjunction with the formulas:

[0056] y k =prelu(conv(cat(x) k ,conv(x pro )))+x k );

[0057] Where, x k That is, the i-th sub-rotational isomorphic feature in the above steps, y k This refers to the i-th enhanced sub-rotational equivariant feature. "cat" indicates concatenating the two features on the channel, "conv" indicates normal convolution, "+" indicates pixel-wise addition, and "prelu" indicates activation function.

[0058] It should be noted that by learning a stable cyclic shift from the prototype, each x k It can achieve better directional semantics. Furthermore, because the prototype possesses "standard" semantics for all group channels, xk It can learn to implicitly activate the semantic relationships between group channels within itself.

[0059] Step S22: The K enhanced sub-rotational isomorphic features are spliced ​​together to obtain the enhanced rotational isomorphic features.

[0060] It should be noted that in the above steps S21-S22, each of the K sub-rotational isovariant features is enhanced by the rotational robust prototype, which makes the rotational isovariant features more stable and the semantic enhancement effect better.

[0061] Step S306: Target detection is performed on the target image based on the enhanced rotational equivariance features.

[0062] In an exemplary embodiment, step S306 includes: when using a one-stage detector to perform target detection on the target image, directly performing object classification and position regression based on the enhanced rotational isotropic features to obtain the target detection result.

[0063] In an exemplary embodiment, step S306 further includes: when using a two-stage detector to detect targets in the target image, generating a set of regions of interest using the enhanced rotational isotropic features through a region generation network; performing object classification and position regression on the set of regions of interest based on the rotational isotropic features to obtain target detection results; or performing object classification and position regression on the set of regions of interest based on the enhanced rotational isotropic features to obtain target detection results.

[0064] In an exemplary embodiment, target detection of the target image based on the enhanced rotational isomorphic features includes: when the target image is a visible light satellite image or a visible light aerial image, detecting objects (e.g., houses, vehicles, etc.) in the target image based on the enhanced rotational isomorphic features.

[0065] Steps S302-S306 above generate a rotationally robust prototype from the rotational equivariance features of the target image determined by the equivariant backbone network. The rotational equivariance features are then enhanced based on the robust prototype (both stabilization and semantic enhancement). Finally, target detection is performed on the target image based on the enhanced rotational equivariance features. By improving the stability of the rotational equivariance features and adding semantic enhancements, the problem of poor target detection performance caused by directly using the rotational equivariance features determined by the equivariant backbone network for target detection is solved, thus improving the overall target image detection performance.

[0066] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. The present invention will now be specifically described in conjunction with the embodiments.

[0067] In one example, to obtain stable directional encoding, this invention proposes a Rotationally Robust Prototype Generation (RPG) method, comprising a stabilization module and an enhancement module. In the stabilization module, a rotationally robust prototype is generated to increase the stability of cyclic displacements. In the enhancement module, the acquired rotationally robust prototype is used to improve the feature's response to the target semantics; that is, the channel group learns residuals from the rotationally robust prototype to enhance the semantic information of the features. The RPG method can be used as a plug-and-play module in both one-stage and two-stage detectors.

[0068] The RPG method is described in detail below. Part A provides an overview of the stabilization module in the RPG method. Descriptions of the enhancement modules in the RPG method are provided in Part B. Then, Part C discusses the application of this method in single-stage and two-stage networks.

[0069] A. Channel displacement stabilization

[0070] The process of generating a rotationally robust prototype can be briefly described as follows:

[0071] x pro =φ(ψ(x,K));

[0072] Here, ψ(x,K) represents dividing x into K parts according to channels. φ represents the aggregation operation, which aggregates the results of ψ to finally obtain the rotationally robust prototype x. pro , where x is a rotationally isomorphic feature.

[0073] It should be noted that aggregation operations can be varied. A basic addition method can be used, which is simple and efficient, and φ can then be expressed as:

[0074]

[0075] Where, x k Let ψ represent the k-th feature generated by ψ.

[0076] like Figure 4 As shown on the right, channel group features are considered in one dimension. Each channel group feature has a high response to the location of the object of interest and also contains random outliers. The involvement of rotation-robust prototypes can suppress misactivated regions (i.e., regions that do not belong to the original object but are misactivated when the orientation changes), thus making the model more stable to rotation.

[0077] B. Semantic Enhancement

[0078] The rotationally robust prototype contains stable cyclic displacements and preserves significant semantic information in each channel group. Prototype features can be used to enhance rotationally equivariant features in each channel group.

[0079] First, a convolutional layer is applied to the prototype to reduce the numerical difference between it and the slice features. Then, convolution is used to extract complementary semantic features by concatenating the prototype with each slice feature. Finally, the obtained semantic features are added to the slice features to achieve enhancement. This process can be represented as:

[0080] y k =prelu(conv(cat(x) k ,conv(x pro )))+x k );

[0081] Here, `cat` concatenates two features along a channel, `conv` represents normal convolution, `+` indicates pixel-wise addition, `prelu` represents the activation function, and finally, all y... k The final y is generated by splicing the components.

[0082] By learning stable cyclic shifts from the prototype, each x k It can achieve better directional semantics. Furthermore, because the prototype has 'standard' semantics for all group channels, x k It can learn to implicitly activate the semantic relationships between group channels within itself.

[0083] C. Insertion detector

[0084] Because the input x and output y have the same dimension, this ensures that RPG can be used in different detector frameworks, such as... Figure 5 As shown, the RPG module was incorporated into both the one-stage and two-stage detectors.

[0085] First, S2ANet is used as the baseline for the first-stage detector. The original ResNet50-FPN backbone is replaced by an isovariant backbone, ReResNet50-ReFPN, to extract rotationally isovariant features; then, an RPG module is embedded after ReFPN. For S2ANet, anchor boxes are generated by ARN, and feature alignment is performed based on the anchor boxes through alignment convolutions. ARF is then used to encode orientation information for localization.

[0086] In the two-stage detector, Oriented R-CNN was used as the baseline. Similar to the one-stage detector described above, the original ResNet50-FPN backbone was replaced with ReResNet50-ReFPN to extract rotationally equivalent features. The RPG module was embedded into the Region Proposal Network (RPN) stage to generate RoIs using midpoint offset representation; then, it was fine-tuned in the second stage for localization and classification.

[0087] Since both networks are anchor-based, the RPG module can also be incorporated into the one-stage anchorless network FCOS. Similar to the S2ANet method, the original backbone of the network is replaced, and the RPG module is embedded after ReFPN.

[0088] As an optional example, the effect of using the technical solution of this application for target detection is shown in the figure. Figure 6 As shown.

[0089] It should be noted that this application proposes an efficient method called RPG for aerial target detection tasks, which provides more accurate directional encoding while being robust to outliers. Furthermore, this application proposes two modular solutions to address feature instability: a first-stage segmentation and aggregation approach and a second-stage enhancement approach.

[0090] It should be noted that because the ReDet network does not consider solving the problem of feature instability, the rotational equivariant features obtained by the existing ReDet are unstable, while the technical solution of this application can obtain stable rotational representations.

[0091] Furthermore, the method for handling isomorphic features in ReDet is conditional on the usage environment, while the RPG method in this application is plug-and-play.

[0092] Furthermore, since channel rearrangement requires knowledge of the current target's rotation angle, it must be performed in the second stage of a two-stage network. The proposed solution, however, requires no additional target information and can be used plug-and-play in various types of detectors (e.g., effective in single-stage, two-stage, anchor-based, or anchor-free networks).

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0094] This embodiment also provides a target detection device. Figure 7 This is a structural block diagram of a target detection device according to an embodiment of the present invention, such as... Figure 7 As shown, the device includes:

[0095] The first processing module 72 is used to process the target image through an equivariant backbone network to obtain the rotational equivariant features of the target image;

[0096] The second processing module 74 is used to generate a rotationally robust prototype from the rotationally equivariant features, and to perform enhancement processing on the rotationally equivariant features based on the rotationally robust prototype to obtain enhanced rotationally equivariant features, wherein the enhancement processing includes stability enhancement processing and semantic enhancement processing.

[0097] The detection module 76 is used to perform target detection on the target image based on the enhanced rotational equivariance features.

[0098] The aforementioned device generates a rotationally robust prototype from the rotational equivariant features of the target image determined by the equivariant backbone network, and then enhances the rotational equivariant features based on the robust prototype (through stabilization enhancement and semantic enhancement). Finally, it performs target detection on the target image based on the enhanced rotational equivariant features. By improving the stability of the rotational equivariant features and adding semantic enhancements, it solves the problem of poor target detection performance caused by directly using the rotational equivariant features of the image determined by the equivariant backbone network, thus improving the target detection performance of the target image.

[0099] In an exemplary embodiment, the second processing module 74 is further configured to divide the rotational equivariance features according to the number of channels of the rotational equivariance features of the target image to obtain K sub-rotational equivariance features, wherein K is a positive integer and K is divisible by the number of channels of the rotational equivariance features of the target image; and to perform an aggregation operation on the K sub-rotational equivariance features to obtain the rotationally robust prototype.

[0100] In an exemplary embodiment, the second processing module 74 is further configured to add the K sub-rotational isomorphic features to obtain the rotationally robust prototype.

[0101] In an exemplary embodiment, the second processing module 74 is further configured to enhance each of the K sub-rotational isomorphic features according to the rotational robust prototype to obtain K enhanced sub-rotational isomorphic features; and to splice the K enhanced sub-rotational isomorphic features to obtain the enhanced rotational isomorphic features.

[0102] In an exemplary embodiment, the second processing module 74 is further configured to enhance the i-th sub-rotational isomorphic feature among the K sub-rotational isomorphic features in the following manner, so as to enhance each sub-rotational isomorphic feature, wherein i is a positive integer greater than or equal to 1 and less than or equal to K: convolving the rotational robust prototype and concatenating the convolved rotational robust prototype with the i-th sub-rotational isomorphic feature to obtain a first feature; convolving the first feature and adding the convolved first feature with the i-th sub-rotational isomorphic feature to obtain a second feature; and processing the second feature through an activation function to obtain the i-th enhanced sub-rotational isomorphic feature.

[0103] In an exemplary embodiment, the detection module 76 is further configured to, when performing target detection on the target image using a one-stage detector, directly perform object classification and position regression based on the enhanced rotational isotropic features to obtain the target detection result.

[0104] In an exemplary embodiment, the detection module 76 is further configured to, when performing target detection on the target image using a two-stage detector, generate a set of regions of interest using a region generation network and the enhanced rotational isotropic features; perform object classification and position regression on the set of regions of interest based on the rotational isotropic features to obtain a target detection result; or perform object classification and position regression on the set of regions of interest based on the enhanced rotational isotropic features to obtain a target detection result.

[0105] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0106] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0107] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0108] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0109] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0110] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0111] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0112] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method, characterized in that, include: The target image is processed by an equivariant backbone network to obtain the rotational equivariant features of the target image; A rotationally robust prototype is generated using the rotationally equivariant features, and the rotationally equivariant features are enhanced based on the rotationally robust prototype to obtain enhanced rotationally equivariant features. The enhancement process includes stability enhancement and semantic enhancement. Target detection is performed on the target image based on the enhanced rotational isomorphic features; The generation of a rotationally robust prototype through the rotationally equivariant features includes: The rotational isomorphic features of the target image are divided into K sub-rotational isomorphic features according to the number of channels of the rotational isomorphic features of the target image, where K is a positive integer and K is divisible by the number of channels of the rotational isomorphic features of the target image; the K sub-rotational isomorphic features are aggregated to obtain the rotationally robust prototype. The rotationally isotropic feature is enhanced based on the rotationally robust prototype to obtain the enhanced rotationally isotropic feature, including: According to the rotation robust prototype, each of the K sub-rotational isomorphic features is enhanced to obtain K enhanced sub-rotational isomorphic features; the K enhanced sub-rotational isomorphic features are then concatenated to obtain the enhanced rotational isomorphic feature.

2. The method according to claim 1, characterized in that, The rotationally robust prototype is obtained by aggregating the K sub-rotationally equivariant features, including: The rotationally robust prototype is obtained by adding the K sub-rotational isomorphic features together.

3. The method according to claim 1, characterized in that, According to the rotation robust prototype, each of the K sub-rotational isomorphic features is enhanced, including: The i-th sub-rotational isomorphic feature among the K sub-rotational isomorphic features is enhanced in the following way, so as to enhance each sub-rotational isomorphic feature, where i is a positive integer greater than or equal to 1 and less than or equal to K: The rotationally robust prototype is convolved, and the convolved rotationally robust prototype is concatenated with the i-th sub-rotationally equivariant feature to obtain the first feature; The first feature is convolved, and the convolved first feature is added to the i-th sub-rotational isomorphic feature to obtain the second feature; The second feature is processed by an activation function to obtain the i-th enhanced sub-rotational isomorphic feature.

4. The method according to claim 1, characterized in that, Target detection is performed on the target image based on the enhanced rotational isomorphic features, including: When using a one-stage detector to perform target detection on the target image, object classification and position regression are performed directly based on the enhanced rotational equivariance features to obtain the target detection result.

5. The method according to claim 1, characterized in that, Target detection is performed on the target image based on the enhanced rotational isomorphic features, including: When using a two-stage detector to perform target detection on the target image, a set of regions of interest is generated by a region generation network using the enhanced rotational equivariance features. The target detection result is obtained by classifying and regressing the object in the set of regions of interest based on the rotational isotropic features; or by classifying and regressing the object in the set of regions of interest based on the enhanced rotational isotropic features.

6. A target detection device, characterized in that, include: The first processing module is used to process the target image through an equivariant backbone network to obtain the rotational equivariant features of the target image; The second processing module is used to generate a rotationally robust prototype from the rotationally equivariant features, and to enhance the rotationally equivariant features based on the rotationally robust prototype to obtain enhanced rotationally equivariant features, wherein the enhancement process includes stability enhancement process and semantic enhancement process. The detection module is used to perform target detection on the target image based on the enhanced rotational equivariance features; The second processing module is further configured to divide the rotational equivariance features according to the number of channels of the rotational equivariance features of the target image to obtain K sub-rotational equivariance features, wherein K is a positive integer and K is divisible by the number of channels of the rotational equivariance features of the target image; and to perform an aggregation operation on the K sub-rotational equivariance features to obtain the rotationally robust prototype. The second processing module is further configured to enhance each of the K sub-rotational isomorphic features according to the rotational robust prototype to obtain K enhanced sub-rotational isomorphic features; and to concatenate the K enhanced sub-rotational isomorphic features to obtain the enhanced rotational isomorphic features.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 5.

Images