Object detection apparatus and method

Abstract

The present invention relates to an object detection device and method. More specifically, an device for detecting an object having any directionality according to the present disclosure comprises: a backbone for extracting a feature map from an input image; a neck for fusing the feature map extracted from the backbone; and a rotational head for predicting the bounding box, angle, and class of the object from the fused feature map; wherein the backbone may include a rotation-cognitive deformable convolution (RDC) module that predicts an offset of a sampling position for convolution operations based on parameters in polar coordinates. Additionally, the present invention may include an aspect ratio adaptive label assignment (AALA) strategy for performing label assignment for objects with a large aspect ratio. Other embodiments are also possible.

Description

Technology Field

[0001] The present disclosure relates to an object detection device and method. More specifically, the present disclosure relates to an device and method for efficiently and accurately detecting objects having any orientation and various aspect ratios present in an image. Background Technology

[0002] Recently, technology that identifies specific objects by analyzing aerial imagery captured by satellites or drones can be utilized significantly in various fields, including military, industrial, and disaster response. Such aerial images provide a panoramic view of a wide area and may feature objects of various sizes rotated in arbitrary directions or exist in forms with very large aspect ratios, such as bridges or ships.

[0003] Conventional general object detection models can typically be designed to predict the horizontal bounding box (HBB) of an object. However, this HBB method has a limitation in that detection performance deteriorates when the object is rotated at an arbitrary angle (Oriented Bounding Box, OBB), such as in aerial imagery, as it fails to accurately reflect the object's shape and orientation.

[0004] To overcome these limitations, various models for rotating object detection have been proposed. For example, two-stage detectors or transformer-based models can achieve high detection accuracy, but they are computationally very complex and cost-intensive, making them unsuitable for applications requiring real-time inference. Conversely, one-stage detectors have the advantage of fast inference speed, but their ability to robustly learn the features of rotated objects is limited, leading to a tendency for significantly reduced accuracy.

[0005] The performance degradation of such first-stage detectors may be attributed to the conventional Convolutional Neural Network (CNN) structure. Since general convolution operations sample features using kernels in the form of a fixed grid, they are not aligned with the shape of the rotated object, resulting in the inclusion of unnecessary information such as the background; this can lead to problems with misaligned feature alignment. Although Deformable Convolution (DCN) has been proposed to partially improve this issue, DCN predicts the offset of the sampling position based on the Cartesian coordinate system, Since the displacements of the x-axis and y-axis are interdependent, there may be inefficiency in having to learn complex mappings to simultaneously accommodate changes in the scale and rotation of an object. In other words, there may be limitations in that the stability of learning is reduced and the convergence speed is slowed down because the rotation angle and size information of the object cannot be decoupled into independent components.

[0006] In addition, conventional one-stage detectors may experience performance degradation due to limitations in label assignment strategies during the learning process. In particular, existing dynamic assignment methods may fail to select the optimal anchor when the aspect ratio of an object is extremely large, as the prediction difficulty varies significantly depending on the location of the anchor point.

[0007] Therefore, there is a need for a new object detection device and method that can increase learning stability by separating the scale and angle of an object to extract features that explicitly consider rotational directionality while maintaining the high speed and efficiency of a first-stage detector, and by assigning labels fairly and without bias even to objects with large aspect ratios. Prior art literature

[0008] Patent Document: Korean Published Patent Application 10-2025-0151874 The problem to be solved

[0009] The present disclosure aims to provide an object detection device and method capable of providing accurate detection performance while enabling real-time inference by simultaneously achieving high accuracy of a two-stage or transformer-based model while maintaining the high efficiency of a conventional one-stage object detector.

[0010] In addition, the present disclosure may aim to overcome the limitations of conventional fixed grid methods or offset prediction methods based on Cartesian coordinate systems, such as the interdependence between coordinate components and the resulting difficulty in optimization. To this end, the present disclosure aims to provide an object detection device and method capable of efficiently extracting robust rotation recognition features aligned with the actual rotation direction and shape of an object by predicting the sampling position by decoupling scale and angle into independent components based on parameters of a polar coordinate system (r, θ).

[0011] Furthermore, the present disclosure aims to solve the bias problem caused by conventional fixed rectangular area-based label assignment methods when learning objects with large aspect ratios, such as bridges or ships. To this end, it may aim to provide an object detection device and method that perform fair and stable label assignment regardless of the object's shape by converting the object into a normalized space and calculating aspect ratio-invariant centrality.

[0012] In addition, the present disclosure aims to solve the conventional problem in which some correct objects (e.g., external objects) are not assigned positive samples during the top k assignment process when objects overlap, such as a soccer field inside a stadium. To this end, the present disclosure may aim to provide an object detection device and method that maximize the stability of learning by including a collision resolution strategy that ensures all correct objects are forcibly assigned at least one positive sample. means of solving the problem

[0013] An apparatus for detecting an object having any directionality according to the present disclosure for achieving the above-described technical problem comprises: a backbone for extracting a feature map from an input image; a neck for fusing the feature map extracted from the backbone; and a rotary head for predicting the bounding box, angle, and class of the object from the fused feature map; wherein the backbone may include a rotation-cognitive deformable convolution (RDC) module that predicts the offset of a sampling position for convolution operations based on parameters in polar coordinates.

[0014] In addition, the rotation-cognitive deformable convolution (RDC) module can learn and predict the radius (r) and angle (θ) as parameters of the polar coordinate system.

[0015] In addition, the rotational deformable convolution (RDC) module can be predicted by applying deep convolution (DWConv) to input features (x), and then using two branched 1x1 convolution heads, the radius (r) is limited to a non-negative value through a ReLU activation function, and the angle (θ) is limited to a certain range using a Softsign function and then converted to an angle in the range of -π to π through scaling.

[0016] In addition, the rotation-cognitive deformable convolution (RDC) module can convert the predicted radius (r) and angle (θ) into an offset in the Cartesian coordinate system through trigonometric function transformation, and apply the converted offset to a deformable convolution operation to extract rotation-cognitive features.

[0017] In addition, the rotation-cognitive deformable convolution (RDC) module is trained so that the sampling position induced by the transformed offset is better aligned with the actual shape of the object, thereby reducing the influence of the background area and extracting rotation-cognitive features.

[0018] In addition, the above-mentioned rotary-cognitive deformable convolution (RDC) module can configure an RDC-CSPLayer by replacing the standard convolution of the CSPLayer within the CSPNeXt structure of the backbone.

[0019] In addition, unlike the channel attention of a standard CSPLayer, the RDC-CSPLayer may include a lightweight spatial attention to effectively emphasize spatial alignment features extracted from the rotation-cognitive deformable convolution (RDC).

[0020] In addition, the Neck can receive multi-scale feature maps extracted from the backbone including the RDC-CSPLayer and perform multi-scale feature fusion through the PAFPN structure.

[0021] In addition, the Rotated Head can perform predictions for each of the fused feature maps by including a bounding box regression branch that predicts the location of an object; an angle regression branch that predicts the tilt of an object; and a classification branch that predicts the type of object.

[0022] Additionally, the rotary head can be configured to predict any directional bounding box within a range including 0 or 90 degrees for an angle (θ), as rotation angles exceeding 90 degrees due to the geometric characteristics of the rectangular bounding box result in the same shape being repeated.

[0023] Additionally, the device may include a processor that, during the learning phase, calculates the total loss (L) by calculating the outputs of the bounding box regression branch and the angle regression branch as the Rotated IoU loss (Lr) and the output of the classification branch as the Focal loss (Lc), and the processor may determine the positive / negative samples to be used for the loss calculation according to an aspect ratio adaptive label assignment (AALA) strategy.

[0024] In addition, the processor can dynamically match correct objects based on the predicted rotation-object intersection union ratio (RIoU), confidence, and aspect ratio-invariant centrality at anchor points within square regions according to the aspect ratio adaptive label assignment (AALA) strategy, unlike the conventional method of statically assigning positive samples at the center of the correct answer.

[0025] In addition, the processor can define a central sampling region having different scaling factors and strides for each feature level through Multi-scale Centric Sampling (MCS).

[0026] In addition, the processor can, as part of AMS (Aspect-ratio aware Metric based Selection), transform a candidate anchor point within the central sampling area into a normalized 1x1 coordinate system space with a large aspect ratio and calculate aspect ratio-invariant centrality based on a normalized distance within the transformed space.

[0027] Additionally, the processor may calculate a matching score by combining the calculated aspect ratio-invariant centrality, the rotated intersection-union ratio (Rotated IoU) between the predicted box and the correct box, and the predicted classification confidence from the classification branch of the rotated head with a predetermined weight, in order to calculate the prediction score of the candidate anchor points as part of the AMS.

[0028] In addition, when calculating the matching score, the processor can emphasize semantic information by setting the weight of the classification confidence higher than the weight of the aspect ratio-invariant centrality and the weight of the rotated intersection-union ratio (Rotated IoU), thereby reflecting the characteristic that it is difficult to distinguish between the background and the object.

[0029] Additionally, the processor may, as part of the AMS, assign the top k candidate anchor points with the highest scores as positive samples for each correct object based on the calculated matching score.

[0030] In addition, the processor can ensure, through Conflict Resolution for Overlapping (CRO), that a correct answer object that did not receive a positive sample during the top k allocation processes is forced to receive at least one positive sample from the anchor point with the highest centrality.

[0031] A method performed by a device for detecting an object having any directionality according to the present disclosure for achieving the technical problem described above comprises: a step in which a backbone extracts a feature map from an input image; a step in which a neck fuses the feature map extracted from the backbone; and a step in which a rotary head predicts the bounding box, angle, and class of the object from the fused feature map; and may further comprise a step in which a rotational cognitive deformable convolution (RDC) module predicts an offset of a sampling position for convolution operations based on parameters of polar coordinates.

[0032] Additionally, the rotation-cognitive deformable convolution (RDC) module may include a step of learning and predicting a radius (r) and an angle (θ) as parameters of the polar coordinate system.

[0033] In addition to this, a computer-readable recording medium for recording a computer program for implementing the present disclosure may be further provided. Effects of the invention

[0034] According to the means for solving the problem described above in the present disclosure, it is possible to provide the effect of simultaneously achieving high efficiency of a first-stage detector and high accuracy of a second-stage detector. Specifically, it is possible to achieve detection accuracy (e.g., 81.37% mAP) equivalent to or greater than that of a conventional high-performance model (e.g., RTMDet-RL), while simultaneously securing a faster inference speed (e.g., 35.3 FPS), and reducing the number of model parameters and computational load by nearly half, thereby providing an excellent effect of enabling high-performance detection even in environments requiring real-time inference.

[0035] According to the aforementioned means for solving the problem of the present disclosure, unlike conventional offset prediction methods based on Cartesian coordinate systems, a rotation-cognitive deformable convolution (RDC) module that predicts offset based on polar coordinate (r, θ) parameters can be provided. Through this, the inefficiency of conventional methods requiring the learning of interdependent coordinate components (x, y) is improved, and by learning scale and angle by decoupling them into independent components, the sampling position of the convolution operation is learned to be better aligned with the actual shape and rotation angle of the object, thereby reducing unnecessary influence from the background and effectively extracting the object's unique rotation-cognitive features, thereby providing the effect of improving detection accuracy.

[0036] According to the aforementioned means for solving the problem of the present disclosure, when learning objects with extremely large aspect ratios, such as legs, the problem of learning bias caused by conventional fixed rectangular area-based label assignment methods can be resolved. Since the present disclosure converts the object into a normalized space and calculates aspect ratio-invariant centrality, positive samples are assigned in a bias-mitigated manner not only to the center of the object but also to candidate anchor points located at the elongated ends, thereby providing the effect of improving learning stability and detection performance for elongated objects.

[0037] According to the aforementioned means for solving the problem of the present disclosure, the problem of label assignment conflicts occurring in situations where objects overlap, such as when a soccer field overlaps inside an athletics track, can be resolved. The present disclosure ensures that at least one positive sample is forcibly assigned even to a correct object (e.g., an external object) that has not been assigned a positive sample during the top k assignment processes, thereby providing the effect of enabling the model to learn all correct objects without omission and increasing detection accuracy.

[0038] According to the aforementioned means for solving the problem of the present disclosure, when calculating a matching score for selecting positive samples, not only location information (centrality, RIoU) but also classification reliability can be considered with high weight. Through this, even in environments where the distinction between an object and a background is ambiguous, such as aerial imagery, semantic information can be further emphasized, thereby providing the effect of selecting the highest quality candidate anchor point as a positive sample.

[0039] According to the aforementioned means for solving the problem of the present disclosure, the device of the present invention is not limited to the field of aerial imaging but can be applied to all technical fields requiring any directional object detection, such as robotics, autonomous driving, and medical imaging. Furthermore, since the present disclosure is a broader concept that includes a horizontal bounding box with a rotation angle of 0 degrees or 90 degrees, it is compatible with general object detection tasks and can have the effect of providing high versatility and scalability.

[0040] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0041] FIG. 1 is a block diagram showing a simplified structure of an object detection device according to an example of the present invention. FIG. 2 is a flowchart illustrating an object detection method according to an embodiment of the present invention. FIG. 3 is a block diagram showing in detail the configuration of the overall architecture, RDC-CSPLayer, and AALA algorithm according to an example of the present invention. FIG. 4 is a flowchart showing the detailed operation of a rotation-cognitive deformable convolution (RDC) module according to an example of the present invention. FIG. 5 is an illustrative diagram schematically explaining the concepts of rotation-cognitive convolution (RDC) and aspect ratio-invariant centrality (AALA) according to an example of the present invention. FIG. 6 is a block diagram illustrating the detailed internal operation process of a rotation-cognitive deformable convolution (RDC) module of the present invention according to an example of the present invention. FIG. 7 is a flowchart showing the detailed operation of a learning step (AALA) according to an example of the present invention. FIG. 8 is an illustrative diagram for explaining the concept of coordinate transformation of aspect ratio adaptive normalization used in aspect ratio-invariant centrality calculation according to an example of the present invention. FIG. 9 is an example diagram comparing how effectively an AALA strategy (right) according to an example of the present invention assigns labels to an elongated object (top) and an overlapping object (bottom) compared to a prior art (FCOS, left). FIG. 10 is a visual comparison diagram showing that the RDCNet backbone (top) according to an example of the present invention extracts rotational cognitive feature maps more clearly than the prior art (middle, bottom). FIG. 11 is an example diagram comparing how well final detection is performed for small objects, elongated objects, and overlapping objects compared to the prior art according to an example of the present invention. Specific details for implementing the invention

[0042] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general content in the art to which this disclosure pertains or content that overlaps between embodiments is omitted. The terms part, module, component, and block as used in the specification may be implemented in software or hardware, and depending on the embodiments, a plurality of parts, modules, components, and blocks may be implemented as a single component, or a single part, module, component, or block may include a plurality of components.

[0043] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are directly connected but also cases where they are indirectly connected, and indirect connections include connections made via a wireless communication network.

[0044] Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0045] Throughout the specification, when it is stated that a component is located "on" another component, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0046] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0047] Singular expressions include plural expressions unless there is an obvious exception in the context.

[0048] In each step, identification codes are used for convenience of explanation and do not describe the order of the steps; the steps may be performed differently from the specified order unless a specific order is clearly indicated in the context.

[0049] The operating principles and embodiments of the present disclosure will be described below with reference to the attached drawings.

[0050] In this specification, the device according to the present disclosure includes all various devices capable of performing computational processing and providing results to a user. For example, the device according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may be in the form of any one of them.

[0051] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0052] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0053] The above portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0054] FIG. 1 may be a block diagram showing a simplified structure of an object detection device according to an example of the present invention.

[0055] Referring to FIG. 1, in one embodiment, an object detection device (10) can detect an object having any orientation within an image. Such an object detection device (10) may include a backbone (100), a neck (200), a rotary head (300), a processor (400), an input unit (500), and an output unit (600).

[0056] In one embodiment, the backbone (100) can perform the function of receiving an input image from an input unit (500) and extracting a feature map. The backbone (100) may be designed to integrate rotation recognition functions while achieving lightweighting by adopting a CSPNeXt architecture as the basic structure instead of a conventional YOLOv8 backbone. For example, the backbone (100) may include a rotation recognition deformable convolution (RDC) module (110), which is a core component of the present invention, and an RDC-CSPLayer (120) including the RDC module (110).

[0057] In one embodiment, the rotation-cognitive deformable convolution (RDC) module (110) can extract rotation-cognitive features by predicting the offset of the sampling position by decoupling the scale and angle into independent components based on parameters in a polar coordinate system (r, θ), unlike conventional methods using offsets based on a fixed grid or Cartesian coordinate system (x, y). The RDC module (110) can predict the radius (r) and the angle (θ) respectively through two branched 1x1 convolution heads after applying deep convolution (DWConv) to the input feature (x). At this time, the radius (r) is limited to a non-negative value through a ReLU activation function, and the angle (θ) is limited to a certain range using a Softsign function and then converted to a range of -π to π through scaling to be predicted. Subsequently, the RDC module (110) can convert the predicted radius (r) and angle (θ) into an offset in the Cartesian coordinate system through trigonometric function transformation, and apply the converted offset to a deformable convolution operation. Through this process, the sampling position of the RDC module (110) can be aligned with the shape of the object to extract object-related features that reduce the influence of the background.

[0058] In one embodiment, the RDC-CSPLayer (120) may be a module that replaces the standard CSPLayer within the CSPNeXt structure of the backbone (100). Referring to FIG. 3, the RDC-CSPLayer (120) may be configured by replacing the 3x3 convolution module of the standard CSPLayer with the 3x3 RDC module (110) of the present invention. Additionally, unlike the channel attention of the standard CSPLayer, the RDC-CSPLayer (120) may include lightweight spatial attention to improve feature representation in both channel and spatial dimensions.

[0059] In one embodiment, the neck (200) can receive multi-scale feature maps such as P3, P4, and P5 extracted from the backbone (100). The neck (200) adopts the PAFPN structure of YOLOv8 to effectively fuse the multi-scale feature maps through a bottom-up path and a top-down path. The neck (200) can transmit the fused feature maps to a rotary head (300).

[0060] In one embodiment, the rotational head (300) receives a multi-scale feature map fused from the neck (200) and can perform the function of ultimately predicting the bounding box, angle, and class of an object. The rotational head (300) operates in an anchor-free manner and is configured to predict not only arbitrary directional bounding boxes (OBB) but also horizontal bounding boxes (HBB) where the angle (θ) is predicted to be 0 or 90 degrees, thereby having high versatility. For example, the rotational head (300) may include a bounding box regression branch (310), an angle regression branch (320), and a classification branch (330).

[0061] In one embodiment, the bounding box regression branch (310) can output a regression vector that predicts the distance (t, l, b, r) from an anchor point to four sides of the bounding box of an object.

[0062] In one embodiment, the angle regression branch (320) can output a regression vector that predicts an angle (θ) indicating the tilt of the object.

[0063] In one embodiment, the classification branch (330) can predict the confidence that an object exists at a location and the type of the object (e.g., airplane, bridge, etc.).

[0064] In one embodiment, the processor (400) controls the overall operation of the object detection device (10) and, in particular, may oversee the training stage. The processor (400) may calculate the Rotated IoU loss (Lr) based on the outputs of the bounding box regression branch (310) and the angle regression branch (320), and calculate the Focal loss (Lc) based on the output of the classification branch (330) to calculate the total loss (L). In particular, the processor (400) may perform an Aspect Ratio Adaptive Label Assignment (AALA) strategy, which is a core component of the present invention, to determine positive / negative samples to be used for the loss calculation. When performing the above AALA strategy, the processor (400) can (1) define a central sampling area for each feature level through MCS (Multi-scale Centric Sampling), and (2) calculate aspect ratio-invariant centrality (Ctr) by converting objects with large aspect ratios into a normalized 1x1 space through AMS (Aspect-ratio aware Metric based Selection). Subsequently, the processor (400) calculates a matching score (M) by combining the centrality (Ctr), Rotated IoU (RIoU), and classification confidence (Prob) with a predetermined weight, and (3) assign the top k positive samples based on the matching score (M) while considering task alignment. Finally, the processor (400) can (4) ensure that correct objects (overlapping objects, etc.) that have not been assigned a positive sample are forcibly assigned at least one positive sample through CRO (Conflict Resolution for Overlapping).

[0065] In one embodiment, the input unit (500) can receive various images for which arbitrary directional object detection is required, including remote sensing (RS) images such as aerial images or satellite images.

[0066] In one embodiment, the output unit (600) may output a result in which an arbitrary directional bounding box (OBB) is visualized on an image, or a list containing the coordinates, angle, and class information of an object, based on the prediction result of the rotary head (300) and the processing of the processor (400).

[0067] FIG. 2 may be a flowchart illustrating an object detection method according to an embodiment of the present invention.

[0068] Referring to FIG. 2, in one embodiment, in operation 101, the input unit (500) can receive an input image. For example, the input image may be a remote sensing (RS) image obtained via a satellite or a drone, and the image may include objects with arbitrary orientations such as an airplane (PL), a bridge (BR), a ship (SH), and objects with a large aspect ratio such as a bridge. The input unit (500) can receive such an image and transmit it to the backbone (100).

[0069] In one embodiment, in operation 102, the backbone (100) can extract a feature map. For example, the backbone (100) may include an RDC-CSPLayer (120) in which the standard convolution operation is replaced by a rotation-cognitive deformable convolution (RDC) module (110) based on the CSPNeXt architecture. When extracting features from an input image, the backbone (100) can predict the offset of the sampling position for the convolution operation by decoupling the scale and angle into independent components based on parameters of polar coordinates (e.g., radius (r) and angle (θ)) instead of a fixed grid method or an offset prediction method based on Cartesian coordinates (Δx, Δy). Through this, the backbone (100) can secure a sampling position aligned with the rotational direction of the object, and extract rotation-aware features from a position placed on the actual shape of the object and transmit them to the neck (200).

[0070] In one embodiment, in operation 103, the neck (200) can fuse multi-scale feature maps. For example, the neck (200) can receive multi-scale feature maps such as P3 (8x downsampling), P4 (16x downsampling), and P5 (32x downsampling) extracted from the backbone (100). The neck (200) can effectively fuse low-level feature maps and high-level semantic feature maps by combining bottom-up and top-down paths through a PAFPN structure. The fused multi-scale feature maps can be transmitted to a rotary head (300).

[0071] In one embodiment, in operation 104, the rotational head (300) can predict an object based on a fused feature map. For example, the rotational head (300) can receive a fused feature map from the neck (200) in an anchor-free manner and predict the possibility of an object existing at each position (anchor point) within the image and specific information about the object. The rotational head (300) may include a bounding box regression branch (310), an angle regression branch (320), and a classification branch (330) for prediction.

[0072] In one embodiment, in operation 105, the rotary head (300) can perform a bounding box regression branch operation (position prediction). For example, the bounding box regression branch (310) can predict a four-dimensional regression vector (e.g., t, l, b, r) representing the distance from an anchor point to four sides (top, bottom, left, right) defining the bounding box of an object according to an anchor-free method (e.g., FCOS).

[0073] In one embodiment, in operation 106, the rotary head (300) can perform an angle regression branch operation (tilt prediction). For example, the angle regression branch (320) can output a one-dimensional regression vector that predicts an angle (θ) representing the tilt of the object. The angle regression branch (320) is a configuration added to a general horizontal object detection head and can perform a key prediction that enables the present invention to detect any directional object.

[0074] In one embodiment, in operation 107, the rotary head (300) can perform a classification branch operation (type prediction). For example, the classification branch (330) can predict a classification confidence / probability that determines whether the anchor point is an object or a background, and if it is an object, class information indicating what kind of object it belongs to (e.g., airplane, ship, etc., 15 categories of the DOTA dataset).

[0075] In one embodiment, in operation 108, the output unit (600) may output a detection result of an arbitrary directional object (OBB / HBB). For example, the output unit (600) may output a final detection result by combining the bounding box, angle, and class information predicted in operations 105 to 107. The detection result may be expressed as an arbitrary directional bounding box (OBB) in the form of a 5-dimensional vector (t, l, b, r, θ) or the coordinates of four vertices of an object (8-dimensional vector). In particular, since the present invention is a concept that includes a horizontal bounding box (HBB) where the angle (θ) is predicted to be 0 or 90 degrees, it can provide a detection result of high versatility applicable not only to aerial imagery but also to general object detection tasks.

[0076] FIG. 3 may be a block diagram showing in detail the configuration of the overall architecture, RDC-CSPLayer, and AALA algorithm according to an example of the present invention.

[0077] Referring to FIG. 3, the architecture of the top (RDCNet) can illustrate the overall data flow of the object detection device (10) according to the present invention. The architecture can be largely composed of three main components: a backbone (100), a neck (200), and a rotary head (300).

[0078] Specifically, the backbone (100) may be based on a lightweight CSPNeXt structure as illustrated in CSPNeXt with RDC-CSPLayer, but may be in a form in which the RDC-CSPLayer (120), which is the core module of the present invention, is integrated. The backbone (100) can extract multi-scale feature maps such as P3 (4x downsampling), P4 (8x downsampling), and P5 (16x downsampling) from an input image and transmit them to the neck (200).

[0079] As illustrated in YOLOv8 PAFPN, the neck (200) can receive P3, P4, and P5 feature maps from the backbone (100) and perform multi-scale feature fusion through bottom-up and top-down paths.

[0080] The feature map fused from the neck (200) can be transferred to a rotating head (300) named YOLOv8RotatedHead. The rotating head (300) may include three parallel branches: Bbox regression (bounding box regression branch (310)), Angle regression (angle regression branch (320)), and Classification (classification branch (330)).

[0081] FIG. 3 illustrates different data flows during inference and training. During inference, the prediction results (h,w,4), (h,w,1), and (h,w,n) of the three branches (310, 320, 330) can be combined to output a final detection result. On the other hand, during training, the prediction results of the three branches can be passed to an AALA module (i.e., an aspect ratio adaptive label assignment strategy performed by the processor (400)). The AALA strategy can determine positive / negative samples and, based on this, calculate a Rotated IoU loss (Lr) for the bounding box / angle branch and a Focal loss (Lc) for the classification branch to train the model.

[0082] The lower left of FIG. 3 can illustrate the detailed configuration of the RDC-CSPLayer (120) in comparison with the standard CSPLayer. Unlike the standard CSPLayer, which includes a 3x3 convolution module and a 5x5 deep convolution (DW ConvModule) and ultimately uses Channel Attention, the RDC-CSPLayer (120) of the present invention can replace the convolution modules with two sequential 3x3 RDC modules (110). Additionally, the RDC-CSPLayer (120) can further emphasize spatial features by using Spatial Attention instead of Channel Attention.

[0083] The bottom right of Figure 3 can be used to visualize the core operations of the AALA strategy. The MCS diagram represents the Multi-scale Centric Sampling (MCS) operation and shows the definition of a central sampling area (red, yellow, blue, and green rectangles) with different scaling factors (rl) and strides (sl) for each feature level (l=3, 4, 5) for a single ground truth object (GTF). The AMS and CRO diagrams show the results of applying the Aspect-ratio aware Metric based Selection (AMS) and Conflict Resolution for Overlapping (CRO) strategies. Unlike conventional methods where samples are statically assigned to the center of the object, this method visually demonstrates that positive samples are evenly allocated by performing dynamic assignment that considers Task Alignment based on the predicted Rotation-Object Cross-Union Ratio (RIoU), confidence, and aspect ratio-invariant centrality at each anchor point.

[0084] FIG. 4 may be a flowchart showing the detailed operation of a rotation-cognitive deformable convolution (RDC) module according to an example of the present invention.

[0085] Referring to FIG. 4, in one embodiment, in operation 201, a rotation-cognitive deformable convolution (RDC) module (110) may apply deep convolution (DWConv) to an input feature (x). For example, this may mean that the RDC module (110) receives a feature map (x) from a previous layer within the RDC-CSPLayer (120) of the backbone (100). As illustrated in FIG. 4(b), the RDC module (110) may apply 3x3 deep convolution (DWConv) to the input feature (x) as a preliminary feature extraction step for offset prediction. Subsequently, the module (110) may normalize the feature and add non-linearity by sequentially applying activation functions such as LayerNorm and ReLU.

[0086] In one embodiment, in operation 202, the rotation-cognitive deformable convolution (RDC) module (110) can start polar coordinate parameter prediction through branched 1x1 convolution heads. For example, the feature map passed through the DWConv can be passed to two branched 1x1 convolution heads to predict polar coordinate parameters, unlike conventional DCNs which directly predict offsets in Cartesian coordinates (Δx, Δy). This may be a structure for the present invention to explicitly learn the scale and angle of the offset by decoupling them into independent components, and may lead to a radius (r) prediction branch (operation 203) and an angle (θ) prediction branch (operation 204), respectively.

[0087] In one embodiment, in operation 203, the rotation-cognitive deformable convolution (RDC) module (110) can predict the radius (r) (ReLU activation function). For example, the first 1x1 convolution head of the two branched heads can predict the radius (r) as a distance component in polar coordinates. At this time, the RDC module (110) can apply the ReLU activation function to the output of the 1x1 convolution so that the predicted radius (r) value is always limited to a non-negative value greater than or equal to 0.

[0088] In one embodiment, in operation 204, the rotation-cognitive deformable convolution (RDC) module (110) can predict an angle (θ) (Softsign function, range -ð to ð). For example, the second 1x1 convolution head among the two branched heads can predict an angle (θ) which is a directional component in polar coordinates. The RDC module (110) can apply a Softsign function to the output of the 1x1 convolution to normalize it to the range (-1, 1) and then scale it by π so that the predicted angle (θ) value is limited to the range -π to π. This may be intended to prevent divergence of the offset and improve stability in the early stages of learning, while expressing all 360 degrees of directionality.

[0089] In one embodiment, in operation 205, the rotation-cognitive deformable convolution (RDC) module (110) can convert the predicted radius (r) and angle (θ) into Cartesian coordinate offsets (Δx, Δy) through trigonometric transformation. For example, the polar coordinate parameters (r, θ) predicted in operations 203 and 204 can be converted back into Cartesian coordinates to be applied to subsequent deformable convolution operations. The RDC module (110) can calculate the final Cartesian coordinate offsets (Δx, Δy) by performing trigonometric transformations such as Δx = r·cos(θ) and Δy = r·sin(θ).

[0090] In one embodiment, in operation 206, the rotation-cognitive deformable convolution (RDC) module (110) can apply the transformed offset to a deformable convolution operation. For example, the transformed offset (Δx, Δy) calculated in operation 205 can now be used as an offset input for a standard deformable convolution operation. That is, the RDC module (110) may replace the Offset Generation block, in which the conventional DCN directly predicts (Δx, Δy), with the process of operations 201 to 205, which predicts and transforms (r, θ).

[0091] In one embodiment, in operation 207, the rotation-cognitive deformable convolution (RDC) module (110) can extract rotation-cognitive features by having the sampling position rest on the actual shape of the object. For example, through a deformable convolution operation with the transformed offset applied, the sampling position of the RDC module (110) can be flexibly extended beyond a fixed grid. The sampling position (red dot) predicted through polar coordinate system (r, θ) learning is distributed while maintaining a consistent sampling pattern aligned with the object's principal axis and the actual shape (e.g., the tilted wing and fuselage of an airplane), thereby effectively capturing object-related features while reducing the influence of the background area. Through this, the RDC module (110) can effectively extract pure rotation-cognitive features that even consider the rotational directionality of the object.

[0092] In one embodiment, in operation 208, the RDC-CSPLayer (120) can perform spatial attention on the extracted rotation recognition features. For example, the rotation recognition features extracted in operation 207 can be passed to a subsequent processing step of the RDC-CSPLayer (120) including the RDC module (110). Referring to the lower left of FIG. 3, the RDC-CSPLayer (120) can perform lightweight spatial attention to emphasize the spatial importance of the feature map, unlike conventional CSPLayers which use channel attention.

[0093] In one embodiment, in operation 209, the RDC-CSPLayer (120) may output a rotation-cognitive feature map with spatial attention applied. For example, the final feature map, which has undergone spatial attention, may be transmitted to the next stage or neck (200) of the backbone (100) as an output of the RDC-CSPLayer (120). Since the output feature map is a rotation-cognitive feature map richly encoded with rotation and shape information of an object, it can serve as a basis for significantly improving detection accuracy in the subsequent neck (200) and rotating head (300).

[0094] FIG. 5 may be an illustrative diagram for schematically explaining the concepts of rotation-cognitive convolution (RDC) and aspect ratio-invariant centrality (AALA) according to an example of the present invention.

[0095] Referring to FIG. 5, (a) can illustrate the operational concept of the rotation-aware deformable convolution (RDC) module (110) of the present invention. For a rotated object (e.g., an airplane) within the input image in the upper left corner, the present invention can predict parameters (e.g., radius (r) and angle (θ)) in polar coordinates, rather than the conventional Cartesian coordinate system (△x, △y), through Rotated Offset Prediction. In particular, the present invention can solve optimization problems caused by interdependence between coordinate components by predicting the scale and angle of the offset by decoupling them into independent components. The predicted polar coordinate offset can be combined with a Rotation-aware Deformable Convolution Kernel to derive the sampling position (red dot) of the deformable convolution. As shown in the result image at the top right, the induced sampling positions are stably distributed in accordance with the actual shape and rotation angle of the object, thereby enabling the extraction of rotation recognition features of the object in a direction that reduces the influence of the background area.

[0096] Figures 5(b) and 5(c) can illustrate the necessity and effectiveness of the aspect ratio adaptive label assignment (AALA) strategy of the present invention.

[0097] Specifically, FIG. 5(b) can illustrate the Centerness bias problem of the prior art that occurs in elongated objects, such as legs (BR). Since the prior art labeling method (e.g., FCOS) assigns positive samples based on a fixed square or rectangular area, there may be a limitation in that positive samples are biased and assigned only to the center area, which is very short compared to the length of the object, as shown by the red dots in FIG. 5(b).

[0098] In contrast, FIG. 5(c) illustrates the result of applying aspect ratio agnostic centerness according to the AALA strategy of the present invention. The present invention allows a processor (400) to transform a correct object with a large aspect ratio into a normalized 1x1 coordinate system space and calculate aspect ratio agnostic centerness based on normalized distances within the transformed space. As a result, candidate anchor points can be dynamically and evenly assigned as positive samples along the long-edge axis of the object, taking into account task alignment, as shown by the red dots in FIG. 5(c). This effectively solves the problem of conventional learning bias and can improve detection performance for elongated objects.

[0099] FIG. 6 may be a block diagram illustrating the detailed internal operation process of a rotation-cognitive deformable convolution (RDC) module of the present invention according to an example of the present invention.

[0100] Referring to FIG. 6, (a) can be illustrated by visualizing the process of the rotation-cognitive deformable convolution (RDC) module (110) adjusting sampling points to capture rotation-cognitive features. Conventional convolution kernels (the blue grid at the center) are not aligned with the shape of the rotated object (e.g., an airplane) and may include unnecessary features of the background area. In contrast, the RDC module (110) of the present invention can predict offsets based on parameters of polar coordinates by decoupling scale and angle into independent components through offsets Gen. The sampling positions (red dots) induced by the predicted offsets are distributed in a direction aligned with the actual shape of the object, thereby enabling the extraction of rotation-cognitive features in a direction that reduces the influence of the background area.

[0101] FIG. 6(b) may illustrate a detailed internal block diagram of such an RDC module (110). The RDC Module block on the left illustrates a conventional Deformable Convolution operation structure and can receive input features (x) and offsets (Δpn) and output features (y). The present invention may be characterized by replacing the Offset Generation block (dotted line), which is used in the conventional DCN to directly predict the offset (Δpn=(Δx,Δy)) of the Cartesian coordinate system where the coordinate components are interdependent, with the new offset prediction blocks in the center and on the right.

[0102] Specifically, the detailed block diagram on the right illustrates the Get dist & angle process. Input features (x) can first undergo preliminary feature extraction by passing through 3x3 deep convolution (DWConv), LayerNorm, and a ReLU activation function. Subsequently, the features can be passed to two branched 1x1 convolution heads. One 1x1 convolution head can predict by passing through a ReLU activation function to restrict the distance component, the radius (r), to non-negative values. The other 1x1 convolution head can predict by using a Softsign function to restrict the direction component, the angle (θ), to a certain range, and then converting it to a range of -π to π through scaling.

[0103] The central Coordinate Transformation block can receive the polar coordinate system parameters (r, θ) predicted from the right block. The RDC module (110) can convert the polar coordinate system parameters back into Cartesian coordinate system offsets (Δx, Δy) by performing trigonometric function transformations such as Δx=r·cos(θ) and Δy=r·sin(θ).

[0104] Finally, the transformed offset (Δpn) is applied to the deformable convolution operation of the left block and can be used to extract rotational cognitive features (y).

[0105] FIG. 7 may be a flowchart showing the detailed operation of a learning step (AALA) according to an example of the present invention.

[0106] Referring to FIG. 7, in one embodiment, in operation 301, the processor (400) may receive the prediction output of the head branch (bounding box, angle, classification). For example, in the current iteration of the training stage, the processor (400) may receive the prediction values ​​output by the three branches (310, 320, 330) of the rotary head (300), namely the predicted bounding box, the predicted angle (θ), and the predicted classification confidence. These prediction values ​​may be used to calculate the loss by comparing them with the ground truth.

[0107] In one embodiment, in operation 302, the processor (400) may initiate sample allocation (AALA strategy) for loss calculation. For example, the processor (400) may initiate an aspect ratio adaptive label allocation (AALA) strategy that dynamically matches correct objects based on the predicted rotation-object intersection union ratio (RIoU), confidence, and aspect ratio-invariant centrality at anchor points within a square area per feature level, unlike a conventional method of statically allocating positive samples at the center of the correct answer.

[0108] In one embodiment, in operation 303, the processor (400) can define a [MCS] centric sampling region for each feature level. For example, as the first step of the AALA strategy, the processor (400) can define a centric sampling region (Sj) with different scaling factors (rl ∈ 1.5, 2.0, 1.5) and strides (sl) for each feature level (l ∈ 3, 4, 5) through (1) Multi-scale Centric Sampling (MCS). As illustrated in the lower right corner (w. MCS) of FIG. 3, this can correspond to defining multiple rectangular regions (red, yellow, blue, etc.) of different sizes for a single correct object.

[0109] In one embodiment, in operation 304, the processor (400) can analyze candidate anchor points within the [AMS] center sampling area. For example, the processor (400) can start an analysis to perform (2) AMS (Aspect-ratio aware Metric based Selection) strategy on candidate anchor points (p) located within the center sampling area (Sj) defined in operation 303.

[0110] In one embodiment, in operation 305, the processor (400) may convert a correct object with a large aspect ratio into a normalized 1x1 space. For example, the processor (400) may convert the correct object into a normalized 1x1 coordinate system space (Aspect-Agnostic Normalization), as illustrated in the concepts of FIG. 5 (b) and (c), to resolve learning bias caused by a correct object (GT bbox) with a large aspect ratio, such as a leg (BR). This may be a preprocessing step to evaluate all correct objects in the same 1x1 space regardless of the object's original width (w) and height (h) ratios.

[0111] In one embodiment, in operation 306, the processor (400) can calculate aspect ratio-invariant centrality within the transformed space. For example, the processor (400) can calculate aspect ratio-invariant centrality (Ctr) based on the normalized distance (d) between the candidate anchor point (p) and the correct object center within the transformed 1x1 space. Unlike conventional biased centrality as in FIG. 5 (b), this can assign high centrality values ​​along the long-edge axis of the object as in FIG. 5 (c).

[0112] In one embodiment, in operation 307, the processor (400) can calculate a matching score. For example, the processor (400) can calculate a matching score (M) by combining (a) the calculated aspect ratio-invariant centrality (Ctr) (reflecting normalized distance (r)), (b) the rotational cross-union ratio (RIoU) between the predicted box and the correct box (reflecting angle (θ)), and (c) the predicted classification confidence (Confidence) from the classification branch (330) of the rotational head (300) with predetermined weights (α, β, γ) to finally evaluate the degree of agreement between the candidate anchor point (p) and the correct object (g). At this time, by setting the weight of the classification confidence (γ=0.5) higher than the weight of the aspect ratio-invariant centrality (α=0.25) and the weight of the rotation cross-union ratio (RIoU) (α=0.25), task alignment that harmoniously reflects the classification score and positional accuracy can be induced, while simultaneously emphasizing semantic information by reflecting the characteristics of aerial images where it is difficult to distinguish between background and objects.

[0113] In one embodiment, in operation 308, the processor (400) may assign the top k positive samples based on the calculated matching score. For example, the processor (400) may, as part of the AMS, assign (select) the top k candidate anchor points (e.g., k=15) with the highest scores for each correct object (g) based on the calculated matching score (M) as positive samples. This may be a method of dynamically selecting the k highest quality samples from the perspective of task alignment, unlike the conventional method of determining whether they are inside (in / out) a square area.

[0114] In one embodiment, in operation 309, the processor (400) can check if there is a correct object (such as an overlapping object) that has not been assigned a [CRO] positive sample. For example, the processor (400) can check if there is a correct object (g) that has not been assigned a single positive sample during the Top-k assignment process of operation 308 in order to perform the (3) Conflict Resolution for Overlapping (CRO) strategy. This may be to prevent the problem where, as in the example at the bottom of FIG. 7, when a soccer field (SBF) overlaps inside a running track (GTF), positive samples are assigned only to the inner object (SBF) with the higher score and the outer object (GTF) is omitted.

[0115] In one embodiment, if there is a correct answer object that has not been assigned a positive sample, in operation 310, the processor (400) may forcibly assign one anchor with the highest centerness to the object as a positive sample. For example, if, as a result of checking in operation 309, there is a missing correct answer object (e.g., GTF) (Y), the processor (400) may ensure that for the missing object, an anchor point with the highest centerness is forcibly assigned as at least one positive sample, even if the matching score (M) is low.

[0116] In one embodiment, if there are no correct objects that have not been assigned a positive sample, the processor (400) may proceed to operation 311 without additional assignment. For example, if, as a result of checking operation 309, all correct objects (GT) have already secured one or more positive samples through the top k assignments (N), the processor (400) may skip the forced assignment of operation 310 and proceed directly to the next operation 311.

[0117] In one embodiment, in operation 311, the processor (400) may determine the final list of positive / negative samples. For example, as a [branch integration] step, the processor (400) may determine the final list of positive samples (P) by integrating all the top k positive samples assigned in operation 308 and the positive samples forcibly assigned in operation 310 (if operation 310 is performed). All anchor points remaining in the entire set of candidate anchor points (P), excluding the final set of positive samples (P), may be determined as negative samples (N).

[0118] In one embodiment, in operation 312, the processor (400) can calculate the Rotated IoU loss (Lr) and Focal loss (Lc) based on the confirmed samples. For example, the processor (400) can calculate the loss based on the final list of positive samples (P) and negative samples (N) confirmed in operation 311. Specifically, for the outputs of the bounding box regression branch (310) and the angle regression branch (320), the Rotated IoU loss (Lr) can be calculated using only the positive samples (P). Additionally, for the output of the classification branch (330), the Focal loss (Lc) can be calculated using both the positive samples (P) and the negative samples (N).

[0119] In one embodiment, in operation 313, the processor (400) can update the model parameters by calculating the total loss (L). For example, the processor (400) can calculate the total loss (L) by summing the Rotated IoU loss (Lr) and the Focal loss (Lc) (L = Lr + Lc). The calculated total loss (L) can be used for backpropagation to update all learnable parameters (weights) of the object detection device (10) including the backbone (100), neck (200), and rotating head (300).

[0120] FIG. 8 may be an illustrative diagram for explaining the concept of coordinate transformation of aspect ratio adaptive normalization used in aspect ratio-invariant centrality calculation according to an example of the present invention.

[0121] Referring to FIG. 8, the principle of solving the aspect ratio bias problem of the prior art (e.g., FCOS) when the processor (400) of the present invention performs an aspect ratio adaptive label assignment (AALA) strategy, particularly an aspect-ratio aware metric-based selection (AMS) strategy, can be illustrated.

[0122] The left diagram of Fig. 8 can represent the original coordinate system. The distance from the center (blue dot) of a correct object (light blue rectangle) with different widths (W) and heights (H), such as a bridge (BR), that is, a large aspect ratio, to a specific candidate anchor point (red dot) can be expressed by the vector d=(dx, dy).

[0123] The right-hand diagram of FIG. 8 may represent a normalized 1x1 coordinate system space to which Aspect-Agnostic Normalization is applied. The processor (400) may convert the correct object into a normalized 1x1 coordinate system space in which the width (w) and height (h) are each 1 in order to remove aspect ratio bias.

[0124] At this time, the distance vector d=(dx, dy) of the original coordinate system can be converted into a distance d = (2dx / w, 2dy / h) normalized by the actual width (w) and height (h) of the object in the transformed space.

[0125] The processor (400) can calculate the aspect ratio-invariant centrality (Ctr) based on the normalized distance (d) within the transformed space. Since this measures only the distance from the relative center within a 1x1 normalized space that is isotropic in the horizontal and vertical directions, regardless of the object's original aspect ratio, it can solve the problem of positive samples being biased toward the short edge of the object as in the prior art and provide the effect of enabling uniform sample selection along the long edge as well.

[0126] FIG. 9 may be an example diagram comparing how effectively an AALA strategy (right) according to an example of the present invention assigns labels to elongated S-shaped objects (GT bbox) and elongated objects (Elongated bbox (BR), top) and overlapping objects (Overlapped bboxes (GTF, SBF), bottom) compared to a prior art (FCOS, left).

[0127] Referring to Fig. 9, the top and middle Elongated bbox (BR) comparisons can illustrate the label assignment results for objects with a large aspect ratio, such as legs. The central FCOS Assign result can show that, due to the conventional method of using a fixed square sampling area, positive samples (green dots) are sparsely assigned only to a very small central area of ​​the object. This causes learning bias, which can hinder the model from learning the entire shape of the object.

[0128] In contrast, the AALA Assign result on the right can illustrate the result of applying the aspect ratio adaptive label assignment (AALA) strategy of the present invention. The processor (400) of the present invention can dynamically assign positive samples (green dots) considering task alignment based on aspect-agnostic centerness and Multi-scale Centric Sampling (MCS), the predicted rotation-object cross union ratio (RIoU) and confidence at each anchor point, and aspect ratio-agnostic centerness. As a result, positive samples are uniformly and abundantly assigned along the long-edge axis of the object, thereby solving the conventional bias problem and stabilizing learning for elongated objects.

[0129] Additionally, the comparison of the bottom Overlapped bboxes (GTF, SBF) in Fig. 9 can illustrate the label assignment results for an object in which a soccer field (SBF) is overlaid inside a running track (GTF). The central FCOS Assign result may show a problem where positive samples are assigned only to the inner object with a high score (SBF, white dot), while the outer object with a relatively low score (GTF) is not assigned positive samples.

[0130] In contrast, the AALA Assign result on the right illustrates the result of applying the Conflict Resolution for Overlapping (CRO) strategy of the present invention. The CRO strategy checks whether there are ground truth objects (e.g., GTF) that were not assigned positive samples during the top k assignment process, and ensures that the anchor point with the highest centrality for such objects is forcibly assigned to at least one positive sample (blue dot). Through this, the present invention can improve the performance of overlapping object detection by ensuring that all ground truth objects are learned without omission.

[0131] FIG. 10 may be a visual comparison diagram showing that the RDCNet backbone (top) according to an example of the present invention extracts rotational cognitive feature maps more clearly than the prior art (middle, bottom).

[0132] Referring to Fig. 10, the figure allows for the visualization and comparison of P4 level feature maps extracted by three different backbone architectures for LV (Large vehicle), HB (Harbor), and PL (Plane), which are representative object classes of the DOTA v1.0 dataset.

[0133] The top row of FIG. 10 above may show a feature map extracted by the RDCNet backbone (100) of the present invention (i.e., CSPNeXt with RDC-CSPLayer). The middle row may show a feature map extracted by the CSPNeXt backbone of the prior art. The bottom row may show a feature map extracted by the YOLOv8 backbone of the prior art.

[0134] As can be seen from the comparison, the feature maps of the conventional technologies CSPNeXt (middle) and YOLOv8 (bottom) may show activations across a relatively wide and dispersed area regardless of the object's orientation or shape, even when the object is rotated. In particular, the PL (airplane) class may not clearly distinguish the rotated shape of individual objects, and the LV or HB classes may also exhibit characteristics that are not aligned along the elongated shape of the object.

[0135] In contrast, the RDCNet backbone (top) of the present invention can explicitly learn the rotational directionality of an object by decoupleing the scale and angle of the offset through the rotation-cognitive deformable convolution (RDC) module (110). As a result, the feature map (top) of the RDCNet shows that the activation (bright area) in all classes PL, LV, and HB appears more clearly aligned with the actual shape and orientation of the object.

[0136] This may visually demonstrate that the backbone (100) of the present invention effectively learns a rotation-aware feature representation superior to that of the prior art. This superior feature map can be transmitted to the subsequent neck (200) and rotary head (300) to provide a stronger foundation for improving the final detection accuracy.

[0137] FIG. 11 may be an example diagram comparing how well final detection is performed for small objects, elongated objects, and overlapping objects compared to the prior art according to an example of the present invention.

[0138] Referring to FIG. 11, the figure can be illustrated by visually comparing the final detection results of the RDCNet of the present invention (top row) with the detection results of the prior art CSPNeXt (middle row) and YOLOv8 (bottom row) for challenging images of the DOTA v1.0 test dataset. The comparison can be performed for three scenarios that are traditionally difficult in aerial image object detection (OOD): (1) small objects, (2) large aspect-ratio objects, and (3) overlapped objects.

[0139] Specifically, referring to the Small objects (left column) scenario, the conventional technologies CSPNeXt and YOLOv8 (middle, bottom) fail to detect and miss some of the parked small vehicle objects, whereas the RDCNet (top) of the present invention can successfully detect even smaller objects through the RDC module (110) and AALA strategy, thereby demonstrating superior detection performance.

[0140] Furthermore, referring to the Large aspect-ratio (middle column) scenario, for objects with a large aspect ratio, such as ships docked in a port (HB), conventional techniques (middle, bottom) may show a tendency for bounding boxes (bboxes) not to be precisely aligned with the actual shape of the object or to omit some objects. In contrast, the RDCNet (top) of the present invention clearly extracts rotation-cognitive features and is learned through aspect ratio-invariant centrality, thereby achieving more accurate localization accuracy even for elongated objects.

[0141] In addition, referring to the Overlapped objects (right column) scenario, in situations where objects are close to or overlap each other, such as airplanes (PL) or stadiums (GTF), conventional technologies (middle, bottom) may fail to distinguish between objects or the boundaries may be unclear. On the other hand, the RDCNet (top) of the present invention maintains discriminability for overlapping objects through the CRO of the AALA strategy, and can detect each object more reliably.

[0142] As a result, FIG. 11 can visually demonstrate that the RDCNet of the present invention achieves enhanced robustness and superior detection performance across complex and challenging aerial imagery conditions compared to conventional technology through the effective design of the RDC-CSPLayer (120) and AALA strategy.

[0143] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. The instructions may be stored in the form of program code and, when executed by a processor, may generate a program module to perform the operation of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0144] Computer-readable recording media include all types of recording media that store instructions that can be decoded by a computer. Examples include ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0145] As described above, the disclosed embodiments have been explained with reference to the attached drawings. Those skilled in the art will understand that the present disclosure may be practiced in forms different from the disclosed embodiments without changing the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be interpreted restrictively. Explanation of the symbols

[0146] 10: Object detection device 100: Backbone 110: Rotation-Aware Deformable Convolution (RDC) Module 120: RDC-CSPLayer 200: Neck 300: Rotary head 310: Bounding Box Regression Branch 320: Angle regression branch 330: Classification Branch 400: Processor 500: Input section 600: Output section

Claims

Claim 1 An object detection device for detecting an object having arbitrary directionality, comprising: a backbone for extracting a feature map from an input image; a neck for fusing the feature map extracted from the backbone; and a rotary head for predicting the bounding box, angle, and class of the object from the fused feature map; wherein the backbone includes a rotation-cognitive deformable convolution (RDC) module that predicts an offset of a sampling position for convolution operations based on parameters of polar coordinates, and the rotation-cognitive deformable convolution (RDC) module learns and predicts a radius (r) and an angle (θ) as parameters of the polar coordinates. Claim 2 An object detection device according to claim 1, wherein the rotational perception deformable convolution (RDC) module applies deep convolution (DWConv) to an input feature (x), and then, through two branched 1x1 convolution heads, the radius (r) is limited to a non-negative value by passing through a ReLU activation function, and the angle (θ) is normalized to a range of (-1, 1) by passing through a Softsign function, and then scaled to π to be limited to a range of -π to π. Claim 3 An object detection device according to claim 2, wherein the rotational perception deformable convolution (RDC) module converts the predicted radius (r) and angle (θ) into an offset in a Cartesian coordinate system through trigonometric function transformation, and applies the converted offset to a deformable convolution operation to extract rotational perception features. Claim 4 In paragraph 3, the object detection device is characterized in that the rotation-cognitive deformable convolution (RDC) module extracts rotation-cognitive features in which the sampling position induced by the transformed offset is aligned with the shape of the object to reduce the influence of the background. Claim 5 An object detection device according to claim 4, characterized in that the backbone includes a layer (hereinafter referred to as 'RDC-CSPLayer') configured by replacing the 3x3 convolution module with the rotational perception deformable convolution (RDC) module in a layer comprising a 3x3 convolution module and a deep convolution (DWConv) module. Claim 6 An object detection device according to claim 5, wherein the RDC-CSPLayer includes a lightweight spatial attention that emphasizes spatial alignment features extracted from the rotational perception deformable convolution (RDC) module, instead of channel attention. Claim 7 An object detection device according to claim 6, wherein the neck receives multi-scale feature maps extracted from the backbone and fuses the multi-scale feature maps by combining a bottom-up path and a top-down path. Claim 8 An object detection device according to claim 7, wherein the Rotated Head performs prediction by including, for each of the fused feature maps, a Bbox regression branch for predicting the position of an object; an Angle regression branch for predicting the tilt of an object; and a Classification branch for predicting the type of object. Claim 9 The object detection device according to claim 8, further comprising a processor that calculates a matching score combining aspect ratio-invariant centrality, rotation intersection-union ratio (Rotated IoU), and classification confidence for candidate anchor points during learning, assigns the top k anchor points with high matching scores as positive samples, and for a correct answer object that was not assigned a positive sample among the top k assignments, forcibly assigns at least one anchor point having the highest centrality as a positive sample. Claim 10 A method for detecting an object having arbitrary directionality, performed by a device, comprising: a step in which a backbone extracts a feature map from an input image; a step in which a neck fuses the feature map extracted from the backbone; and a step in which a rotary head predicts the bounding box, angle, and class of the object from the fused feature map; wherein a rotational cognitive deformable convolution (RDC) module predicts the offset of a sampling position for a convolution operation based on parameters of polar coordinates; and further comprising the step of learning and predicting a radius (r) and an angle (θ) as parameters of polar coordinates.

Images