Method for improving tiny object detection based on distribution focal loss

Abstract

The present specification describes a method for improving Distribution Focal Loss (DFL)-based Tiny Object Detection by a terminal, comprising: generating a feature map based on an input image; generating an enhanced feature that strengthens the features of a Tiny Object based on the feature map; generating a distribution logit corresponding to a plurality of bins based on the enhanced feature; applying Softmax to the distribution logit to generate a prediction probability distribution for a boundary offset; performing distribution smoothing on the prediction probability distribution to generate a smoothed prediction probability distribution; representing the Ground Truth offset in an extended bin structure to generate a refined target distribution; and calculating an improved Distribution Focal Loss based on the smoothed prediction probability distribution and the refined target distribution.

Description

Technology Field

[0001] This specification relates to computer vision-based object detection technology, and more specifically, to a method and apparatus for improving Tiny Object Detection that enhances boundary prediction accuracy and learning stability by expanding negative offset representation and improving bin resolution, in application fields requiring tiny object detection performance such as construction safety monitoring, UAV video, and CCTV-based surveillance environments, by improving the structural limitations of the Distribution Focal Loss (DFL) used for bounding box regression in one-stage detection models. Background Technology

[0003] Conventional one-stage object detection models, such as the YOLO and PP-YOLOE series, use a Distribution Focal Loss (DFL)-based boundary prediction method that considers each location on a feature map as an anchor point and predicts the distance from that location to the four boundaries of an object in the form of a distribution. This method is widely adopted because it can express prediction uncertainty as a probability distribution, which is superior to existing L1 and L2 regression-based boundary estimation methods.

[0004] However, such DFL-based architectures are generally designed based on medium-to-large object-centric datasets such as MS COCO, and reveal structural limitations in real-world environments containing many tiny objects (e.g., CCTV-based construction safety monitoring, UAV aerial imagery). For example, in the TAL (Task Alignment Learning)-based anchor selection stage, a problem frequently arises where no positive points are selected at all when the actual size of an object is smaller than the stride between anchors, which causes the loss of learning opportunities for tiny objects.

[0005] Furthermore, since conventional DFLs use only uniformly spaced integer bins with a positive range, they cannot represent cases where an anchor point is located outside an object and a negative offset occurs, resulting in clipping to zero. In addition, the offsets of most microscopic objects are concentrated in the 0–1 range; however, the existing integer-spaced bin structure has extremely limited expressive power for this range, causing the probability distribution to effectively function like a single-dimensional regression. These issues are cited as major causes of reduced IoU, training instability, and detection failures for tiny objects. The problem to be solved

[0007] The purpose of this specification is to solve the problem in which no positive candidates are selected or are extremely scarce in the conventional TAL (Task Alignment Learning) based anchor selection process when the size of a tiny object is smaller than the stride of a feature map.

[0008] Furthermore, the purpose of this specification is to provide a new boundary regression method with an extended bin range and a bin-densified distribution structure suitable for tiny object characteristics.

[0009] The technical problems that this specification aims to solve are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this specification belongs from the detailed description of the specification below. means of solving the problem

[0011] One aspect of the present specification may include a method for improving Distribution Focal Loss (DFL)-based Tiny Object Detection by a terminal, comprising: generating a feature map based on an input image; generating an enhanced feature that strengthens the features of a Tiny Object based on the feature map; generating a distribution logit corresponding to a plurality of bins based on the enhanced feature; applying Softmax to the distribution logit to generate a prediction probability distribution for a boundary offset; performing distribution smoothing on the prediction probability distribution to generate a smoothed prediction probability distribution; representing the Ground Truth offset in an extended bin structure to generate a refined target distribution; and calculating an improved Distribution Focal Loss based on the smoothed prediction probability distribution and the refined target distribution.

[0012] Additionally, the method may further include the step of calculating the bounding box of the Tiny Object based on the boundary regression value calculated from the smoothed prediction probability distribution.

[0013] In addition, enhanced features that strengthen the features of the above Tiny Object can be generated through a Tiny Feature Enhancement module including spatial attention or channel attention.

[0014] In addition, the distribution logit may be based on a signed bin structure including negative bins and positive bins or a densified bin structure subdivided into non-uniform intervals.

[0015] In addition, the refined target distribution may include a multi-bin target distribution generated by interpolating the Ground Truth offset proportionally to the distance between adjacent bins.

[0016] In addition, the step of calculating the improved Distribution Focal Loss may include Adaptive DFL Weighting, which adjusts the loss weights according to the size of the object or the prediction uncertainty.

[0017] Another aspect of the present specification is a terminal for improving Distribution Focal Loss (DFL)-based Tiny Object Detection, comprising: a communication module; a memory; and a processor for functionally controlling the communication module and the memory; wherein the processor

[0018] A step of generating a feature map based on an input image; a step of generating an enhanced feature that strengthens the features of a Tiny Object based on the feature map; a step of generating a distribution logit corresponding to a plurality of bins based on the enhanced feature; a step of generating a prediction probability distribution for a boundary offset by applying Softmax to the distribution logit; a step of generating a smoothed prediction probability distribution by performing distribution smoothing on the prediction probability distribution; a step of generating a refined target distribution by expressing the Ground Truth offset in an extended bin structure; and an improved Distribution Focal Loss can be calculated based on the smoothed prediction probability distribution and the refined target distribution. Effects of the invention

[0020] According to the embodiments of the present specification, when the size of a tiny object is smaller than the stride of a feature map, the problem of no positive candidates being selected or being extremely scarce in the conventional TAL (Task Alignment Learning) based anchor selection process can be solved.

[0021] In addition, according to the embodiments of this specification, a new boundary regression method having an extended bin range and a bin-densified distribution structure suitable for tiny object characteristics can be provided.

[0022] The effects obtainable in this specification are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this specification belongs from the description below. Brief explanation of the drawing

[0024] FIG. 1 is a block diagram for illustrating an electronic device related to the present specification. FIG. 2 is a block diagram of an AI device according to one embodiment of the present specification. FIG. 3 illustrates a DISTRIBUTION FOCAL LOSS algorithm to which the present specification may be applied. FIG. 4 illustrates a method to improve DFL-based Tiny Object Detection to which the present specification can be applied. Specific details for implementing the invention

[0025] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols will be assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the concept and technical scope of this specification.

[0026] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0027] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0028] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0029] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0030] FIG. 1 is a block diagram for illustrating an electronic device related to the present specification.

[0031] The above electronic device (100) may include a wireless communication unit (110), an input unit (120), a sensing unit (140), an output unit (150), an interface unit (160), a memory (170), a control unit (180), and a power supply unit (190), etc. Since the components illustrated in FIG. 1 are not essential for implementing the electronic device, the electronic device described herein may have more or fewer components than those listed above.

[0032] More specifically, among the above components, the wireless communication unit (110) may include one or more modules that enable wireless communication between the electronic device (100) and a wireless communication system, between the electronic device (100) and another electronic device (100), or between the electronic device (100) and an external server. Additionally, the wireless communication unit (110) may include one or more modules that connect the electronic device (100) to one or more networks.

[0033] This wireless communication unit (110) may include at least one of a broadcast receiving module (111), a mobile communication module (112), a wireless internet module (113), a short-range communication module (114), and a location information module (115).

[0034] The input unit (120) may include a camera (121) or video input unit for inputting a video signal, a microphone (122) or audio input unit for inputting an audio signal, and a user input unit (123, e.g., a touch key, a mechanical key, etc.) for receiving information from a user. Voice data or image data collected from the input unit (120) may be analyzed and processed into a control command by the user.

[0035] The sensing unit (140) may include one or more sensors for sensing at least one of information within the electronic device, information about the surrounding environment surrounding the electronic device, and user information. For example, the sensing unit (140) may include at least one of a proximity sensor (141), an illumination sensor (142), a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor: infrared sensor), a fingerprint sensor (finger scan sensor), an ultrasonic sensor, an optical sensor (e.g., see camera (121)), a microphone (see 122), a battery gauge, an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor, etc.), and a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the electronic device disclosed in this specification can utilize information sensed by at least two of these sensors in combination.

[0036] The output unit (150) is intended to generate output related to sight, hearing, or touch, and may include at least one of a display unit (151), an acoustic output unit (152), a haptic module (153), and an optical output unit (154). The display unit (151) may form a layered structure with a touch sensor or be formed integrally to implement a touch screen. Such a touch screen functions as a user input unit (123) that provides an input interface between the electronic device (100) and the user, and at the same time can provide an output interface between the electronic device (100) and the user.

[0037] The interface section (160) serves as a passage for various types of external devices connected to the electronic device (100). This interface section (160) may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. In response to an external device being connected to the interface section (160), the electronic device (100) can perform appropriate control related to the connected external device.

[0038] Additionally, the memory (170) stores data that supports various functions of the electronic device (100). The memory (170) can store a number of application programs (or applications) running on the electronic device (100), data for the operation of the electronic device (100), and commands. At least some of these application programs may be downloaded from an external server via wireless communication. Also, at least some of these application programs may exist on the electronic device (100) from the time of shipment for the basic functions of the electronic device (100) (e.g., phone incoming and outgoing functions, message receiving and outgoing functions). Meanwhile, the application programs may be stored in the memory (170), installed on the electronic device (100), and driven by the control unit (180) to perform the operation (or function) of the electronic device.

[0039] In addition to operations related to the application program, the control unit (180) typically controls the overall operation of the electronic device (100). The control unit (180) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through the components described above, or by running an application program stored in memory (170).

[0040] Additionally, the control unit (180) can control at least some of the components examined together with FIG. 1 in order to run an application program stored in memory (170). Furthermore, the control unit (180) can operate at least two or more of the components included in the electronic device (100) in combination with each other to run the application program.

[0041] The power supply unit (190) receives external power and internal power under the control of the control unit (180) and supplies power to each component included in the electronic device (100). This power supply unit (190) includes a battery, and the battery may be a built-in battery or a replaceable battery.

[0042] At least some of the above components may operate in cooperation with each other to implement the operation, control, or control method of an electronic device according to various embodiments described below. Additionally, the operation, control, or control method of the electronic device may be implemented on the electronic device by running at least one application program stored in the memory (170).

[0043] In this specification, the electronic device (100) may be collectively referred to as a server, and the server may include a cloud server. Additionally, the terminal may include all or part of the configuration of the electronic device (100), and may include a tablet PC.

[0044] FIG. 2 is a block diagram of an AI device according to one embodiment of the present specification.

[0045] The AI ​​device (20) may include an electronic device including an AI module capable of performing AI processing, or a terminal including the AI ​​module. Additionally, the AI ​​device (20) may be configured to be included as at least a part of the configuration of the electronic device (100) shown in FIG. 1 to perform at least a part of the AI ​​processing together.

[0046] The above AI device (20) may include an AI processor (21), memory (25) and / or a communication unit (27).

[0047] The AI ​​processor (21) can be configured to execute program instructions stored in memory (25) to receive a feature map obtained from an input image, a predicted logit per anchor, and information on the size and location distribution of an object as input, and to regress the bounding box location of a tiny object based on probability. Specifically, the AI ​​processor (21) can apply D-TAL (Task Alignment Learning with Distance Guidance), a distance-based anchor expansion technique, to each anchor point on the feature map to reinforce positive samples, and then perform S-DFL (Signed Distribution Focal Loss) and D-DFL (Densified Distribution Focal Loss) modules to predict the boundary offset in the form of a probability distribution. The AI ​​processor (21) integrates the information produced by these three modules to generate a probability distribution vector corresponding to each anchor point and calculates the expected value, thereby finally calculating the boundary regression value.

[0048] In particular, the AI ​​processor (21) uses an expanded bin set that can represent even when offsets occur in the negative direction in order to overcome the limited bin structure based on positive integers of the existing Distribution Focal Loss, and applies a non-uniform (bin-densified) segmentation structure of 0.25 units to enhance the prediction accuracy of the 0±1 region where the offsets of tiny objects are concentrated. At this time, the AI ​​processor (21) sets the expanded bin structure as the prior distribution, and then updates the posterior distribution by calculating the loss for the target distribution given during the learning process. Through this, the AI ​​processor (21) naturally reflects fine boundary movements at the sub-pixel level and performs dynamic probability-based boundary regression that includes the uncertainty of position changes of tiny objects or anchor placement.

[0049] In addition, the AI ​​processor (21) monitors the prediction sensitivity and distribution variability in each empty interval in real time to maintain constant tiny object detection performance, and automatically adjusts the distribution threshold (*Threshold*) for boundary regression according to the concentration and deformation pattern of the distribution. To this end, the AI ​​processor (21) analyzes the matching results between the predicted boundary and the actual boundary to calculate the True Positive Rate (TPR) and False Positive Rate (FPR), searches for the optimal Threshold value for tiny object detection, and stores it in memory (25). This Threshold correction function ensures stable tiny object detection performance while maintaining the false positive and false negative rates at a constant level, despite various environmental changes such as changes in shooting conditions (illumination and noise), morphological variations of small objects, and long-distance shooting. Consequently, the AI ​​processor (21) implements a self-adaptive probability-based boundary regression engine that performs high-precision boundary prediction required for micro-object detection by self-adjusting the resolution, range, and threshold of the probability distribution.

[0050] Meanwhile, the AI ​​processor (21) that performs the functions described above may be a general-purpose processor (e.g., CPU), but may be an AI-dedicated processor for artificial intelligence learning (e.g., GPU, graphics processing unit).

[0051] The memory (25) can store various programs and data required for the operation of the AI ​​device (20). The memory (25) can be implemented as non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), or solid-state drive (SDD). The memory (25) is accessed by the AI ​​processor (21), and the AI ​​processor (21) can perform reading / writing / modification / deletion / updating of data. Additionally, the memory (25) can store a neural network model (e.g., a deep learning model) generated through a learning algorithm for data classification / recognition according to one embodiment of the present specification.

[0052] Meanwhile, the AI ​​processor (21) may include a data learning unit that learns a neural network for data classification / recognition. For example, the data learning unit may learn a deep learning model by acquiring training data to be used for learning and applying the acquired training data to a deep learning model.

[0053] The communication unit (27) can transmit the AI ​​processing results by the AI ​​processor (21) to an external electronic device.

[0054] Here, external electronic devices may include other terminals and / or servers (cloud servers).

[0055] Meanwhile, although the AI ​​device (20) illustrated in FIG. 2 is described by functionally separating it into an AI processor (21), memory (25), and communication unit (27), the aforementioned components may be integrated into a single module and referred to as an AI module or an artificial intelligence (AI) model.

[0056] FIG. 3 illustrates a DISTRIBUTION FOCAL LOSS algorithm to which the present specification may be applied.

[0057] Referring to FIG. 3, the terminal can perform a DFL algorithm including the following steps to regress the object bounding box (offset) from an input image based on a distribution. For example, the following describes steps performed by the terminal to predict the object bounding box from an input image, and each step includes parts performed commonly in the training and inference stages and parts performed only in specific stages.

[0058] The terminal generates a feature map based on the input image (S301).

[0059] Based on the input image, the terminal generates a multi-scale feature map by executing the neural network backbone and Feature Pyramid Network (FPN). The generated feature map contains various spatial and hierarchical information regarding the location, size, and shape of objects, and serves as the basis for subsequent boundary regression processing.

[0060] The terminal considers each pixel location of the feature map as an anchor point and structures it so that the location becomes a representative point for surrounding objects. The multi-scale feature map generated in this way is used as input to perform offset regression in four directions (left, right, top, bottom).

[0061] The terminal generates an offset distribution logit based on the feature map (S302).

[0062] For each anchor point of the feature map, the terminal generates logit values ​​corresponding to multiple bins to regress the distance to the four boundaries of the object in a distribution form. Here, each logit is a raw score indicating the probability that a specific boundary offset belongs to that bin.

[0063] The logit vectors generated at this time are calculated independently for each boundary direction, for example, if integer bins from 0 to K are predefined, logit values ​​are output as many times as the number of bins. These logits are normalized into a probability distribution through a subsequent softmax transformation.

[0064] The terminal generates a Softmax-based probability distribution based on the offset distribution logit (S303).

[0065] The terminal performs a softmax operation on the logit vector calculated in S302 to generate a probability distribution for each bin. Through this, the logit is normalized to a probability value in the range of 0 to 1, and the sum of all bin probabilities becomes 1.

[0066] The generated probability distribution exists independently for each of the four boundaries, and this distribution becomes a predicted probability distribution indicating what value the terminal predicts for the corresponding boundary offset. This probability distribution is used in subsequent steps for calculating the expected value (S304) and the learning loss (S306).

[0067] The terminal calculates an expected value-based boundary regression value based on a probability distribution (S304).

[0068] The terminal calculates the expectation for each offset based on the probability distribution generated in S303. For example, the expectation is calculated by multiplying the bin value by the probability of the corresponding bin and summing the results, ultimately resulting in a real-valued boundary offset value.

[0069] This expected value is used as the final regression value to calculate the bounding box during the inference stage. The distribution-based expected value calculation method has the advantage of reflecting subtle boundary changes more smoothly than conventional scalar regression, and particularly improves boundary regression performance for tiny objects.

[0070] For learning, the terminal generates a target distribution based on the ground truth offset (S305).

[0071] During the training phase, the terminal obtains the Ground Truth (GT) bounding boxes provided along with the input image and converts them into coordinate systems based on each anchor point to calculate the GT offsets in four directions. The terminal generates a target distribution by encoding the GT offsets into a distribution form aligned with a bin structure.

[0072] For example, if the GT offset is located between integer bins such as 3.6, the terminal constructs a target distribution by assigning distance-proportional weights to two adjacent bins. The target distribution thus generated is used as a reference value for comparison with the prediction distribution (S303) in step S306.

[0073] The terminal calculates DFL Loss based on the probability distribution and the target distribution (S306).

[0074] The terminal calculates the Distribution Focal Loss (DFL) based on the predicted probability distribution (S303) and the target distribution (S305). The DFL Loss has a smaller value as the predicted distribution becomes more similar to the target distribution, and is a loss function designed to learn distribution-based regression more precisely.

[0075] The terminal uses the calculated DFL Loss for backpropagation to update the parameters of the entire neural network model, including the backbone, feature map generator, and logit generator. Through this, the terminal continuously improves the accuracy of object boundary regression.

[0076] The terminal calculates a bounding box based on the boundary regression value (S307).

[0077] The terminal calculates the final bounding box of the object from each anchor point based on the four-directional boundary regression values ​​(expected values) calculated in step S304. This is implemented by adding or subtracting offset regression values ​​to the anchor point coordinates, and as a result, a bounding box of the form (x_min, y_min, x_max, y_max) is calculated.

[0078] The terminal can generate a final object detection result by performing post-processing techniques, such as Non-Maximum Suppression (NMS), to remove overlapping bounding boxes as needed. Step S307 is performed during the inference stage and can operate independently of the loss calculation in S306.

[0079] A general Distribution Focal Loss (DFL)-based boundary regression structure estimates the boundaries of an object through feature map generation (S301), distribution logit generation (S302), Softmax probability distribution calculation (S303), and expectation-based regression (S304), as illustrated in the aforementioned Fig. 3. However, in this existing structure, since the target distribution (S305) generated during the training phase only reflects the local shape of the Ground Truth offset, there is a problem in that the peak of the distribution becomes excessively narrow for tiny objects and becomes sensitive to quantization errors. In particular, for tiny objects, the actual GT offset value is smaller than the bin-unit resolution or fluctuates non-linearly, so the DFL Loss (S306) does not sufficiently reflect the fine differences in the distribution, and consequently, there was a limitation in that the precision of the final Bounding Box calculated in step S307 was significantly reduced.

[0080] To solve these problems, this specification is designed to improve structural inefficiencies occurring in the processes of distribution generation, expectation calculation, target distribution formation, and DFL Loss calculation performed in steps S303 to S306 of the processing flow of FIG. 3. Specifically, this specification provides separate pre-processing, post-processing, and intermediate transformation modules that mitigate the problem of the probability distribution becoming excessively sharp for small objects and correct discontinuities in the GT offset, thereby further stabilizing the expectation-based regression (S304) and ensuring that the DFL Loss (S306) reflects the actual boundary difference in a balanced manner. Accordingly, this specification can provide a technical effect that enables the calculation of a more reliable Bounding Box in step S307 by effectively resolving issues such as the degradation of boundary regression performance, distribution distortion, and quantization errors for small objects that were clearly observed in the existing DFL structure.

[0081] FIG. 3 illustrates a DISTRIBUTION FOCAL LOSS algorithm to which the present specification may be applied.

[0082] Referring to FIG. 3, the terminal can perform a DFL algorithm including the following steps to regress the object bounding box (offset) from an input image based on a distribution. For example, the following describes steps performed by the terminal to predict the object bounding box from an input image, and each step includes parts performed commonly in the training and inference stages and parts performed only in specific stages.

[0083] The terminal generates a feature map based on the input image (S301).

[0084] Based on the input image, the terminal generates a multi-scale feature map by executing the neural network backbone and Feature Pyramid Network (FPN). The generated feature map contains various spatial and hierarchical information regarding the location, size, and shape of objects, and serves as the basis for subsequent boundary regression processing.

[0085] The terminal considers each pixel location of the feature map as an anchor point and structures it so that the location becomes a representative point for surrounding objects. The multi-scale feature map generated in this way is used as input to perform offset regression in four directions (left, right, top, bottom).

[0086] The terminal generates an offset distribution logit based on the feature map (S302).

[0087] For each anchor point of the feature map, the terminal generates logit values ​​corresponding to multiple bins to regress the distance to the four boundaries of the object in a distribution form. Here, each logit is a raw score indicating the probability that a specific boundary offset belongs to that bin.

[0088] The logit vectors generated at this time are calculated independently for each boundary direction, for example, if integer bins from 0 to K are predefined, logit values ​​are output as many times as the number of bins. These logits are normalized into a probability distribution through a subsequent softmax transformation.

[0089] The terminal generates a Softmax-based probability distribution based on the offset distribution logit (S303).

[0090] The terminal performs a softmax operation on the logit vector calculated in S302 to generate a probability distribution for each bin. Through this, the logit is normalized to a probability value in the range of 0 to 1, and the sum of all bin probabilities becomes 1.

[0091] The generated probability distribution exists independently for each of the four boundaries, and this distribution becomes a predicted probability distribution indicating what value the terminal predicts for the corresponding boundary offset. This probability distribution is used in subsequent steps for calculating the expected value (S304) and the learning loss (S306).

[0092] The terminal calculates an expected value-based boundary regression value based on a probability distribution (S304).

[0093] The terminal calculates the expectation for each offset based on the probability distribution generated in S303. For example, the expectation is calculated by multiplying the bin value by the probability of the corresponding bin and summing the results, ultimately resulting in a real-valued boundary offset value.

[0094] This expected value is used as the final regression value to calculate the bounding box during the inference stage. The distribution-based expected value calculation method has the advantage of reflecting subtle boundary changes more smoothly than conventional scalar regression, and particularly improves boundary regression performance for tiny objects.

[0095] For learning, the terminal generates a target distribution based on the ground truth offset (S305).

[0096] During the training phase, the terminal obtains the Ground Truth (GT) bounding boxes provided along with the input image and converts them into coordinate systems based on each anchor point to calculate the GT offsets in four directions. The terminal generates a target distribution by encoding the GT offsets into a distribution form aligned with a bin structure.

[0097] For example, if the GT offset is located between integer bins such as 3.6, the terminal constructs a target distribution by assigning distance-proportional weights to two adjacent bins. The target distribution thus generated is used as a reference value for comparison with the prediction distribution (S303) in step S306.

[0098] The terminal calculates DFL Loss based on the probability distribution and the target distribution (S306).

[0099] The terminal calculates the Distribution Focal Loss (DFL) based on the predicted probability distribution (S303) and the target distribution (S305). The DFL Loss has a smaller value as the predicted distribution becomes more similar to the target distribution, and is a loss function designed to learn distribution-based regression more precisely.

[0100] The terminal uses the calculated DFL Loss for backpropagation to update the parameters of the entire neural network model, including the backbone, feature map generator, and logit generator. Through this, the terminal continuously improves the accuracy of object boundary regression.

[0101] The terminal calculates a bounding box based on the boundary regression value (S307).

[0102] The terminal calculates the final bounding box of the object from each anchor point based on the four-directional boundary regression values ​​(expected values) calculated in step S304. This is implemented by adding or subtracting offset regression values ​​to the anchor point coordinates, and as a result, a bounding box of the form (x_min, y_min, x_max, y_max) is calculated.

[0103] The terminal can generate a final object detection result by performing post-processing techniques, such as Non-Maximum Suppression (NMS), to remove overlapping bounding boxes as needed. Step S307 is performed during the inference stage and can operate independently of the loss calculation in S306.

[0104] A general Distribution Focal Loss (DFL)-based boundary regression structure estimates the boundaries of an object through feature map generation (S301), distribution logit generation (S302), Softmax probability distribution calculation (S303), and expectation-based regression (S304), as illustrated in the aforementioned Fig. 3. However, in this existing structure, since the target distribution (S305) generated during the training phase only reflects the local shape of the Ground Truth offset, there is a problem in that the peak of the distribution becomes excessively narrow for tiny objects and becomes sensitive to quantization errors. In particular, for tiny objects, the actual GT offset value is smaller than the bin-unit resolution or fluctuates non-linearly, so the DFL Loss (S306) does not sufficiently reflect the fine differences in the distribution, and consequently, there was a limitation in that the precision of the final Bounding Box calculated in step S307 was significantly reduced.

[0105] To solve these problems, this specification is designed to improve structural inefficiencies occurring in the processes of distribution generation, expectation calculation, target distribution formation, and DFL Loss calculation performed in steps S303 to S306 of the processing flow of FIG. 3. Specifically, this specification provides separate pre-processing, post-processing, and intermediate transformation modules that mitigate the problem of the probability distribution becoming excessively sharp for small objects and correct discontinuities in the GT offset, thereby further stabilizing the expectation-based regression (S304) and ensuring that the DFL Loss (S306) reflects the actual boundary difference in a balanced manner. Accordingly, this specification can provide a technical effect that enables the calculation of a more reliable Bounding Box in step S307 by effectively resolving issues such as the degradation of boundary regression performance, distribution distortion, and quantization errors for small objects that were clearly observed in the existing DFL structure.

[0106] FIG. 4 illustrates a method to improve DFL-based Tiny Object Detection to which the present specification can be applied.

[0107] Referring to FIG. 4, a method for improving DFL-based Tiny Object Detection according to one embodiment of the present specification is described in more detail. In this embodiment, the terminal performs a series of steps S401 through S408 to calculate the bounding box of a tiny object more precisely from an input image. This method includes improved distribution generation, sophisticated GT representation, and adaptive weight-based learning to overcome the structural limitations of the existing Distribution Focal Loss (DFL)—loss of negative offset information, insufficient bin resolution, and probability clustering of tiny objects.

[0108] The terminal generates a feature map based on the input image (S401).

[0109] The terminal first executes a backbone neural network and a feature pyramid network (FPN) on the input original image to generate multilayer feature maps with different resolutions, strides, and channel counts. These feature maps are used as a basic representation space for object detection and can be re-corrected in a subsequent Tiny Feature Enhancement step, taking into account the limited number of pixels for small objects.

[0110] Here, the terminal is the generated feature map It can be passed as input to a tiny object-specific feature enhancement module (S402). For example, the probability distribution space on which the existing DFL is based can be expressed as a basic probability simple field of the following Equation 1.

[0111]

[0112] However, this distribution structure has a problem in that it does not sufficiently represent the actual boundary information of tiny objects, as it only represents information for positive numbers (bin index 0~n).

[0113] The terminal performs Tiny Feature Enhancement based on the feature map (S402).

[0114] The terminal detects regions within the feature map generated by S401 that primarily contain small objects and amplifies the features using a small object-dedicated enhancement module. For example, this module can perform spatial / channel attention and high-resolution correction to compensate for the low-contrast and low-resolution features of small objects.

[0115] The following mathematical equation 2 exemplifies the linear combination form implemented by the Tiny Feature Enhancement module.

[0116]

[0117] Here, F_base is the original feature extracted from S401, F_tiny is the small object-based enhancement feature, can mean a tiny object emphasis ratio (0 <= α <= 1). In this way, by securing a tiny object-only representation, the boundary characteristics of small objects can be reflected more clearly in subsequent logit generation (S403).

[0118] The terminal generates an enhanced logit generation based on Tiny Feature Enhancement (S403).

[0119] The terminal has the reinforcement features generated in step S402. A logit that takes as input and represents the offset distance to the four directional (left, top, right, bottom) boundaries for each bin Creates.

[0120] Referring to the following mathematical formula 3, logit generation can be expressed as the following convolution-based formula.

[0121]

[0122] Here, is the "logit score" for the k-th bin (e.g., -m~n), is a tiny feature map enhanced in the S402 stage, is the parameter of the CNN-based regression head for the k-th bin, and "*" can denote a convolution operation. In the DFL structure, the "boundary offset" is expressed in the form of a probability distribution rather than a single numerical value. Therefore, logits for each bin (e.g., 0–15 or -4–15) must be generated first. When these logits pass through softmax, they become the "probability distribution" of S404.

[0123] In addition, if a tiny object-only branch is included, the outputs of the two branches can be combined as shown in Equation 4 below.

[0124]

[0125] Here, is the basic boundary regression head, is a regression head dedicated to tiny objects, is a tiny feature-only map, and "+" can mean the fusion of the outputs of two paths (branch).

[0126] For example, if the terminal has the following two parallel paths (branches):

[0127] 1. basic branch - Create default logit for general object size

[0128] 2. tiny branch - Additional correction path reflecting only the characteristics of small objects

[0129] These two results can be combined to generate the final logit.

[0130] Since small objects have boundaries concentrated at very short distances (bins 0–1) and the distribution is distorted in the existing DFL, the terminal tiny branch can generate a logit optimized for tiny objects by using a filter with a smaller receptive field, larger resolution, and a more sensitive gradient to correct this distortion.

[0131] The terminal generates a softmax-based probability distribution based on an improved distribution logit (S404).

[0132] The terminal is the logit calculated in S403 For , perform a softmax operation as shown in the following mathematical formula 5 to obtain the probability value for each bin You can obtain .

[0133]

[0134] Referring to mathematical formula 5, the logit of each bin obtained in step S403 probability value Defines a softmax operation for normalization. Here, is the regression score (logit) of the k-th bin, is the probability value after softmax, and the denominator can mean the exponential sum of the logits of all bins -> normalization constant.

[0135] The terminal performs Distribution Smoothing based on the probability distribution (S405).

[0136] If the probability distribution calculated in S404 is concentrated in a single bin or a small number of bins, the terminal performs Gaussian-based distribution smoothing according to the following mathematical formula 6.

[0137]

[0138] Here is the distribution after smoothing, is the distance-based Gaussian weight between bin k and bin j, represents the smoothing strength. The terminal can normalize the distribution back to 1 after smoothing according to Equation 6. The terminal considers the probability distribution p^_k smoothed in S405 as the final prediction distribution, and for convenience, it is denoted as p_k below.

[0139] This smoothing process can fundamentally mitigate gradient bursting, prediction instability, and distribution distortion of tiny objects. Therefore, Equation 6 is a mathematical definition of "the process of readjusting the probability of each bin by reflecting the probability of neighboring bins."

[0140] The terminal performs GT Refinement for learning and generates a target distribution (S406).

[0141] The terminal performs the GT Refinement module to represent the offset of the Ground Truth (GT) bounding box more precisely. Since the existing DFL supports only positive offsets, information regarding negative offset areas (especially fine movement values ​​around small objects) is discarded.

[0142] To this end, the terminal applies a signed DFL structure to represent the entire range from negative to positive as an extended bin set as shown in Equation 7 below.

[0143]

[0144] Equation 7 extends the bin structure of the existing Distribution Focal Loss (DFL), which previously only represented positive offsets from 0 to n, to include a positive-negative symmetric range of negative (-m to -1) + 0 + positive (1 to n). Through this, the terminal can express an accurate offset as a distribution even if the GT boundary location is to the left (or above) the anchor point.

[0145] And the refined target GT distribution can be expressed in the form of an expectation value as in Equation 8.

[0146]

[0147] Here, is the GT target probability corresponding to bin k, is the offset candidate value (bin index), can refer to the expected offset value expressed by the refined GT. That is, the refined target distribution Based on this, mathematical formula 8 calculates the expected value of the GT offset.

[0148] In the existing DFL, GT distribution was as follows:

[0149] 1. In the two nearest bins

[0150] 2. Linear proportional weights

[0151] 3. Use only positive bin

[0152] Therefore, most of the fine GT position information of the tiny object was lost here. To compensate for this, Equation 8 was designed to represent the Tiny object GT with a richer probability distribution.

[0153] The terminal calculates DFL Loss based on the probability distribution and the target distribution (S407).

[0154] The terminal is the probability distribution generated in S404~S406 and target distribution Using this, the Distribution Focal Loss (DFL) is calculated according to the following mathematical formula 9.

[0155]

[0156] In addition, the terminal can apply Adaptive DFL Weighting according to the following mathematical formula 10 to selectively increase the learning weight for tiny objects.

[0157]

[0158] In mathematical formula 10, the tiny object weight can be defined as in mathematical formula 11.

[0159]

[0160] Referring to mathematical equation 11, since a larger weight is assigned as the size of the object decreases, the tiny object loss is reflected more significantly, and the overall Tiny Object Detection performance can be improved.

[0161] The terminal calculates a boundary regression value based on the predicted probability distribution on which Distribution Smoothing has been performed, and calculates a Bounding Box for the Tiny Object according to the boundary regression value (S408).

[0162] The terminal can restore the bounding box coordinates corresponding to each anchor point using the predicted probability distribution calculated in S404 and S405 and the distribution-based boundary regression value calculated based on the network parameters learned in step S407. For example, when the coordinates of a specific anchor point on a feature map are (x_a, y_a), the terminal can calculate the bounding box in the original image coordinate system in the form of (x_min, y_min, x_max, y_max) by applying expected offset values ​​(e.g., Δ_l, Δ_r, Δ_t, Δ_b) for the left, right, up, and down directions. At this time, since the expected offset values ​​are precisely expressed at the sub-pixel level through a signed bin structure and a densified bin structure, the terminal can smoothly and continuously estimate the boundary location even for tiny objects.

[0163] Furthermore, the terminal can significantly reduce the positional error of tiny objects compared to the existing DFL-based boundary regression method by using regression values ​​that reflect the learning results of distribution smoothing performed in S405, the refined target distribution generated in S406, and Adaptive DFL Weighting applied in S407. Specifically, the terminal generates candidate bounding boxes by considering both the prediction score and the boundary regression value for anchor points corresponding to tiny objects, and then removes duplicate boxes by applying post-processing techniques such as Non-Maximum Suppression (NMS) or Soft-NMS when multiple candidate boxes overlap. Through this, the terminal prevents the phenomenon of an excessive number of boxes being output for the same object and can selectively maintain only the final Tiny Object Detection results with high reliability.

[0164] Furthermore, the terminal may additionally apply a post-processing policy specialized for tiny object detection. For example, for bounding boxes where the object size is less than or equal to a predefined threshold, a more relaxed NMS threshold may be used, or score re-scoring linked to the tiny object weight (w_tiny) calculated in S407 may be performed, thereby reducing misses for small objects and improving detection sensitivity. In this way, step S408 serves as a final step that converts the improved distribution-based boundary regression result obtained through the entire processing flow of FIG. 4 into a detection result in the form of an actual bounding box, thereby performing the role of finally implementing the Tiny Object Detection improvement effect of this specification.

[0165] The foregoing specification may be implemented as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drives (HDDs), Solid State Disks (SSDs), Silicon Disk Drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc., and also include implementations in the form of carrier waves (e.g., transmission over the Internet). Accordingly, the above detailed description should not be interpreted restrictively in all respects and should be considered exemplary. The scope of this specification should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this specification are included within the scope of this specification.

[0166] Furthermore, although the above description has focused on the services and embodiments, this is merely illustrative and does not limit the scope of this specification. Those skilled in the art will understand that various modifications and applications not exemplified above are possible without departing from the essential characteristics of the services and embodiments. For example, each component specifically shown in the embodiments may be modified and implemented. Differences related to such modifications and applications should be interpreted as being included within the scope of this specification as defined in the appended claims.

Claims

Claim 1 A method for improving Distribution Focal Loss (DFL)-based Tiny Object Detection by a terminal comprises: generating a plurality of feature maps having different resolutions and strides based on an input image; applying spatial attention or channel attention to a region containing a Tiny Object among the plurality of feature maps to generate an enhanced feature that strengthens the features of the Tiny Object; generating a distribution logit corresponding to a plurality of bins to represent each boundary offset from an anchor point to the left boundary, right boundary, upper boundary, and lower boundary of the Tiny Object in the form of a probability distribution based on the enhanced feature; applying Softmax to the distribution logit to generate a predicted probability distribution for each of the boundary offsets; readjusting the probability value of each bin of the predicted probability distribution based on a Gaussian weight according to the distance between the bin and another bin and the probability value of the other bin, and normalizing the readjusted probability values ​​so that the sum of the readjusted probability values ​​becomes 1 to generate a smoothed predicted probability distribution; and a Ground Truth bounding box A step of converting each of the left boundary, the right boundary, the upper boundary, and the lower boundary based on the anchor point into a Ground Truth offset; a step of representing the Ground Truth offset as an extended bin structure of non-uniform spacing, comprising at least one negative bin corresponding to a negative offset value, a bin corresponding to an offset value of 0, and a plurality of positive bins corresponding to a positive offset value, wherein the adjacent bin spacing of the offset interval including 0 is set narrower than the adjacent bin spacing outside the offset interval;A method for improvement comprising: a step of generating a refined target distribution by assigning target probability values ​​to adjacent bins in proportion to the distance between the Ground Truth offset and the offset value corresponding to each of the adjacent bins when the Ground Truth offset is located between adjacent bins of the extended bin structure; a step of calculating a weighted Distribution Focal Loss by applying a loss weight determined to increase as the size of the Tiny Object decreases to the Distribution Focal Loss between the smoothed prediction probability distribution and the refined target distribution; and a step of calculating boundary regression values ​​for each of the left boundary, the right boundary, the upper boundary, and the lower boundary based on an expected value calculated using the offset value corresponding to each bin included in the smoothed prediction probability distribution and the probability value of the corresponding bin, and calculating the bounding box of the Tiny Object based on the boundary regression values. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 In a terminal for improving DFL (Distribution Focal Loss)-based Tiny Object Detection, a communication module; a memory; and a processor for functionally controlling the communication module and the memory; wherein the processor generates a plurality of feature maps having different resolutions and strides based on an input image, applies spatial attention or channel attention to a region containing a Tiny Object among the plurality of feature maps to generate an enhanced feature in which the features of the Tiny Object are strengthened, generates a distribution logit corresponding to a plurality of bins to express each boundary offset from an anchor point to the left boundary, right boundary, upper boundary, and lower boundary of the Tiny Object in the form of a probability distribution based on the enhanced feature, applies Softmax to the distribution logit to generate a prediction probability distribution for each of the boundary offsets, readjusts the probability value of each bin of the prediction probability distribution based on a Gaussian weight according to the distance between the corresponding bin and another bin and the probability value of the other bin, normalizes the readjusted probability values ​​so that the sum of the readjusted probability values ​​becomes 1 to generate a smoothed prediction probability distribution, and the Ground Truth bounding box Converting each of the left boundary, right boundary, upper boundary, and lower boundary based on the anchor point into a Ground Truth offset, and representing the Ground Truth offset as an extended bin structure of non-uniform spacing that includes at least one negative bin corresponding to a negative offset value, a bin corresponding to an offset value of 0, and a plurality of positive bins corresponding to a positive offset value, wherein the adjacent bin spacing of the offset interval including 0 is set narrower than the adjacent bin spacing outside the offset interval.A terminal that, when the Ground Truth offset is located between adjacent bins of the extended bin structure, generates a refined target distribution by assigning target probability values ​​to the adjacent bins in proportion to the distance between the Ground Truth offset and the offset value corresponding to each of the adjacent bins, calculates a weighted Distribution Focal Loss by applying a loss weight determined to increase as the size of the Tiny Object decreases to the Distribution Focal Loss between the smoothed prediction probability distribution and the refined target distribution, calculates boundary regression values ​​for the left boundary, the right boundary, the upper boundary, and the lower boundary respectively based on the expected value calculated using the offset value corresponding to each bin included in the smoothed prediction probability distribution and the probability value of the corresponding bin, and calculates the bounding box of the Tiny Object based on the boundary regression values.

Images