Data construction method and device for synthesizing objects into drone viewpoint background images and automatically generating synchronous labels

Abstract

The present invention relates to a device for simultaneously generating a training image and a pixel-unit correct answer label by synthesizing a human object onto a drone viewpoint background image, comprising: a step of forming a placement allowance mask by extracting a walkable area using semantic segmentation and excluding a non-walkable area based on a vegetation index; a step of calculating the on-screen pixel size of a person for each placement candidate point using drone shooting parameters and scene coordinates; a step of selecting a final placement point using multi-constraint scoring including boundary offset, avoidance of overlap with existing structures, minimum distance of adjacent people, direction alignment, and the degree of pixel size matching; a step of synthesizing a foreground person image at the selected point using alpha or Poisson blending by geometrically transforming the foreground person image; and a step of synchronously generating a pixel-unit correct answer label by applying a transformation identical to the geometric transformation to a foreground mask.

Description

Technology Field

[0001] The present invention relates to a method for simultaneously generating a training image and a pixel-level correct answer label by compositing a human object onto a drone viewpoint background image. Background Technology

[0002] The application areas of drone imagery are rapidly expanding to include disaster response, public security and safety control, crowd management, facility inspection, sports, event broadcasting, logistics, and construction monitoring. In these applications, the performance of visual intelligence—such as human recognition and detection, segmentation, and pose estimation—depends heavily on the quality and diversity of the data. However, much of the publicly available training data is constructed primarily from ground camera views, which differs significantly from the distribution of aerial drone imagery. Drone footage possesses unique geometric characteristics, such as changes in flight altitude, strong downward pitch, wide field of view, rotation and vibration, ground flatness and repetitive textures, strong perspective reduction, and vanishing line effects. Consequently, models trained on ground-view data suffer from performance degradation, including missed detections, scale mismatches, and an increase in false positives.

[0003] Collecting and labeling large-scale training data from a drone's perspective is excessively costly and time-consuming. Scenes captured from the air containing hundreds to thousands of human bodies simultaneously involve a large number of objects, and as object sizes shrink to the tens of pixels level, outlines become indistinct; therefore, a high level of expertise is required for pixel-level segmentation, instance differentiation, and keypoint labeling. Furthermore, non-technical constraints—such as security and portrait rights issues at filming locations, demographics, clothing, regional biases in pose distribution, drone flight permits, and weather restrictions—also act as bottlenecks in data construction. Consequently, it is difficult to satisfy the demand for "long-tail" data—which encompasses the diverse scenes, angles, densities, and environmental conditions required for actual services—using only natural data collection methods.

[0004] To compensate for this shortcoming, approaches utilizing synthetic data or generative models have been attempted. However, text-to-image-based generation alone makes it difficult to simultaneously satisfy three requirements: precise perspective and scale consistency for an aerial drone viewpoint, contextual matching with the real background (such as roads, coat lines, boundaries, railings, and stair elevations), and the generation of high-precision, pixel-level ground truth that is immediately available for training.

[0005] Simple “copy-and-paste” compositing methods also often result in arbitrary placement positions or haphazard size estimations, frequently causing problems such as people overlapping on line markers or being awkwardly placed over vegetation, in the center of lanes, or on boundaries. At this time, the combination of boundary halos, the absence of shadows, and unnatural compositing due to color or contrast inconsistencies leads to side effects where the model learns unrealistic cues.

[0006] In particular, geometric consistency is key in drone viewpoints. Even for people of the same height, the pixel size on the screen varies systematically depending on their position due to factors such as camera pitch, focal length, sensor size, flight altitude, and the location of the ground being filmed. If this relationship is ignored, the relative sizes of people in front and behind within the same scene become incorrect, breaking the sense of perspective, and a problem arises where the model mistakes unintended cues, such as background patterns or line thickness, for scale indicators during the training process.

[0007] Therefore, drone composite data requires a system that quantitatively calculates the scale by location and consistently integrates the calculation results through placement, resizing, and label generation.

[0008] Furthermore, reflecting the semantic context of the scene is essential. In real-world environments, people stand in walkable areas such as roads, sidewalks, plazas, and courts, avoiding collisions with grass, flowerbeds, trees, railings, stair edges, vehicles, and facilities. In the synthesis of drone data, these constraints must also be observed for the model to learn realistic distributions. To achieve this, a placement strategy that simultaneously considers multiple constraints is required, including distinguishing permitted or prohibited areas through semantic segmentation, vegetation identification, maintaining boundary offsets, minimizing distances between adjacent people, and aligning with road and line directions. During synthesis, post-processing techniques that enhance visual coherence, such as alpha / gradient (Poisson) blending, color and brightness histogram matching, and simple shadow generation, must be performed in parallel.

[0009] Finally, synchronization of ground truth labels is crucial. In many synthesis pipelines, geometric transformations between images and labels are separated, leading to omissions or inconsistencies, or only bounding boxes are provided, making it difficult to utilize them directly for pixel-level training. In drone viewpoint training, the consistency of ground truths (GTs) for segmentation directly impacts model performance due to small objects, perspective distortion, and complex ground patterns; therefore, a method is required to automatically and synchronously apply the same transformations used in synthesis to masks, keypoints, and instance labels. Furthermore, domain adaptation procedures reflecting color spaces, noise, compression characteristics, population, clothing, and pose distributions tailored to the domestic operating environment must be established to ensure generalization performance in actual deployments. The problem to be solved

[0010] This invention aims to resolve the structural limitations associated with efficiently constructing training data for recognizing, segmenting, detecting, and estimating the pose of human objects from an aerial drone perspective. Specifically, drone footage exhibits an image distribution entirely different from a ground viewpoint due to altitude changes, large downward angles, wide fields of view, strong perspective reduction, repetitive ground textures, roll, and vibration. Conventional simple synthesis or text-based generation methods are prone to failing to simultaneously secure positional scale consistency, alignment with scene semantic context, and pixel-level ground truth labels that perfectly match the synthesis. Consequently, the size of the synthesized person is inconsistent or placed unrealistically over roads, lines, boundaries, and vegetation. Furthermore, the accumulation of boundary dissonance, absence of shadows, and color and contrast inconsistencies increases training noise and domain gaps. Additionally, actually collecting and labeling large-scale drone data is impractical due to constraints such as cost, time, regulations, and portrait rights.

[0011] Accordingly, the present invention aims to provide a consistent and reproducible data construction system that automatically generates image-label pairs by quantitatively calculating the expected pixel size for each candidate point based on drone shooting parameters and scene coordinates, leaving only walkable areas through semantic segmentation and vegetation identification, selecting a natural final placement through an evaluation integrating multiple constraints such as boundary offset, avoidance of structure overlap, minimum distance, direction alignment, and scale match, and synchronously applying geometric transformations and occlusion relationships identical to those of synthesis to foreground masks, keypoints, and bounding boxes.

[0012] The problems solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0013] According to an embodiment of the present invention as a means for solving the aforementioned technical problem, an apparatus for simultaneously generating a training image and a pixel-unit correct answer label by synthesizing a human object onto a drone viewpoint background image comprises: a step of forming a placement allowance mask by extracting a walkable area using semantic segmentation and excluding a non-walkable area based on a vegetation index; a step of calculating the on-screen pixel size of a person for each placement candidate point using drone shooting parameters and scene coordinates; a step of selecting a final placement point using multi-constraint scoring including boundary offset, avoidance of overlap with existing structures, minimum distance of adjacent people, direction alignment, and the degree of pixel size matching; a step of synthesizing a foreground person image at the selected point using alpha or Poisson blending by geometrically transforming the foreground person image; and a step of synchronously generating a pixel-unit correct answer label by applying a transformation identical to the geometric transformation to a foreground mask.

[0014] The above vegetation index is the ExG-ExR index and is characterized by including a shadow removal filter.

[0015] The above semantic partitioning uses the U-Net, DeepLab, or SegFormer backbone and outputs boundary uncertainty as a soft mask to be used as a penalty during batch evaluation.

[0016] In the step of calculating the pixel size above, the camera pitch, focal length, sensor size, y-coordinate on the screen, and the actual height or reference pixel height of the reference person are inputs.

[0017] The above-mentioned multiple constraint scoring is characterized by using a weighted sum of boundary offset, minimum distance, direction alignment, and scale error minimization, and performing global optimization for the simultaneous placement of multiple figures.

[0018] The above synthesizing step includes color or brightness histogram matching and simple shadow synthesizing.

[0019] The above label synchronous generation step is characterized by additionally generating a bounding box, an instance ID, or a human body keypoint label in addition to the split mask.

[0020] A device for simultaneously generating training images and pixel-unit correct labels by compositing a human object onto a drone viewpoint background image comprises: a semantic segmentation unit that divides scene elements of the background image to output a walkable area confidence map; a vegetation determination unit that detects prohibited placement areas using vegetation indices and color and texture features; a scale calculation unit that calculates the on-screen pixel size of a person for each candidate point using shooting parameters and scene coordinates; a placement evaluation unit that determines a final placement point based on boundary offset, overlap avoidance, minimum distance, direction alignment, and scale match; a composite blending unit that composites a foreground person image onto the determined point by geometrically transforming and blending it; and a label synchronization generation unit that generates pixel-unit correct labels by applying a transformation identical to the geometric transformation to a foreground mask.

[0021] The above semantic division unit includes a U-Net, DeepLab, or SegFormer backbone.

[0022] The above vegetation determination unit includes an ExG-ExR index and a shadow removal filter, and the above scale calculation unit uses the actual height of a reference person or the reference pixel height within the scene as an anchor.

[0023] The above-mentioned composite blending unit performs alpha blending or Poisson blending and includes color and brightness matching and shadow synthesis. The above-mentioned label synchronization generation unit generates a bounding box, instance ID, or human body keypoint label in addition to a segmentation mask.

[0024] The above-described label synchronization generation unit is characterized by including a process of performing domain adaptation that reduces the difference between the composite and real-world distributions through color space alignment, noise and blur injection, and correction of complex and pose distributions. Effects of the invention

[0025] According to the present invention, since scene meaning and shooting geometry are quantitatively permeated throughout the entire pipeline, perspective consistency is naturally maintained within a single frame, where people in the foreground appear larger and those in the distance appear smaller, and a layout in which the relationships between lines and boundaries, and between vegetation and structures, correspond to reality is automatically obtained in various backgrounds such as roads, courts, parking spaces, and plazas.

[0026] Since geometric transformations and occlusion information recorded during the synthesis stage are synchronously applied to the labels, labels for segmentation, detection, posing, and tracking are produced in a perfectly aligned state with the resulting image, without even a single pixel of misalignment. This allows the unique characteristics of a drone viewpoint—such as small objects, strong perspective, and partial occlusion—to be accurately reflected in the training.

[0027] Furthermore, through color space alignment, matching of noise compression characteristics, and domain adaptation considering domestic population, clothing, and pose distributions, the gap between the synthetic and real-world distributions is reduced, thereby improving generalization performance in actual deployments.

[0028] Consequently, the present invention significantly reduces the cost and time required to build large-scale drone training data, ensures visual naturalness regarding scale, orientation, ground sensation, lighting, and texture, and provides an automated generation procedure that guarantees the consistency and quality of labels, thereby effectively enhancing human recognition performance across a wide range of drone applications, including surveillance, safety, crowd management, facility inspection, and sports broadcasting. Brief explanation of the drawing

[0029] Figure 1 shows the entire process of naturally synthesizing and arranging human objects from a background image captured by a drone and a public human body dataset using geometric and semantic information of the scene in a single flow. Figure 2 explains the processing flow for naturally compositing a person object onto a drone background image step by step. Figure 3 sequentially shows the procedure for obtaining training images and labels simultaneously by compositing a full-body human object onto a drone viewpoint background. FIG. 4 shows the module configuration of a device for naturally compositing a full-body human object onto a drone viewpoint background and simultaneously automatically generating correct answer labels for learning. Figure 5 shows an example of applying a drone viewpoint crowd image generated by the proposed synthesis pipeline to four scenes with different locations, materials, and perspectives. Specific details for implementing the invention

[0030] Further objects, features, and advantages of the present invention can be more clearly understood from the following detailed description and the accompanying drawings.

[0031] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols are given the same reference number, and redundant descriptions thereof will be omitted.

[0032] In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description is omitted.

[0033] In addition, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that they include all modifications, equivalents, and substitutions that fall within the concept and technical scope of the present invention.

[0034] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0035] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0036] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0037] Figure 1 illustrates the entire process of naturally synthesizing and arranging human objects by taking background images captured by a drone and a public human body dataset, utilizing geometric and semantic information of the scene. First, the vertical and horizontal resolutions and shooting tilt of the photos are read from the background images, and the on-screen size of referenceable objects within the scene is extracted. This geometric information reflects the characteristic that the scene is an aerial view looking at the ground and is subsequently used as a standard to consistently calculate the scale of human objects by location. At the same time, through U-Net segmentation and vegetation index-based analysis, placement-allowable areas such as roads and pedestrian zones are automatically detected, and non-pedestrian areas such as grass and trees are excluded, thereby obtaining candidate zones for synthesis in the form of a mask.

[0038] On the human object side, the actual height of the object and the on-screen size information from the original image are obtained together from datasets such as full-body and pose data of Koreans. The actual and on-screen pair information obtained in this way is combined with the shooting geometry calculated from the background and used to precisely calculate the pixel-unit scale required when placing a person at a specific point. That is, for each candidate point within the allowed area, the appropriate height-to-screen height is calculated by considering the camera tilt and ground perspective, and the foreground person image is resized to match that value.

[0039] Subsequently, an object placement algorithm determines the final position within the allowed area. The algorithm maintains a safe offset from the boundary, guarantees a minimum distance from adjacent figures, and avoids unnatural overlap with objects in the existing scene, such as crosswalk markings and structures.

[0040] If necessary, the orientation of the figures is aligned with the direction of travel to enhance harmony with the scene context. Once the position and size are finalized, each person object is composited using alpha and gradient-based blending (or Poisson blending) to reduce the visual dissonance between the background and the objects, resulting in a final composite image with multiple figures naturally distributed on the screen. The final result cut of the drawing visually demonstrates that the placement has been successfully completed by marking the objects, to which the scale, position, and orientation controls calculated through this process have been applied, with yellow boxes.

[0041] Figure 2 explains the processing flow for naturally compositing a human object onto a drone background image step by step. First, areas classified as vegetation, such as grass and trees, are identified in the input image, and areas where placement is prohibited are removed, leaving only walkable areas. Next, within the remaining area, candidate points where the object can be placed are selected by considering factors such as separation from boundaries, avoidance of overlap with existing markers and objects, and scene context.

[0042] Next, the pixel size of the person on the screen is calculated using the camera's downward angle and the perspective of the target point, and the foreground figure is resized to a scale suitable for each candidate point by matching it with the actual height range. Then, synthesis is performed using alpha or blending techniques to minimize the discrepancy between the background and the foreground. Finally, by synchronously applying the same geometric transformation applied during synthesis to the foreground mask to automatically generate a pixel-level ground truth mask, image-label pairs that can be immediately used for training are completed.

[0043] FIG. 3 sequentially illustrates the procedure for simultaneously obtaining training images and labels by compositing a full-body human object onto a drone viewpoint background. First, a background video is captured by a drone to secure geometric standards such as flight altitude, shaking, roll and pitch angles, and the height of a reference human figure within the scene. Next, the position angle of the insertion point to be composited is estimated; the perspective scale of the point is estimated by calculating the downward angle from the y-coordinate of the point in the vertical position within the screen and the shooting focal length. At the same time, the camera's pitch (downward) angle is separately estimated and corrected to determine the standard for the projection scale to be applied to the entire scene.

[0044] Based on the camera parameters obtained in this way, the actual height of the reference figure within the scene is fixed during the scale anchoring phase, and the height distribution of the virtual figures is defined. Subsequently, the normalized height information and the pixel height of the reference figure measured from the image are used together to precisely calculate the pixel-to-actual length conversion ratio in the scene coordinate system. Next, the estimated pixel height for each insertion point is calculated. This is a process that derives the pixel-unit size that a figure standing at a given point should occupy on the screen by combining the previously obtained camera downward angle, focal length, ground assumption (plane), and the point's y-coordinate. Placement zones are extracted through semantic segmentation. Specifically, roads and pedestrian areas are retained, while the ExG-ExR vegetation index is applied to detect and remove prohibited placement areas such as grass, trees, and flowerbeds.

[0045] When determining placement candidate regions within the filtered masks, contextual constraints—such as avoiding overlap with structures like crosswalks and lanes, safe offsets from boundaries, and alignment with the road direction—are applied to retain only natural locations.

[0046] In the final synthesis, the foreground figure is resized to match the expected pixel height of the candidate points, and alpha blending or Poisson blending is performed to reduce boundary disparity. Finally, geometric transformations identical to those of the synthesis, scaling, rotation, translation, and perspective correction are synchronously applied to the foreground mask to automatically generate GT labels on a pixel-by-pixel basis. This produces a pair of visually natural-looking synthesized figures and registered Ground Truth masks that can be used directly for training for each resulting image.

[0047] FIG. 4 illustrates the module configuration of a device for naturally compositing a full-body human object onto a drone viewpoint background and simultaneously automatically generating correct answer labels for training. The device is broadly composed of a semantic segmentation unit, a scale calculation unit, a vegetation identification unit, a composite blending unit, a placement evaluation unit, and a label synchronization generation and domain application unit, and each module operates as follows.

[0048] The semantic segmentation unit classifies scene elements such as roads, sidewalks, crosswalks, building facades, vehicles, and street trees in the input background image at the pixel level. At this stage, a lightweight U-Net-type network or a state-of-the-art segmentation backbone is used to calculate walkable areas in the form of confidence maps, and uncertainties near the boundaries are represented by soft masks so that they can be utilized for subsequent deployment evaluations.

[0049] The scale calculation unit receives shooting parameters such as flight altitude, camera pitch, focal length, and sensor size, as well as the placement position within the frame, and calculates the pixel height on the screen that a person standing at that point should have. It consistently determines the scale based on perspective for each location by using the actual height of a reference person or a reference scale within the scene as an anchor.

[0050] The vegetation detection unit detects areas where human placement is inappropriate, such as grass, trees, and flowerbeds, by utilizing vegetation indices like ExG-ExR and color / texture features along with semantic segmentation results. Detected areas are marked as hard masks or reduced-weight regions and are preferentially excluded from candidate locations.

[0051] The placement evaluation unit generates candidate points within the allowed zones provided by the semantic segmentation unit and the vegetation discrimination unit, and selects the final placement point using multi-constraint scoring that includes maintaining boundary offsets, minimizing overlap with existing objects, securing minimum distances between adjacent individuals, aligning with the road direction, and minimizing errors with the estimated pixel height calculated by the scale calculation unit. If necessary, it performs global optimization that considers multiple individuals simultaneously.

[0052] The composite blending section resizes and rotates the foreground figure image to match the target pixel height of the selected point, and then applies alpha blending, gradient domain blending, color / brightness histogram matching, and simple ground shadow compositing to reduce boundary discontinuities with the background. The output of this module is a visually natural composite result image.

[0053] The label synchronization generation and domain application unit automatically generates pixel-level Ground Truth by applying the same geometric transformations applied to the foreground subject during the synthesis process to the mask. In addition, it reduces the distribution difference between the synthesized data and the actual collected data by performing domain adaptations tailored to the characteristics of domestic drone data, such as color space correction, texture and noise style transfer, demographics, and clothing and pose distribution correction. Finally, this module produces image-mask pairs, along with additional labels such as instance IDs, keypoints, and bounding boxes if necessary.

[0054] By linking each module in this way, the system implements a consistent pipeline for natural character placement that simultaneously considers the meaning and geometry of the background scene, accurate scaling that conforms to perspective, and automatic generation of correct answer labels synchronized with the composition.

[0055] Figure 5 shows an example of applying drone viewpoint crowd images generated by the proposed synthesis pipeline to four scenes with different locations, materials, and perspectives. In all scenes, the on-screen size of the figures is calculated based on the shooting pitch and ground perspective, so that figures are positioned larger in the foreground closer to the camera and smaller in the background further away. Additionally, positions are selected to satisfy multiple constraints, such as boundary offset, minimum spacing, and orientation alignment, within an allowable zone obtained through semantic segmentation and vegetation identification. The synthesis minimizes visual dissonance through alpha / Poisson blending and simple shadow processing. Pixel-level GT labels for each figure are also generated simultaneously by synchronously applying the same geometric transformation as the synthesis to the mask.

[0057] In the step of forming a placement allowance mask by extracting walkable areas using semantic segmentation and excluding non-walkable areas based on vegetation index, walkable areas are first extracted using semantic segmentation to leave only areas where a person is likely to actually stand from the input drone background image, and non-walkable areas such as grass and trees are removed by discrimination based on vegetation index, and then a placement allowance mask is completed through boundary correction and confidence post-processing.

[0058] First, the image undergoes simple preprocessing, such as color space alignment, gamma and white balance correction, and shadow / highlight mitigation. Subsequently, a semantic segmentation model predicts, pixel by pixel, walkable surfaces—such as roads, sidewalks, plaza pavements, sports courts, and parking spaces—as walkable, while classifying flowerbeds, trees, grass, water surfaces, railings and stair edges, vehicles, and equipment as non-walkable. During this process, the model handles changes in resolution and perspective from an aerial viewpoint using multi-scale inference and tiling, outputting a walkable probability for each pixel. Uncertain areas, such as boundary adjacents, are represented with a low probability, allowing for the application of an offset—a buffer distance—in a later stage.

[0059] Next, non-walkable areas are identified separately using the unique colors and textures of vegetation. In cases where only visible light is present, distributions such as ExG-ExR, Excess Green / Red, and Hue-Saturation are blended. Vegetation indices are binarized using adaptive thresholding, and small point noise is removed based on opening / closing and the minimum area of ​​connected components. This vegetation mask is combined with the semantic segmentation results as a strong exclusion signal to reliably remove grass and tree boundary pixels that were incorrectly identified as walkable.

[0060] Subsequently, the two results are combined to create an initial allowable mask. The combination process is performed by retaining only pixels above a threshold in the pedestrian probability map and excluding non-pedestrian masks—such as vegetation masks, water surfaces, and vehicles—using set difference subtraction. To ensure boundary safety, the outer edges of the allowable mask are receded based on distance transformation to secure a safety offset from crosswalk lines, court lines, curbs, and stair edges.

[0061] In addition, areas that are too narrow for a person to stand on, such as strips or islands, are removed using minimum width and minimum area conditions, while adjacent allowable areas separated by small gaps are connected using a closing operation to reflect the continuity of the actual walking space.

[0062] Finally, confidence weights for the mask are calculated. A pixel-wise weight map is created by synthesizing factors such as walkability probability, inaccuracies in vegetation indices like shadows and light reflection, distance from boundaries, and surface slope estimation, and this is passed directly to the scores in the candidate generation and placement evaluation stages. Consequently, the output of this stage consists of a placement-allowed binary mask and the weights and confidence map used for candidate scoring; these two outputs serve as criteria for evaluating constraints such as candidate point generation, boundary separation, minimum spacing, and orientation alignment in subsequent stages.

[0063] Thanks to the allowable mask configured in this way, the composite figure systematically avoids unrealistic locations such as grass, trees, water surfaces, vehicles, or directly above lines / boundaries, and only placements that correspond to the actual environmental distribution are passed to the next stage.

[0065] The step of calculating the on-screen pixel size of a person at each candidate placement point using drone shooting parameters and scene coordinates is a process that utilizes the geometric information of the background captured by the drone to determine the pixel height at which a person should appear naturally on screen when standing at each candidate point within the allowable mask. First, lens distortion is cleaned up from the video, and shooting parameters such as focal length, principal point, pitch, roll, and flight altitude are restored to the extent possible using drone logs or markers within the video, such as horizons, vanishing points, and coat line spacing. The scene is basically assumed to be a flat ground, but areas where the planar assumption is deviated, such as stairs or ramps, are processed separately based on semantic segmentation results or standard marker information. Through these cleaned internal and external parameters, the projection relationship between the camera and the ground is determined, and a position-dependent transformation ratio is obtained regarding how much the length appears reduced due to perspective as the vertical position of the screen changes.

[0066] Next, the screen coordinates of each candidate point selected from the allowable mask are mapped to ground coordinates, and the position of the feet and head is projected assuming an adult is standing at that point to determine how they would appear on the camera. The vertical pixel difference between the feet and the head obtained from the projection represents the estimated pixel height at that point; this tends to increase relatively as the pitch increases, the drone flies higher, or the focal length lengthens. If the actual human height is given as a range rather than a single value, the same procedure is repeated for the lower and upper limits of the height to calculate the allowable scale range, and constraints are applied during subsequent compositing to ensure that the resizing falls within this range.

[0067] When shooting metadata is incomplete or unreliable, parameters are corrected using regular structures within the scene. For example, absolute scales are fine-tuned using known line spacing, such as on a basketball court or parking space, and pitch or elevation is re-estimated to minimize error using the pixel heights of people already present in the frame and general height distributions. For sloped areas, the ground equations are updated only for those regions to re-project them, and in cases of large roll, rotation correction is applied to ensure the direction of the head and feet aligns with the vertical axis of the screen. During this process, parameter uncertainty and the potential for violating flatness assumptions are recorded together and preserved as scale reliability; results that are too small for low learning utility or overscaled beyond the frame are excluded in advance.

[0068] Ultimately, each candidate point is assigned an estimated value, an acceptable range, and the confidence level of the estimate regarding “how many pixels high a person should appear on screen at that point.” This value is immediately used as the resizing ratio for the foreground figure template, and during the placement evaluation phase, it is reflected in the score to reduce the error with the actual height after synthesis. Furthermore, the same projection relationship is reused in the mask during the label synchronization phase, ensuring that a natural perspective—where people in the foreground appear larger and those in the distance appear smaller—is consistently maintained throughout the entire process, even within a single frame.

[0070] The step of selecting the final placement point using multi-constraint scoring—which includes boundary offset, avoidance of overlap with existing structures, minimum distance to adjacent figures, orientation alignment, and pixel size match—is a process of calculating a single comprehensive score from the five constraints—boundary offset, avoidance of structure overlap, minimum distance to adjacent figures, orientation alignment, and pixel size match—to retain only realistic placements among the candidate points selected within the allowable mask, and selecting the point with the highest score as the final placement. The key lies in modeling each constraint as a soft constraint that "adds if satisfied and subtracts if violated" to ensure stable operation across various scenes.

[0071] First, the boundary offset requires that the figure be kept at a certain distance from areas where it is unstable to stand, such as the edges of the allowable mask, lines, curbs, and stair edges. The implementation uses a “pixel distance map to the boundary” calculated by a distance transformation to award higher scores when the offset exceeds a threshold, and rapidly deducts points when it falls below the threshold. This systematically avoids situations where a figure stands awkwardly on a line or edge corner.

[0072] Next, the avoidance of overlap with existing structures is a constraint designed to prevent overlap with crosswalks, lane markings, facilities, and fixed objects. The intersection area with the structure mask obtained through semantic segmentation and object detection is calculated; a score of 0 is awarded full marks, while weighted deductions are applied as the area increases. In this process, different weights are assigned to each object, such as a weak penalty for thin markings and a strong penalty for large, solid structures.

[0073] The minimum distance between adjacent people is a key constraint for controlling crowding. The nearest distance to already established placement points is calculated; if it is greater than or equal to the minimum distance, it is added, and if it is less, it is subtracted in proportion to the distance deficit. In practice, a smoothing curve is used to smoothly change the score in the vicinity to produce an effect similar to Poisson disk sampling. This prevents crowds from appearing excessively clustered or overlapping, and creates natural, even spacing in large spaces.

[0074] Direction alignment is a constraint that aligns a person's body orientation with the road axis, court lines, and pedestrian flow. It obtains the recommended orientation around the location by estimating the local primary direction from an allowable mask or by utilizing the orientation histogram of line segments. The person's rotation at a candidate point is compared with this recommended orientation; a smaller angular error is added, while a larger error is subtracted. In areas where direction changes, such as curved roads, a local direction field is used to apply different criteria for each point.

[0075] Pixel size match evaluates how well the resized result of the foreground subject to be used for actual compositing matches the “estimated pixel height of the person at that point” calculated in the previous step. By normalizing the difference between the target height and the actual height, higher scores are awarded for smaller errors, while points are sharply deducted if the error exceeds the acceptable range. This serves to maintain perspective consistency throughout the frame, ensuring that people in the foreground appear larger and those in the distance appear smaller.

[0076] The partial scores of the five items obtained in this way are normalized to the range of 0 to 1 and grouped into a weighted sum. The weights can be set differently depending on the nature of the scene. Afterwards, the final point is selected using one of two strategies.

[0077] Finally, the calculated placement undergoes quality checks. If the score distribution is skewed, weights are automatically fine-tuned for re-evaluation, and areas with significant visual dissonance, such as the outer edges of the frame or extreme perspective regions, are excluded using clipping rules. Consequently, the remaining points result in a natural placement that maintains boundaries and safe spacing, avoids awkward overlap with structures, features natural spacing between people, aligns with the surrounding flow and orientation, and accurately matches perspective-based scales. This final placement is then carried over directly to the next stages of compositing, blending, and label synchronization.

[0079] In the step of geometrically transforming foreground character images at selected points and compositing them using alpha or Poisson blending, the foreground character images are geometrically transformed at each previously selected placement point to align with the scene's geometry and context. Then, alpha or Poisson blending is applied to naturally connect the boundaries with the background. First, the foreground character is resized to match the estimated pixel height of the corresponding point obtained during the scale calculation step, and yaw rotation is applied according to local directional fields such as the road axis and line angle. In scenes where vertical lines appear tilted due to roll and pitch effects, corrective rotation is added to align the head and foot axes with the screen's vertical axis, and if necessary, the sense of ground contact is adjusted through fine translation based on foot position. In sections where the foreground image is significantly subjected to perspective distortion, light perspective transformation is applied using trapezoid-rectangle mapping to ensure that body proportions harmonize with the scene's vanishing lines.

[0080] Once geometric alignment is complete, anti-aliasing resampling and pixel-level feathering margins are applied to reduce jagged edges. Subsequently, simple color and contrast histogram matching is performed to minimize differences in color temperature, brightness, and contrast between the background and foreground. Finally, mild film grain and compression noise are added to the foreground to match the unique compression and noise textures of drone footage, thereby mitigating texture inconsistencies.

[0081] Compositing is selected from two modes depending on the situation. In Alpha Blending mode, the foreground matte is softly expanded and contracted to suppress edge halos, and the alpha value of the boundary is weighted according to the rate of change of the background texture to reduce edge dissonance. For fine structures such as hair and lace, morphology-based refinement or simple learning matting is applied to ensure that translucent areas blend naturally. In Poisson Blending mode, the foreground gradient is pasted onto the target area, and the Poisson equation is solved to ensure continuity between the background and the gradient at the boundary. This method effectively absorbs differences in hue and lighting, making it effective in scenes with significant lighting variations, such as indoors or sunsets and shadows. However, for high-frequency textures around boundaries, an edge-preserving term is added or the blending weight is lowered to prevent loss of detail.

[0082] To make the person appear grounded, ground shadows are generated via simple projection by estimating the scene's main light direction beneath the feet. The edges of the shadows are blurred using low-frequency Gaussian blur and transparency attenuation, and transparency is fine-tuned according to the background's contrast to avoid excessive prominence. If necessary, in scenes where parts of an object are positioned behind it, z-order processing with the structure mask is applied to ensure the front-to-back relationship between railings, vehicles, and flowerbed boundaries appears natural.

[0083] When compositing multiple figures simultaneously, the Painter's algorithm is used to draw them based on placement score order or on-screen depth estimation, but surrounding pixels are re-evaluated to ensure that closer objects overlap distant objects and do not violate minimum distance constraints between them. Finally, the edge contrast, skin tone hypersaturation, and partial compression afterimages of the composite result are automatically checked; if they exceed a threshold, local re-blending or color correction is repeated, and pixels overflowing beyond the frame boundaries are safely clipped. The final image obtained through this process achieves a natural composite without awkwardness from a drone view, with scale, orientation, texture, and lighting matching the background, and the same geometric transformations are reused for the next step of label synchronization generation.

[0085] The step of synchronously generating pixel-level ground truth labels by applying the same geometric transformation as described above to the foreground mask is a process of creating pixel-level ground truth labels that do not deviate from the resulting image by even a single pixel by applying the same geometric transformation used in the composite to the foreground mask. First, the transformation procedures recorded during the composite—such as resizing, rotation, translation, and, if necessary, perspective mapping and roll-pitch correction—are reapplied to the mask in the same order. Nearest interpolation is used by default to prevent edges from blurring or becoming jagged; soft mattes with translucent boundaries, such as hair, are first smoothed and then binarized using a threshold just before final output. Fine holes or breaks that may occur during the transformation process are cleaned up using small kernel opening and closing operations and hole filling to refine the image so that the human outline remains continuous.

[0086] When a portion of an image is obscured by structures or other figures during compositing, the obscured area is treated as the background, following the exact front-to-back relationship, so that the occlusion is reflected in the labels. When there is a risk of masks touching and mixing between people, a safety margin of approximately 1 to 2 pixels is applied to separate the boundaries and prevent instance confusion. The labels thus organized are saved in semantic or instance formats according to their purpose; if necessary, the mask's minimum containment rectangle is used as a bounding box, and the same transformation is applied to the joint coordinates of the foreground pose to record them together as keypoint labels. Finally, minimal metadata such as the filename, applied transformation values, and occlusion relationships is retained to ensure reproducibility and verifiability. Through this process, the composite image and labels are fully aligned down to the coordinate system, boundaries, and occlusion state, allowing them to be used immediately in subsequent training stages without separate corrections.

[0088] The device of the present embodiment is configured to identify a walkable area from a drone viewpoint background image, calculate a perspective scale for each location, naturally composite a foreground figure at a point satisfying multiple constraints, and automatically generate synchronous labels. The device is implemented on a computing platform including a processor, memory, mass storage, and a GPU or NPU accelerator, and each module shares common metadata such as shooting parameters, projection matrices, allowable masks, scale confidence, and occlusion order. Input and output between modules are transmitted serially through a pipeline buffer, but can be processed in parallel using an asynchronous queue if necessary.

[0089] The semantic segmentation unit distinguishes human-standing surfaces and structural obstacles in the input drone background image at the pixel level. The image first undergoes color space alignment, exposure and gamma correction, and lens distortion correction before being fed into a lightweight U-Net or Transformer-based segmentation network. The network derives walkable classes—such as roads, sidewalks, plaza pavements, sports courts, and parking spaces—with high confidence, while identifying non-walkable classes—such as vehicles, railings, stair edges, water surfaces, and building exteriors. Uncertainty near boundaries is expressed as a probability value and utilized as a penalty during the subsequent deployment evaluation phase. This module handles high-resolution drone imagery without loss through tile-based multi-scale inference and provides class maps, walkable probability maps, and line and boundary candidate features as output.

[0090] The vegetation discrimination unit complements the semantic segmentation results to definitively exclude vegetation zones, such as grass, trees, and flowerbeds, as non-walkable areas. When only visible light images are provided, a vegetation mask is calculated by combining exponential, channel statistics, and local texture indices, and false detections caused by shadows or dry grass are resolved through adaptive thresholding and morphological operations. When near-infrared bands are present, NDVI / VARI is applied first to enhance robustness. This module ultimately outputs a vegetation exclusion mask and an exclusion intensity map based on boundary proximity; these are combined with the walkability probability from the semantic segmentation unit to form an allowable placement mask. A safety offset is automatically applied to boundary-adjacent areas, such as flowerbed borders and landscaping stones, to prevent placements where humans encroach upon the boundaries.

[0091] The scale calculation unit calculates the estimated pixel height of a person at each point within the allowable mask using shooting parameters and the scene coordinate system. Given intrinsic parameters such as focal length, principal point, and pixel size, as well as extrinsic parameters such as flight altitude, pitch, roll, and yaw, it assumes the ground is a plane and constructs a ground-to-screen homography to derive a position-dependent transformation ratio (pixels-per-meter). After mapping the screen coordinates of candidate points to ground coordinates, it projects two 3D points corresponding to the foot and head positions to determine the vertical pixel difference as the estimated pixel height. Standard dimensions existing in the scene or the actual height of the reference person are used as absolute scale anchors, and an automatic correction routine is applied if the shooting metadata is incomplete. This module outputs the target height for each point, the allowable scale range, the local transformation ratio, and the estimation uncertainty together as scale confidence.

[0092] The placement evaluation unit generates candidate points within an area restricted by an allowable mask and integrates multiple constraints into a weighted sum score to select only realistic placements. Constraint items include the degree of boundary offset compliance, avoidance of overlap with existing structures, minimum distance from adjacent figures, orientation alignment, and pixel size match; each item is normalized to a range of 0 to 1 and dynamically weighted according to the scene type. The selection strategy is based on a greedy approach that sequentially fixes points starting from the highest score point and updates the prohibition radius and score map at every step; in cases requiring the dense placement of multiple figures, lightweight global optimization including a pairwise distance penalty is applied in parallel. This module provides final placement coordinates, figure orientations, and local prohibition area update information for subsequent compositing.

[0093] The composite blending unit aligns the foreground character images according to the scene geometry at each confirmed placement point and blends them visually naturally with the background. First, it resizes to match the target pixel height of the scale calculation unit and applies yaw rotation based on the local direction field. Visual tilt caused by roll and pitch is corrected so that the head-to-foot axis aligns with the vertical axis of the screen, and if necessary, the vanishing line and proportions are adjusted using light perspective transformation.

[0094] To suppress jagged edges, anti-aliasing and feathering are applied, and the difference between the scene's color temperature and brightness is corrected using simple histogram matching. Depending on the scene characteristics, either alpha blending or Poisson blending is selected to absorb edge dissonance and lighting and tone inconsistencies. Ground shadows based on daylight direction estimation are composited to eliminate a floating effect, and for multiple figures, the y-depth or vanishing line-based Painter's algorithm is used to naturally maintain the occlusion relationship. This module records the sequence of applied geometric transformations, matrices, and occlusion (z-order) information in a transformation log and passes it to the next module.

[0095] The label synchronization generation unit automatically generates training labels that perfectly match the composite result image by reapplying the geometric transformations and occlusion information used in the composite to the foreground mask, keypoints, and bounding boxes. The mask transformation uses nearest interpolation by default to preserve boundaries, and fine holes or jagged edges resulting from the transformation are cleaned up using small kernel morphological operations and hole filling. Areas obscured by structures or other people are treated as backgrounds by reflecting the same z-order as the composite, and contacts between instances are separated by a safety margin of 1 to 2 pixels.

[0096] This module outputs semantics, instance split labels, bounding boxes, and keypoints, along with quality and confidence maps if necessary, and stores metadata such as file identifiers, random seeds, summaries of capture and projection parameters, scale confidence, occlusion relationships, and blending modes used. As a result, the generated image-label pairs can be immediately utilized in subsequent training pipelines without additional matching processes.

[0098] The pipeline of this embodiment sequentially performs a series of processes starting from background footage captured by a drone, restoring the geometry of the scene, selecting walkable areas, quantifying the perspective scale by location, naturally compositing foreground figures, and automatically generating synchronization labels.

[0099] First, the input image undergoes simple color space alignment, exposure, and gamma correction. For frames from which lens distortion has been removed, shooting parameters such as focal length, principal point, pitch and roll angles, and flight altitude are restored to the extent possible using internal image markers such as drone logs, horizons, vanishing points, and standard line spacing. A flat ground assumption is applied by default, but areas where the planar assumption is deviated, such as stairs or ramps, are marked as separate regions using semantic cues and structural markers. This establishes the projection relationship between the camera and the ground, and the amount of perspective reduction based on the screen position is acquired as a position-dependent transformation ratio.

[0100] To reflect scene semantics, the semantic segmentation network identifies surfaces where people can stand, such as roads, sidewalks, plaza pavements, sports courts, and parking spaces, at the pixel level, and separates vehicles, railings, stair edges, water surfaces, building exteriors, etc., into a non-pedestrian class.

[0101] Next, vegetation identification based on color and texture is performed to reinforce the exclusion of non-pedestrian areas such as grass, trees, and flowerbeds. The two results are combined to form an allowable placement mask, and a boundary offset—a safe separation based on distance transformation—is applied to account for the instability of boundary adjacent areas, ensuring a minimum distance from lines, curbstones, and stair corners. Narrow strips or small island areas where it is difficult for a single person to stand are removed using morphological rules, thereby maintaining the continuity of the actual pedestrian space.

[0102] For each candidate point within the allowable mask, the on-screen pixel height of a person standing at that point is calculated using the previously restored shooting parameters and ground model. This is performed by projecting 3D points representing foot and head positions onto the camera to determine the vertical pixel difference, and the absolute scale is fixed using reference dimensions present in the scene, such as court line spacing or parking space width, or the actual height of a reference person, as anchors. If the actual height is given as a distribution, the allowable scale range is calculated by performing calculations for the lower and upper limits in parallel, and the uncertainty of the shooting parameter estimation is recorded as scale confidence, which is then used as a weight in the subsequent evaluation stage.

[0103] Placement evaluation is performed using multiple constraint scoring to ensure realistic placement. Each candidate point obtains a partial score in the range of 0-1 based on the degree of boundary offset satisfaction, the overlap area with existing structures such as lanes, text, and facilities, the satisfaction of the minimum distance which is the closest distance to already confirmed placement points, the angle error with the local direction field, and the error between the target pixel height obtained in the previous step and the actual height after synthesis.

[0104] Partial scores are combined into a weighted sum tailored to the scene's nature, and collisions are resolved by making greedy selections starting from the highest score point while updating the forbidden radius and score map with each selection. If necessary, multiple characters are aligned simultaneously using lightweight global optimization, including fairwise distance penalties, to reduce crowding or overlapping.

[0105] Once the final placement is determined, the compositing stage follows. The foreground figure image is resized to match the expected pixel height of the corresponding point, and yaw rotation is applied to align with the road axis or line direction. Visual tilt caused by roll and pitch is corrected to make the head-to-foot axis appear naturally upright, and if necessary, a thin perspective transformation is applied to enhance harmony with the vanishing line.

[0106] After anti-aliasing and feathering to reduce jagged edges, color and contrast differences with the background are simply adjusted, and depending on the situation, alpha blending or Poisson (gradient domain) blending is used to absorb boundary dissonance and lighting differences. Ground shadows are added by estimating the direction of the main light to eliminate the sense of floating, and when there are multiple figures, y-depth or vanishing line-based Painter's algorithms are used to naturally maintain the occlusion relationship between the front and back.

[0107] Simultaneously with the composition, a label synchronization step is performed to reapply the geometric transformations and z-orders used in the composition to the foreground mask, keypoints, and bounding boxes. The mask is transformed using nearest interpolation to preserve boundaries, and fine holes or discontinuities are cleaned up using morphological decomposition. Areas obscured by structures or other figures are treated as zero, ensuring that not even a single pixel of misalignment remains between the image and the label. At this time, necessary formats such as semantic / instance segmentation labels, bounding boxes, and keypoints are output simultaneously.

[0108] Finally, quality checks and metadata recording are performed. Visual defects, such as excessive contrast at composite boundaries, skin tone oversaturation, and local ringing, are automatically detected based on thresholds to perform local re-blending and color correction, while non-utility cases, such as pixel heights that are too small or exceeding the frame boundaries, are excluded in advance. Metadata including the filename, random seed, summary of shooting and projection parameters, scale confidence, occlusion relationships, and blending modes used is attached to each result to ensure reproducibility and annotation auditing. Through this flow, the pipeline automatically produces natural composites and fully matched labels with consistent perspective, orientation, and ground sense across various locations and shooting conditions, thereby constructing a large volume of high-quality datasets that can be directly fed into training drone viewpoint human recognition models.

Claims

Claim 1 A method for generating composite data, comprising: a device for synthesizing a human object into a drone viewpoint background image to simultaneously generate a training image and a pixel-unit correct answer label, wherein the device includes the steps of: extracting a walkable area using semantic segmentation and excluding a non-walkable area based on a vegetation index to form a placement allowance mask; calculating the on-screen pixel size of a person for each placement candidate point using drone shooting parameters and scene coordinates; selecting a final placement point using multi-constraint scoring including boundary offset, avoidance of overlap with existing structures, minimum distance of adjacent people, direction alignment, and pixel size match; geometrically transforming the foreground person image at the selected point and synthesizing it using alpha or Poisson blending; and synchronously generating a pixel-unit correct answer label by applying the same transformation as the geometric transformation to the foreground mask. Claim 2 A method for generating synthetic data according to claim 1, characterized in that the vegetation index is an ExG-ExR index and includes a shadow removal filter. Claim 3 A method for generating synthetic data according to claim 1, characterized in that the semantic partitioning uses a U-Net, DeepLab, or SegFormer backbone and outputs boundary uncertainty as a soft mask to be used as a penalty during batch evaluation. Claim 4 A method for generating synthetic data according to claim 1, wherein in the step of calculating the pixel size, the camera pitch, focal length, sensor size, screen y-coordinate, and the actual height or reference pixel height of a reference person are inputs. Claim 5 A method for generating synthetic data according to claim 1, characterized in that the multiple constraint scoring uses a weighted sum of boundary offset, minimum distance, direction alignment, and scale error minimization, and performs global optimization for simultaneous placement of multiple figures. Claim 6 In claim 1, the synthesizing step is a method for generating synthetic data that includes color or brightness histogram matching and simple shadow synthesis. Claim 7 A method for generating synthetic data according to claim 1, wherein the synchronous generation step of the label further generates a bounding box, an instance ID, or a human body keypoint label in addition to a segmentation mask. Claim 8 A device for simultaneously generating a training image and a pixel-unit correct answer label by compositing a human object onto a drone viewpoint background image, comprising: a semantic segmentation unit that divides scene elements of the background image to output a confidence map of a walkable area; a vegetation determination unit that detects a prohibited placement area using vegetation index and color and texture features; a scale calculation unit that calculates the on-screen pixel size of a person for each candidate point using shooting parameters and scene coordinates; a placement evaluation unit that determines a final placement point based on boundary offset, overlap avoidance, minimum distance, direction alignment, and scale match; a composite blending unit that composites a foreground person image onto the determined point by geometric transformation and blending; and a label synchronization generation unit that generates a pixel-unit correct answer label by applying a transformation identical to the geometric transformation to a foreground mask. Claim 9 In claim 8, the semantic division unit is a device comprising a U-Net, DeepLab, or SegFormer backbone. Claim 10 In claim 8, the vegetation identification unit is a device comprising an ExG-ExR index and a shadow removal filter. Claim 11 In paragraph 8, the scale calculation unit is a device that uses the actual height of a reference person or the reference pixel height within the scene as an anchor. Claim 12 In claim 8, the above-mentioned synthesis blending unit is a device that performs alpha blending or Poisson blending and includes color and brightness matching and shadow synthesis. Claim 13 In claim 8, the label synchronization generation unit is a device that generates a bounding box, an instance ID, and / or a human body keypoint label in addition to a split mask. Claim 14 The device according to claim 8, wherein the label synchronization generation unit comprises a process of performing domain adaptation to reduce the synthetic-real-world distribution difference through color space alignment, noise and blur injection, and complex and pose distribution correction.

Images