Method and system for synthesizing few-sample images based on rotating target perception decoupling
By employing rotational target detection and texture library perspective normalization, the decoupling problem between the target and the background in image enhancement is solved, generating high-quality and diverse training samples suitable for industrial-grade visual model training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-03
AI Technical Summary
Existing image enhancement methods struggle to generate high-quality training samples in scenarios with few samples, rotated targets, and complex backgrounds. In particular, decoupling enhancement between the target region and the background region is difficult to achieve, resulting in distorted images or inaccurate label alignment.
The foreground and background are separated by rotating target detection, a texture library is constructed for perspective normalization, an adaptive fusion algorithm is used to map standard texture blocks to the foreground area, and the background is enhanced by multi-dimensional perturbation, finally generating pixel-level aligned annotation data.
It achieves high-precision target region replacement, improves the visual naturalness and scene diversity of generated samples, and is suitable for data generation for industrial-grade training tasks.
Smart Images
Figure CN121788937A_ABST
Abstract
Description
Technical Field
[0001] This invention relates primarily to the field of image enhancement technology, and in particular to a few-sample image synthesis method and system based on rotation target perception decoupling. Background Technology
[0002] In the fields of computer vision and image enhancement, image synthesis and data augmentation techniques are crucial for improving the generalization ability and robustness of models. Especially in applications such as industrial inspection, remote sensing recognition, security monitoring, autonomous driving, agricultural disease identification, and medical image analysis, model performance is highly dependent on the quantity and quality of training data. However, due to the diverse appearances of targets, complex perspectives, and the unpredictable appearances of similar products, the detection performance of deep learning-based models is severely affected, particularly under conditions of few samples where models struggle to fully learn the diversity of target appearances and environmental adaptability.
[0003] Traditional image enhancement methods mainly include geometric transformations (such as rotation, cropping, scaling, and translation) and appearance enhancements (such as brightness, contrast, and noise perturbation). While these methods can increase the number of training samples to some extent, their main shortcomings include: 1) the enhancement operation acts on the entire image, lacking independent control over the target foreground and background regions, leading to distortion of the target region or unnatural blending with the background; 2) the range of geometric and lighting changes is limited, making it difficult to simulate pose changes, perspective distortion, and environmental interference in real-world scenes; 3) traditional bounding box annotations cannot accurately describe rotated or tilted targets, resulting in deviations between enhanced samples and actual tasks. In recent years, generative methods based on Generative Adversarial Networks (GANs) or Diffusion Models have been used to synthesize image samples. While these methods perform well in certain areas, they face several challenges in low-sample scenarios: 1) GAN or diffusion models require a large amount of data to converge stably, and are prone to overfitting (mode collapse) in low-sample scenarios; 2) Generative models typically model the entire image, making it difficult to achieve decoupling enhancement between the target and the background; 3) During image generation, the position of the generated target cannot be precisely controlled, resulting in inaccurate alignment between the target region and the label, which affects the training of subsequent target detection models.
[0004] In summary, existing image enhancement or generation methods struggle to obtain high-quality enhanced samples in scenarios with few samples, rotated targets, and complex backgrounds. Therefore, it is necessary to propose an efficient image synthesis scheme based on rotated target detection and foreground / background decoupling mechanisms to achieve automated, diverse, and realistic target enhancement under limited sample conditions. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide a few-sample image synthesis method and system based on rotation target perception decoupling, with the aim of generating a large number of high-quality training images based on few-sample image conditions.
[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:
[0007] On one hand, the present invention provides a few-sample image synthesis method based on rotation target perception decoupling, the method comprising:
[0008] Step S1: Perform rotational target detection on the input image, and generate a foreground mask and an image background mask based on the target region and background region of the image;
[0009] Step S2: Construct a texture library, perform perspective standardization processing on the images in the texture library, and generate standard texture blocks;
[0010] Step S3: Map the standard texture block to the foreground mask region through homography transformation, and use an adaptive fusion algorithm to fuse the image texture mapped to the foreground mask region with the original image background;
[0011] Step S4: Enhance the original background to obtain a composite image.
[0012] Furthermore, step S1 includes:
[0013] Call the rotated object detection model to obtain the rotated bounding box parameters of the input image. ,in To rotate the geometric center or centroid of the target bounding box, and These are the width and height of the rotated target bounding box, respectively. This represents the rotation angle of the target bounding box relative to the horizontal axis.
[0014] The four vertices of the rotating target box are obtained based on the parameters of the rotating bounding box. The four vertices of the rotating target box are sorted clockwise from the top left corner according to the polar angle to generate an ordered four-point polygon. The foreground mask and background mask are generated based on the ordered four-point polygon.
[0015] Furthermore, the generation of the ordered four-point polygon specifically includes:
[0016] Based on the polar angles of the four vertices of the rotated target box relative to the center or centroid and radius ;Will Normalization to Obtain the normalized polar angle ;
[0017] Sort the vertices according to their normalized polar angle and radius. And calculate the area of the polygon. , The sorting direction is determined by the area of the polygon. It is counterclockwise. It is clockwise.
[0018] Furthermore, in step S1, if the target rotation detection fails or the confidence level is below the threshold, the target polygon is restored and a foreground mask and a background mask are generated using VOC, COCO, or a custom JSON / CSV polygon vertex format.
[0019] Furthermore, step S2 includes: performing rotation detection on the texture image; if a target is detected, calculating the homography matrix of the four points of the target to the standard reference quadrilateral, and normalizing the perspective of the texture image to a standard texture block through bilinear interpolation.
[0020] Furthermore, step S2 also includes performing histogram matching or mean-variance alignment on the standardized texture blocks for color and brightness.
[0021] Furthermore, step S3 includes:
[0022] Calculate the homography matrix from the standard texture block to the ordered four-point polygon, and perform perspective deformation of the standard texture block to the target area through reverse mapping to replace the foreground mask;
[0023] Based on the local contrast difference, color difference, and texture spectrum difference between the target area and the background area, the Poisson fusion or mask blending algorithm is adaptively selected for boundary fusion.
[0024] Furthermore, when the contrast difference, color difference, or texture spectrum difference is greater than a set threshold, Poisson fusion is preferred; otherwise, mask blending algorithm is preferred for boundary fusion.
[0025] Furthermore, in step S4, one or more of the following methods are used to enhance the background: lighting direction and shadow intensity, fogging coefficient, color shift gain, and random occlusion area ratio and position.
[0026] On the other hand, the present invention also provides a few-sample image synthesis system based on rotation target perception decoupling.
[0027] The rotating target detection and parsing module obtains the rotation bounding box parameters of the target region in the input image, and generates a foreground mask and an image background mask based on the target region and background region of the image.
[0028] The texture library construction and standardization module builds a texture library, performs perspective standardization processing on the images in the texture library, and generates standard texture blocks.
[0029] The texture perspective mapping and fusion module is used to map the standard texture block to the foreground mask area and fuse the image texture mapped to the foreground mask area with the original image background;
[0030] The background scene enhancement module is used to enhance the original background of the background mask area;
[0031] Composite Image Output Module: Used to output composite images.
[0032] The beneficial effects of this invention are:
[0033] (1) Achieve enhanced perception of rotating targets and decoupling between foreground and background, enabling high-precision pixel-level region replacement for targets at any angle;
[0034] (2) A hybrid detection and annotation rollback mechanism ensures that the system can still operate stably when the detection model is unavailable or the detection results are invalid;
[0035] (3) Texture perspective standardization and fusion smoothing mechanism significantly improve the visual naturalness of generated samples;
[0036] (5) Enhanced multi-dimensional background perturbation effectively expands scene diversity;
[0037] (6) The parallel design of the system improves the speed and stability of large-scale data augmentation and is suitable for data generation for industrial-grade training tasks. Attached Figure Description
[0038] Figure 1 This is a flowchart of the few-sample image synthesis method based on rotation target perception decoupling described in this invention;
[0039] Figure 2 This is a schematic diagram of rotating target detection and mask generation.
[0040] Figure 3 Standardize the process for texture extraction and perspective;
[0041] Figure 4 This is a schematic diagram illustrating the principles of Poisson blending and mask mixing.
[0042] Figure 5 Background enhancement and perturbation synthesis structure;
[0043] Figure 6 Example of generated results for the sample;
[0044] Figure 7 This is a schematic diagram illustrating the system module composition and data flow.
[0045] Figure 8 This describes the system's parallel processing architecture and task allocation process. Detailed Implementation
[0046] The core of the technical solution of this invention to solve the above-mentioned technical problems is as follows: First, the target foreground and complex background in the image are accurately perceived and separated by rotation target detection technology; then, the target texture is subjected to perspective normalization processing to form a unified standard texture block; then, the normalized texture is mapped back to the original target area with high precision through homography transformation; at the same time, the original background is enhanced by multi-dimensional perturbation to enrich the scene diversity; finally, the annotation data that is strictly aligned with the synthesized image at the pixel level is automatically output. The whole process is completed efficiently through a parallel architecture, so that a large number of high-quality, highly diverse and fully annotated training samples can be generated in batches using a very small number of seed images, which significantly improves the applicability and robustness of the downstream visual model in low-sample scenarios.
[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0048] like Figure 1 As shown, the few-sample image synthesis method based on rotation target perception decoupling of the present invention includes the following steps:
[0049] Step S1: As Figure 2 As shown, rotating target detection is performed on the input image, and a foreground mask and an image background mask are generated based on the target region and background region of the image.
[0050] (1) Rotation detection priority strategy: The system prioritizes calling the rotation target detection model for each input image (such as metal screws, PCB boards and other industrial parts) in the dataset to be augmented to obtain candidate box parameters. ,in To rotate the geometric center or centroid of the target bounding box, and These are the width and height of the rotated target bounding box, respectively. The rotation angle (in radians) of the target box relative to the horizontal axis.
[0051] To ensure geometric consistency and prevent failure during subsequent perspective transformations, the vertices of the predicted rotated target box are stably sorted clockwise from the top left corner according to their polar angles. .
[0052] The four vertices of the rotated target box Stable sorting includes:
[0053] Calculate the centroid or geometric center of the rotated target box vertex: ;
[0054] Calculate the polar angles of the four vertices relative to the centroid. With radius ;
[0055] Normalize the angle to : ;
[0056] Sort by angle with a robust criterion: First sort in ascending order (resulting in counter-clockwise order). If the angles are nearly equal, use the radius as the secondary key in descending order (farther angles come later). ;
[0057] To determine whether a direction is clockwise or counterclockwise: use the directed area of a polygon to determine the current direction. , ;like It is counterclockwise. It is clockwise.
[0058] (2) Mask generation: Generate the target foreground mask and background mask (binary map) based on the four-point polygon, and write the metadata to ensure the repeatability of the subsequent homography matrix solution.
[0059] (3) Back-down parsing mechanism: When the detection model is unavailable, the threshold is insufficient, or the NMS is empty, the system automatically back-down parses the general annotations: supports VOC (XML), COCO (PolygonJSON) and custom JSON / CSV vertex formats, and directly restores the target polygon and mask.
[0060] (4) Quality threshold: If the minimum side of the polygon is too short or the convexity test fails, a “re-detection / re-analysis” is triggered once; if the failure still occurs, the convex hull is corrected or the rectangle is used as an approximation.
[0061] Specifically, in this embodiment, when the input image resolution is between 640 and 2048, rotation detection uses a lightweight inference configuration (FP16, Patch_size=1). Four-point sorting uses a dual-key sorting of "polar angle-distance" to prevent swapping caused by near-collinearity. The output foreground and background masks maintain the same size as the original image, with pixel values of 255 and 0 respectively, to ensure alignment with subsequent texture mapping and background enhancement. Furthermore, the detection confidence is written into the metadata table (JSON / CSV).
[0062] Step S2: Construct a texture library, perform perspective standardization processing on the images in the texture library, and generate standard texture blocks.
[0063] like Figure 3 As shown, candidate images (such as metal scratches and textile textures) in the texture library dataset are used to generate standard texture blocks according to the following process:
[0064] (1) Texture library detection and cropping: Rotation detection is performed on the texture library image first; if the target is detected, the distance from the four points of the target to the reference rectangle is calculated. homography matrix The target block perspective is normalized using bilinear interpolation. Standard texture (default) ).
[0065] (2) Failure rollback: When the detection fails, first parse the texture and clip the attached annotations; if it is still unusable, use the whole image as the texture sample and clip and scale it as needed in the subsequent mapping stage.
[0066] (3) Appearance normalization: Perform brightness and color alignment (histogram matching or mean-variance alignment) on standard textures and record the "normalization scheme ID".
[0067] (4) Indexing and storage: A texture index is created using “Category-Attitude Angle-Scale-Material” as the key to accelerate subsequent retrieval.
[0068] Specifically, in this embodiment, when the texture region scale is smaller than... At that time, pyramid magnification and anti-aliasing were used. To suppress high-frequency ringing, a 1–2 pixel pre-blurring was applied to the edges before projection.
[0069] Step S3: Map the standard texture block to the foreground mask region through homography transformation, and use an adaptive fusion algorithm to fuse the image texture mapped to the foreground mask region with the original image background.
[0070] (1) Homography: Using the four-point polygon output in step S1 as the object points, calculate the homography matrix from the standard texture to the four target points. It performs back projection of the standard texture onto the original image coordinate system to achieve geometrically aligned target texture replacement.
[0071] (2) For example Figure 4 As shown, based on the local contrast difference, color difference, and texture spectrum difference between the target area and the background area, the Poisson fusion or mask blending algorithm is adaptively selected for boundary fusion.
[0072] Poisson Blending: Using the target boundary as the Dirichlet condition, the Poisson equation is solved to balance the gradient and eliminate the brightness step at the seam.
[0073] Mask blending: Performs multi-scale / Gaussian smoothing on the blended mask to achieve fast synthesis with softened edges.
[0074] (3) Adaptive selection: Dynamically select the fusion mode based on the local contrast difference, color difference and texture spectrum difference; when the contrast difference is greater than the threshold or the edge spectrum difference is large, Poisson fusion is given priority, otherwise mask blending is used to improve the system execution efficiency.
[0075] Specifically, in this embodiment: if the color temperature difference between the target and the background is >150K (estimated value), white balance alignment is performed first, followed by fusion. The Poisson fusion solution is accelerated using a multi-grid approach; to maintain label alignment, the parameters of the four points of the replaced target and the rotated target bounding box remain unchanged and are directly used as training labels for output.
[0076] Step S4: Enhance the original background to obtain a composite image.
[0077] like Figure 5 As shown, for the area outside the background mask, background enhancement is performed using one or more of the following methods: illumination direction and shadow intensity, fog coefficient, color shift gain, and random occlusion area ratio and position. During the background enhancement process, the shadow kernel size, fog coefficient, color shift gain, occlusion ratio, etc., are all recorded and written into the metadata table for easy traceability and reproducibility during the training phase.
[0078] Lighting and shadow simulation: Estimate the direction of global or local light sources, generate projected shadows using depth approximation or normal assumptions; superimpose soft shadows in the direction consistent with the target pose.
[0079] Environmental element overlay: Extract textures such as roads, metal / vegetation / pipelines from the scene library and apply them to the background surface in an affine or perspective manner, and perform alpha compositing when necessary.
[0080] Atmospheric and color disturbances: Simulating haze, color cast, exposure drift, and particle noise.
[0081] Random occlusion and reflection: Inject semi-transparent occlusion or specular highlights to control the proportion and position distribution of the occlusion area.
[0082] Specifically, in this implementation: the direction of illumination is from... Uniform sampling, shadow intensity From sampling. Atomization coefficient. Color bias gain The area obscured occupies a large portion of the background. The positions follow a block-uniform distribution.
[0083] Step S5: Obtain the synthesized image and its annotation information.
[0084] The output includes: such as Figure 6The synthesized image shown is (PNG / JPEG / TIFF-16bit); the label file contains: rotated bounding box and four-point polygon, category ID; the fusion mask (for quality inspection and visualization); and the augmented meta-information table (JSON / CSV) which records the detection threshold, vertex sorting strategy, fusion mode and weights, background perturbation parameters, etc.
[0085] Batch-level configuration: minimum / maximum number of composites for each target type, background type ratio, etc., to ensure category balance and sufficient attitude coverage.
[0086] Data consistency: Ensuring that the labels are aligned with the generated image objects is one of the key advantages of this invention compared to end-to-end generative methods.
[0087] This invention also provides a few-shot image synthesis system based on rotation target perception decoupling, such as... Figure 7 As shown, the system includes:
[0088] The rotating target detection and parsing module obtains the rotation bounding box parameters of the target region in the input image, and generates a foreground mask and an image background mask based on the target region and background region of the image.
[0089] The texture library construction and standardization module builds a texture library, performs perspective standardization processing on the images in the texture library, and generates standard texture blocks.
[0090] The texture perspective mapping and fusion module is used to map the standard texture block to the foreground mask area and fuse the image texture mapped to the foreground mask area with the original image background;
[0091] The background scene enhancement module is used to enhance the original background of the background mask area;
[0092] Composite Image Output Module: Used to output composite images.
[0093] Preferably, the system further includes a parallel processing module and a metadata recording module, such as... Figure 8 As shown, the system employs a parallel processing architecture and task allocation based on the parallel processing module during execution:
[0094] Master-slave architecture: It adopts a master process + process pool; the master process is responsible for task queue, priority and throughput statistics; the child processes independently execute rotation detection, homography mapping, fusion and background enhancement steps to avoid repeated initialization.
[0095] Shared cache: Standard texture blocks, masks, and intermediate features are reused across processes via shared memory / memory mapping; texture indexes reside in memory to reduce I / O.
[0096] Fault tolerance strategy: When a child process exits abnormally, the main controller automatically restarts and replays the unfinished task; "failure reason code" is recorded for failed images to facilitate quality backtracking.
[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A few-sample image synthesis method based on rotation target perception decoupling, characterized in that, The method includes: Step S1: Perform rotational target detection on the input image, and generate a foreground mask and an image background mask based on the target region and background region of the image; Step S2: Construct a texture library, perform perspective standardization processing on the images in the texture library, and generate standard texture blocks; Step S3: Map the standard texture block to the foreground mask region through homography transformation, and use an adaptive fusion algorithm to fuse the image texture mapped to the foreground mask region with the original image background; Step S4: Enhance the original background to obtain a composite image.
2. The few-sample image synthesis method based on rotational target perception decoupling according to claim 1, characterized in that, Step S1 includes: Call the rotated object detection model to obtain the rotated bounding box parameters of the input image. ,in To rotate the geometric center or centroid of the target bounding box, and These are the width and height of the rotated target bounding box, respectively. This represents the rotation angle of the target bounding box relative to the horizontal axis. The four vertices of the rotating target box are obtained based on the parameters of the rotating bounding box. The four vertices of the rotating box are sorted clockwise from the top left corner according to the polar angle to generate an ordered four-point polygon. The foreground mask and background mask are generated based on the ordered four-point polygon.
3. The few-sample image synthesis method based on rotational target perception decoupling according to claim 2, characterized in that, The generation of the ordered four-point polygon specifically includes: Based on the polar angles of the four vertices of the rotated target box relative to the center or centroid and radius ;Will Normalization to Obtain the normalized polar angle ; Sort the vertices according to their normalized polar angle and radius. And calculate the area of the polygon. , The sorting direction is determined by the area of the polygon. It is counterclockwise. It is clockwise.
4. The few-sample image synthesis method based on rotational target perception decoupling according to claim 1, characterized in that, If the target rotation detection fails or the confidence level is below the threshold in step S1, the target polygon is restored and a foreground mask and a background mask are generated using VOC, COCO or a custom JSON / CSV polygon vertex format.
5. The few-sample image synthesis method based on rotational target perception decoupling according to claim 1, characterized in that, Step S2 includes: performing rotation detection on the texture image; if a target is detected, calculating the homography matrix of the four points of the target to the standard reference quadrilateral, and normalizing the perspective of the texture image to a standard texture block through bilinear interpolation.
6. The few-sample image synthesis method based on rotational target perception decoupling according to claim 5, characterized in that, Step S2 also includes histogram matching of color and brightness or mean-variance alignment of the standardized texture blocks.
7. The few-sample image synthesis method based on rotational target perception decoupling according to claim 1, characterized in that, Step S3 includes: Calculate the homography matrix from the standard texture block to the ordered four-point polygon, and perform perspective deformation of the standard texture block to the target area through reverse mapping to replace the foreground mask; Based on the local contrast difference, color difference, and texture spectrum difference between the target area and the background area, the Poisson fusion or mask blending algorithm is adaptively selected for boundary fusion.
8. The few-sample image synthesis method based on rotational target perception decoupling according to claim 7, characterized in that, When the contrast difference, color difference, or texture spectrum difference is greater than the set threshold, Poisson blending is preferred; otherwise, mask blending algorithm is preferred for boundary blending.
9. The few-sample image synthesis method based on rotational target perception decoupling according to claim 1, characterized in that, In step S4, background enhancement is performed using one or more of the following methods: lighting direction and shadow intensity, fogging coefficient, color shift gain, and random occlusion area ratio and position.
10. A few-shot image synthesis system based on rotational target perception decoupling, used to implement the few-shot image synthesis method based on rotational target perception decoupling as described in any one of claims 1-9, characterized in that, The system includes: The rotating target detection and parsing module obtains the rotation bounding box parameters of the target region in the input image, and generates a foreground mask and an image background mask based on the target region and background region of the image. The texture library construction and standardization module builds a texture library, performs perspective standardization processing on the images in the texture library, and generates standard texture blocks. The texture perspective mapping and fusion module is used to map the standard texture block to the foreground mask area and fuse the image texture mapped to the foreground mask area with the original image background; The background scene enhancement module is used to enhance the original background of the background mask area; Composite Image Output Module: Used to output composite images.