Method and system for generating a level of transparency of a guiding object for detection

Abstract

This invention discloses a method and system for generating object-level transparent layers using detection guidance. The method includes: acquiring a single input image; inputting the single input image into a detection layer model to identify and separate each visual element to form independent bounding boxes; inputting the object layer set into an improved multimodal large model for decomposition into multiple editable RGBA transparent layers; including: reconstructing the object image layer based on specific conditions using a conditionally gated LoRa attention mechanism, and automatically optimizing the separation of foreground and background and edge smoothing through a transparency decoder to generate multiple transparent layers with natural transparency transitions; and outputting the decomposition results. By implementing the method of this invention, multiple independent layers containing complete object information and high-quality transparent boundaries can be efficiently and accurately extracted from a single input image, addressing the difficulty of existing image processing technologies in meeting the needs for independent reconstruction and fine editing of each element of an image.

Description

Technical Field

[0001] This invention relates to artificial intelligence, and more specifically to a method and system for detecting and generating transparent layers at the level of guided objects. Background Technology

[0002] With the rapid advancement of AI-generated content technology, image generation and editing tools based on advanced generative models have been widely applied in digital content creation, visual design, and image processing. These technologies have greatly improved work efficiency and given users unprecedented creative freedom. However, in practical applications, especially when it is necessary to independently reconstruct, finely edit, and flexibly recombine different visual elements in a single image, existing methods still face many challenges. Particularly in complex scenarios, users place higher demands on the structural integrity, boundary clarity, and transparency representation capabilities of various image elements.

[0003] Current technologies mainly focus on semantic segmentation, instance segmentation, and image matting. While these methods can identify and extract target regions in images to some extent, they generally suffer from the following problems: First, insufficient occlusion handling capabilities; existing segmentation techniques often only annotate the visible parts of the image, failing to provide complete structural information for occluded areas. Second, inaccurate transparency modeling; although image matting techniques can generate foreground layers with alpha channels, jagged edges or irregularities often appear at the boundaries. Finally, text-driven generation has limitations; methods relying on text descriptions to guide image generation are prone to result deviations due to unclear descriptions, making it difficult to meet professional-level design requirements.

[0004] Therefore, it is necessary to design a new method to efficiently and accurately extract multiple independent layers containing complete object information and high-quality transparent boundaries from a single input image, in order to address the shortcomings of existing image processing technologies in occlusion handling, transparency modeling, and text-driven generation, which make it difficult to meet the needs of independent reconstruction and fine editing of each element of the image. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for detecting and generating transparent layers at the level of guided objects.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting the generation of a transparent layer at the guide object level, comprising:

[0007] Acquire a single input image; The single input image is input into the detection hierarchical model to identify and separate each visual element to form an independent bounding box, thus obtaining an object layer set; The object layer set is input into an improved multimodal large model to decompose multiple editable RGBA transparent layers to obtain the decomposition results; including: reconstructing the object image layer based on specific conditions using a conditionally gated Lora attention mechanism, and automatically optimizing the separation of foreground and background and edge smoothing through a transparency decoder to generate multiple transparent layers with natural transparency transitions; Output the decomposition results.

[0008] The further technical solution is as follows: The detection layering model is constructed by building a dataset for image element layering and training the model under the Detectron2 framework. During the training process, element separability and spatial independence are the main judgment criteria. Various visual elements in a single input image are uniformly labeled and learned, so that the model can stably output a set of candidate bounding boxes with comprehensive coverage and reasonable boundaries in complex scenes.

[0009] The further technical solution is as follows: the single input image is input into the detection layer model to identify and separate each visual element to form an independent bounding box, resulting in an object layer set, including: The single input image is input into the detection hierarchical model, and multi-scale feature maps are generated through the backbone network to form a feature pyramid; The region proposal network is used to predict multiple candidate regions and their confidence scores based on the feature pyramid to obtain the first candidate region; The first candidate region is filtered based on a confidence threshold to obtain the second candidate region; Perform a region alignment operation on the second candidate region, and based on the aligned candidate region, crop the corresponding part from the single input image as an independent object layer, integrate all object layers into a set to obtain an object layer set, wherein the object layers in the object layer set are all processed according to the timeline.

[0010] The further technical solution is as follows: the condition-gated LoRa attention mechanism includes parallel conditional branching and structured modulation mechanism for instance-level conditional modeling. The condition-gated LoRa attention mechanism embeds structured gating constraints into the low-rank adaptive attention calculation of the improved multimodal large model through parallel conditional branching.

[0011] Its further technical solution is: the object image layer reconstructed based on specific conditions using the conditionally gated LoRa attention mechanism includes: The object layer set is divided into a denoising marker sequence and a conditional image coding sequence to obtain the partitioning result; Design a spatial gating function to activate the conditional subspace of the partitioning result using a binary mask, so as to obtain the activation result; The activation results are applied to a pre-trained standard projection formula to preserve the original parameterization and representation semantics, and the conditional branches are adaptively processed using low-rank parameterization techniques to adjust the scaling factor in order to obtain the processing result. The query, key, and value projections corresponding to the processing results are concatenated along the sequence dimension to form the final input result; The attention output is calculated based on the final input result, and rotational position embedding is applied to maintain spatial alignment in order to obtain denoising components and conditional components to reconstruct the object image layer.

[0012] The further technical solution is as follows: The separation of the foreground and background and edge smoothing are automatically optimized through a transparent decoder to generate multiple transparent layers with natural transparency transitions, including: Pixel brightness evaluation, color difference analysis, and saturation detection are performed on the object image layer to obtain multi-dimensional analysis results; Pixels that meet the requirements of the multi-dimensional analysis results are marked as background to form an initial background mask; By processing the initial background mask through closing and opening operations, holes and noise are removed to obtain a continuous and smooth background mask; The continuous smooth background mask is converted into a binary alpha image, and the foreground transparency and background transparency are defined to obtain the boundary region identifier between the foreground and the background. Based on the background mask, the edge transition region between the foreground and the background is located, and Gaussian filtering is applied only to the corresponding region to smooth the transparency transition, so as to obtain the processed transparency channel. By combining the processed transparency channel with the RGB channel corresponding to the single input image, multiple transparent layers with natural transparency transitions are generated.

[0013] The further technical solution is as follows: The pixel brightness evaluation, color difference analysis, and saturation detection are performed on the object image layer to obtain multi-dimensional analysis results, including: The average intensity of the RGB value of each pixel in the object image layer is calculated. If the average intensity is within the set brightness range, the pixel is determined to conform to the background features in terms of brightness. For pixels that match the background characteristics in terms of brightness, the standard deviation between the RGB channels is calculated. If the standard deviation is lower than a set threshold, the pixel color is distinguished from the colored area in the foreground to obtain the selected pixels. For the selected pixels, check whether the color saturation does not exceed the saturation threshold. If the color saturation does not exceed the saturation threshold, then the multi-dimensional analysis result is determined to meet the requirements.

[0014] The further technical solution is as follows: the initial background mask is processed by closing and opening operations to remove holes and noise, resulting in a continuous and smooth background mask, including: The initial background mask is closed using a predefined structuring element to fill holes and breaks, and then an opening operation is performed to remove noise and artifacts, resulting in a continuous and smooth background mask.

[0015] The further technical solution is as follows: Converting the continuous, smooth background mask into a binary alpha image, defining foreground transparency and background transparency to obtain the boundary region identifier between the foreground and background, includes: Convert a continuous, smooth background mask into a binary alpha image.

[0016] This invention also provides a system for detecting the generation of transparent layers at the level of guided objects, comprising: The acquisition unit is used to acquire a single input image; The detection layer unit is used to input the single input image into the detection layer model to identify and separate each visual element to form an independent bounding box, thereby obtaining an object layer set; A transparent layer generation unit is used to input the object layer set into an improved multimodal large model to decompose multiple editable RGBA transparent layers to obtain decomposition results; including: reconstructing the object image layer based on specific conditions using a conditionally gated Lora attention mechanism, and automatically optimizing the separation of foreground and background and edge smoothing through a transparent decoder to generate multiple transparent layers with natural transparency transitions; The output unit is used to output the decomposition results.

[0017] The advantages of this invention compared to existing technologies are as follows: This invention first feeds a single input image into a detection layer model to identify and separate various visual elements, forming a set of object layers with independent bounding boxes. Then, using an improved multimodal large model combined with a conditionally gated LoRa attention mechanism, it reconstructs the object image layer for each object layer based on specific conditions. Furthermore, it optimizes foreground-background separation and edge smoothing through a transparency decoder. This allows for the efficient and accurate extraction of multiple independent layers containing complete object information and high-quality transparent boundaries from a single input image. This effectively addresses the shortcomings of existing technologies in handling occlusion, transparency modeling, and text-driven generation, meeting the needs for independent reconstruction and fine-tuning of image elements. This method not only improves the quality and accuracy of image decomposition but also enhances the natural transition effect of the generated results, providing strong support for advanced image editing.

[0018] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the method for generating a transparent layer at the level of a detection guide object provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the architecture of the method for generating a transparent layer at the level of a detection guide object provided in an embodiment of the present invention; Figure 3 A schematic diagram of the conditionally gated LoRa attention mechanism provided in an embodiment of the present invention; Figure 4 The comparison diagram of model effects provided in the embodiments of the present invention; Figure 5 A schematic block diagram of a detection and guidance object-level transparent layer generation system provided in an embodiment of the present invention; Figure 6 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] Please see Figure 1 , Figure 1 This is a flowchart illustrating the object-level transparent layer generation method provided in this embodiment of the invention. This method is applied in a server. It acquires a single input image and uses a detection layering model to identify and separate visual elements to form independent bounding boxes. Then, it uses an improved multimodal large model and a conditionally gated LoRa attention mechanism to reconstruct these objects based on specific conditions, optimizing foreground-background separation and edge smoothing, thereby efficiently and accurately generating multiple independent layers with high-quality transparent boundaries. This process particularly emphasizes pixel brightness evaluation, color difference analysis, and saturation detection to determine background pixels, and uses a series of image processing techniques (such as closing and opening operations) to ensure the continuity and smoothness of the background mask. Finally, it combines the processed transparency channel and RGB channel to output the ideal transparent layer result. This method effectively solves the shortcomings of existing technologies in occlusion handling, transparency modeling, and text-driven generation, meeting the needs for independent reconstruction and fine editing of each element of an image.

[0026] Figure 1 This is a flowchart illustrating the method for generating a transparent layer at the level of a guided object, as provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S140.

[0027] S110. Obtain a single input image.

[0028] In this embodiment, a single input image refers to a single, independent digital image file that serves as the starting point for processing. This file contains one or more visual elements (e.g., foreground objects, background, decorative elements, etc.), which may be interwoven or partially occluded in a complex manner. This image is typically a photograph taken in the real world or a computer-generated image with a certain resolution and color depth (e.g., RGB color space). Specific requirements for the input image include, but are not limited to: Resolution: The input image should have sufficient resolution to ensure the integrity of detailed information, facilitating subsequent detection, segmentation, and reconstruction processes. Experimental results show that a resolution of 512x512 pixels achieves superior results in terms of structural consistency, pixel reconstruction accuracy, and perceptual quality, while the computational cost remains within an acceptable range.

[0029] Format: Supports common image formats such as JPEG and PNG to accommodate data input from different sources.

[0030] Content: The input image can contain natural scenes, man-made objects, portraits, etc., covering a wide range of visual themes. The key is that each visual element in the image should be clearly distinguishable; even with some degree of occlusion, effective separation and reconstruction can be achieved using the method proposed in this invention.

[0031] Step S110 marks the beginning of the entire "detection-generation-decoding" process. The main task of this stage is to prepare the image data to be processed, bringing it to a state suitable for further analysis and processing. This step, seemingly simple, is crucial for the successful execution of all subsequent steps, as it directly relates to the ability to accurately identify and separate each visual element in the image, thereby generating a high-quality object-level transparency layer. Furthermore, considering the diversity of real-world application scenarios, input images may have different sizes, proportions, and content complexities. Therefore, after acquiring the input image, necessary preprocessing is often required, such as resizing, cropping, or normalization, to meet the requirements of specific algorithms.

[0032] S120. Input the single input image into the detection layer model to identify and separate each visual element to form an independent bounding box, thereby obtaining an object layer set.

[0033] In this embodiment, the object layer set refers to a group of independent object layers generated by identifying, locating, and segmenting each separable visual element in a single input image. Each object layer corresponds to a specific visual entity (such as a foreground object, decorative element, etc.), and these object layers are defined by independent bounding boxes, ensuring spatial independence and semantic integrity between each object layer. The object layer set provides basic data support for subsequent reconstruction and transparency decoding, enabling each object to be processed and optimized individually, ultimately achieving the generation of high-quality transparency layers.

[0034] Specifically, the detection layering model is constructed by building a dataset oriented towards image element layering and training the model under the Detectron2 framework. During the training process, element separability and spatial independence are the main judgment criteria. Various visual elements in a single input image are uniformly labeled and learned, so that the model can stably output a set of candidate bounding boxes with comprehensive coverage and reasonable boundaries in complex scenes.

[0035] To meet the specific requirements of the "full-element detection and hierarchical analysis" task, we first constructed our own dataset for image element hierarchical analysis. This dataset includes various types of visual elements, covering everything from foreground objects to decorative elements and even semantically ambiguous but editably independent regions. Every visual element in each image is precisely labeled with its bounding box and related attributes (such as category, confidence, etc.). This meticulous annotation not only covers typical target objects but also includes atypical objects and decorative elements that are easily overlooked in general object detection models.

[0036] The model was specifically trained using the Detectron2 framework. Detectron2 is a highly flexible and powerful deep learning platform, particularly suitable for computer vision tasks. We chose Detectron2 as our training framework primarily because it supports multi-scale feature extraction, a flexible object detection architecture, and an efficient training process—features crucial for achieving comprehensive and accurate image element detection.

[0037] During training, we primarily focused on two core metrics: element separability and spatial independence. These two metrics ensured that the model could effectively identify and distinguish various visual elements in an image and generate a reasonable bounding box for each element.

[0038] Element separability refers to the model's ability to correctly identify visual elements in an image that can exist independently. This involves the accurate segmentation and classification of different visual entities in an image, especially those elements with irregular shapes or ambiguous semantic definitions.

[0039] Spatial independence: This emphasizes that each detected visual element should occupy its own unique spatial position, avoiding overlap or confusion with other elements. This is crucial for the integrity and clarity of subsequent layer structures.

[0040] By uniformly labeling each visual element in the dataset (including the position, size, and category of the bounding box), the model can stably output a comprehensive and reasonably bounded set of candidate bounding boxes in complex scenes. This labeling method not only improves the model's learning efficiency but also enhances its performance when faced with diverse and highly uncertain image content.

[0041] Ultimately, through the meticulously designed training process described above, the detection layer model can accurately detect all visual elements in any input RGB image and organize them into independent bounding boxes. This provides a solid foundation for the subsequent construction of the object layer and also greatly improves the robustness and adaptability of the entire system.

[0042] In summary, by constructing a specialized dataset and training it specifically within the Detectron2 framework, the proposed hierarchical detection model effectively overcomes the limitations of existing technologies, achieving comprehensive and accurate detection of various visual elements in a single input image. This achievement not only advances image processing technology but also provides users with a richer and more flexible image editing experience.

[0043] In one embodiment, step S120 described above may include steps S121 to S124.

[0044] S121. Input the single input image into the detection hierarchical model, and generate multi-scale feature maps through the backbone network to form a feature pyramid.

[0045] In this embodiment, the feature pyramid refers to a multi-level feature representation method, which consists of multiple feature maps of different resolutions. Each layer of the feature map captures information at different scales of the input image. Specifically, the feature pyramid extracts multi-scale information of the input image through a backbone network, where each level represents a feature map at a specific spatial resolution. These feature maps jointly encode high-level semantic information and fine-grained details, thereby providing rich feature descriptions for subsequent region proposal networks.

[0046] S122. Using a region proposal network, multiple candidate regions and confidence scores are predicted based on the feature pyramid to obtain the first candidate region.

[0047] In this embodiment, the first candidate region refers to a set of potential target regions predicted by the Region Proposal Network (RPN) based on the feature pyramid, along with their corresponding confidence scores. These candidate regions represent areas that may contain complete, separable visual entities, and the confidence scores are used to measure the probability that each candidate region contains the target object. Through this process, regions that may contain objects of interest can be preliminarily screened.

[0048] S123. Filter the first candidate region according to the confidence threshold to obtain the second candidate region.

[0049] In this embodiment, the second candidate region refers to the set of candidate regions filtered by a confidence threshold. In this step, the system removes candidate regions with low confidence based on a preset confidence threshold, retaining only high-confidence regions for subsequent processing. This effectively reduces the impact of overlapping areas and background noise on the layer construction process, improving the quality and accuracy of the object layer.

[0050] S124. Perform a region alignment operation on the second candidate region, and based on the aligned candidate region, crop the corresponding part from the single input image as an independent object layer, integrate all object layers into a set to obtain an object layer set, wherein the object layers in the object layer set are all processed according to the timeline.

[0051] In this step, the RoI Align operation is first performed on the second candidate region to ensure spatial consistency among candidate regions at different scales. Then, based on the aligned candidate regions, the corresponding image regions are directly cropped from the original input image and defined as independent object layers. Each object layer contains not only the visible appearance information of the object but also its transparent boundary regions with the background or other objects. Finally, all detected and cropped object layers are organized into a set, i.e., the object layer set. To optimize computational efficiency and avoid mutual interference, each object layer is processed sequentially according to its own timeline. This not only improves the overall system stability and usability but also ensures that each object layer achieves the best reconstruction results.

[0052] The purpose of step S120 is to maximize element coverage and hierarchical integrity, that is, to ensure that foreground objects, decorative elements, and independently editable visual entities in the image can be stably detected and mapped to a unique object layer. Existing general object detection models based on Detectron2 are usually trained around a limited set of categories, and their optimization focuses on category discrimination accuracy and detection recall. However, in image layering applications, these models tend to ignore atypical objects, decorative elements, or semantically ambiguous but still independent visual regions in terms of editing semantics, resulting in some elements not being detected or being incorrectly merged into a single object layer, thus affecting the integrity of the subsequent layer structure. In this embodiment, the method is designed for the specific task of "full element detection and layering," and a dataset for image element layering is constructed. The detection layering module is specifically trained within the Detectron2 framework. During training, element separability and spatial independence are used as the main criteria to uniformly label and learn various visual elements in the image, enabling the detection model to stably output a set of candidate bounding boxes with comprehensive coverage and reasonable boundaries in complex scenes, thereby providing a consistent and reliable spatial partitioning foundation for the subsequent object layer construction.

[0053] Given an input RGB image The backbone network extracts the feature pyramid. To capture multi-scale information, where each pyramid level These feature maps represent a specific spatial resolution. They jointly encode high-level semantics and fine-grained details. Subsequently, the Region Proposal Network (RPN) predicts a set of... There are candidate regions, and the formula is as follows: ;in Indicates the first The rectangular bounding boxes of each candidate region in the input image coordinate system. The object confidence score for a given region measures whether the region contains a complete, separable visual entity. In practice, candidate regions are filtered based on a confidence threshold, retaining only high-confidence regions for subsequent steps to reduce the impact of overlapping areas and background noise on the layer construction process.

[0054] Since different candidate regions may originate from feature pyramid layers of different scales, in order to ensure the spatial consistency of regional features, the system performs the RoI Align operation on each retained candidate region.

[0055] Specifically, for each candidate bounding box The steps of RoI Align are as follows: Based on the position of the bounding box in the input image, determine its corresponding feature pyramid layer. The corresponding region is cropped from the feature map; the cropped region is mapped to a feature representation of a fixed size through continuous interpolation.

[0056] This process avoids the coordinate offset problem caused by discrete quantization, thus ensuring that the extracted regional features are spatially aligned precisely with the object region in the original image, unaffected by changes in object scale and aspect ratio.

[0057] After completing the detection and alignment of candidate regions, the system determines the candidate bounding box based on each candidate bounding box. Directly from the original input image The corresponding image region is cropped and defined as an independent object layer: ;in , Through this cropping operation, each object layer fully preserves the visible appearance information of the object in pixel space, including its boundary shape and transparent boundary regions with the background or other objects. It is important to emphasize that this object layer is not a simple image patch, but rather an image representation unit semantically corresponding to a single visual entity, providing explicit object-level input for subsequent transparency layer reconstruction and generation processes. Finally, all detected and cropped object layers are uniformly organized into a single set: In subsequent processing: each object layer Modeling and generation are performed as independent units; different object layers are independent of each other in terms of spatial location and semantic content; in order to reduce the influence of other element image layers on the current element image layer and to generate unnecessary computation, all object layers enjoy a diffusion timeline one by one.

[0058] This not only improves the reconstruction quality of each object layer, but also ensures the efficiency and accuracy of the entire image decomposition process.

[0059] S130. Input the object layer set into the improved multimodal large model to decompose multiple editable RGBA transparent layers to obtain the decomposition result; including: using the conditionally gated Lora attention mechanism to reconstruct the object image layer based on specific conditions, and automatically optimizing the separation of foreground and background and edge smoothing through the transparency decoder to generate multiple transparent layers with natural transparency transitions.

[0060] In this embodiment, the decomposition result refers to the multiple RGBA layers with natural transparency transitions and independent editability generated after processing by the improved multimodal large model and transparent decoder.

[0061] Step S130 involves inputting the object layer set into an improved multimodal large model to decompose multiple editable RGBA transparency layers to obtain the decomposition results. This process includes reconstructing the object image layers based on specific conditions using a conditionally gated Lora attention mechanism, and automatically optimizing the separation of foreground and background and edge smoothing through a transparency decoder to generate transparent layers with natural transparency transitions.

[0062] Specifically, the condition-gated LoRa attention mechanism includes parallel conditional branching and structured modulation mechanisms for instance-level conditional modeling. The condition-gated LoRa attention mechanism embeds structured gating constraints into the low-rank adaptive attention computation of the improved multimodal large model through parallel conditional branching.

[0063] In one embodiment, the object image layer is reconstructed based on specific conditions using a conditionally gated Lora attention mechanism, including steps S131 to S135.

[0064] S131. The object layer set is divided into a denoising marker sequence and a conditional image coding sequence to obtain the division result.

[0065] In this embodiment, the partitioning result refers to dividing the object layer set into a denoised label sequence and a conditional image coding sequence. The denoised label sequence contains text information and noise latent variable encoding, while the conditional image coding sequence corresponds to specific visual conditions.

[0066] S132. Design a spatial gating function to activate the conditional subspace of the partitioning result using a binary mask to obtain the activation result.

[0067] In this embodiment, the activation result refers to the design of the spatial gating function. This function selectively activates the binary mask M within the conditional subspace according to preset rules, ensuring that conditional adaptive processing is applied only in the specified region.

[0068] S133. Apply the pre-trained standard projection formula to the activation result to preserve the original parameterization and representation semantics, and use low-rank parameterization technology to adaptively process the conditional branches, adjusting the scaling factor to obtain the processing result.

[0069] In this embodiment, the processing result refers to first using the standard projection formula to preserve the original parameterization and representation semantics, and then using low-rank parameterization technology to perform efficient adaptive processing on the conditional branches, adjusting the scaling factor λ to control the degree of influence of the adaptive components.

[0070] S134. Concatenate the query, key, and value projections corresponding to the processing results along the sequence dimension to form the final input result.

[0071] In this embodiment, the final input result refers to the concatenation of the processed query (Q'), key (K'), and value (V') projected onto the sequence dimension to form a complete input result for subsequent attention calculation.

[0072] S135. Calculate the attention output based on the final input result, and apply rotational position embedding to maintain spatial alignment to obtain denoising components and conditional components to reconstruct the object image layer.

[0073] In this embodiment, the noise reduction component ( This part of the output is mainly used to guide the diffusion decoding path and help restore image details.

[0074] Conditional components ( ): This can be used to further refine or enhance the generation effect in auxiliary modules.

[0075] Object image layer ( The final generated object image layer containing foreground content is overlaid on a uniform neutral gray background.

[0076] After obtaining the object image layer, the transparency decoder is responsible for converting it into an RGBA layer with an alpha channel. This involves: Pixel color consistency and brightness feature analysis: Identify background regions and improve the quality of the background mask through morphological reconstruction operations.

[0077] Local adaptive transparency refinement strategy: Create a continuous, smooth transparency transition at the foreground edge to avoid jagged edges and unnecessary blurring.

[0078] The decomposition result refers to multiple RGBA layers with natural transparency transitions obtained after all the above steps. These layers can be edited individually, combined, or further processed, greatly improving the flexibility and precision of image editing. Each layer not only contains RGB color information but also has a precise alpha channel, ensuring clear boundaries and natural transitions between image elements.

[0079] In this embodiment, as Figure 3 As shown, this design is based on the Flux.1-dev multimodal generative large-scale model. This model, with a diffusion generation mechanism at its core, enables joint modeling of multi-source information such as images, text, and spatial conditions. It also completes cross-modal feature interaction and high-fidelity image generation within a unified attention framework. Through large-scale pre-training, Flux.1-dev acquires powerful capabilities in handling visual structure, semantic relationships, and generation consistency. It can stably reconstruct and reasonably complete target image content under given constraints, making it suitable for object-level generation and editing tasks in complex scenes.

[0080] Nevertheless, the original architecture of Flux.1-dev focuses more on global image generation tasks, and its attention modeling mechanism is difficult to directly meet specific needs such as object-level reconstruction, fine-grained structure control, and occlusion region inference. Therefore, while maintaining the parameters and basic representation capabilities of the Flux.1-dev pre-trained model, this invention specifically extends its attention structure by introducing parallel conditional branching and structured modulation mechanisms for instance-level conditional modeling to enhance the model's controllability and stability in object-level generation scenarios.

[0081] Specifically, this embodiment proposes a conditionally gated Lora attention mechanism to extend the Flux.1-dev architecture. This mechanism utilizes low-rank adaptation (Lora) for conditional injection while embedding structured gating constraints into the attention computation, thereby achieving structured modulation of feature dependencies while preserving pre-trained representations.

[0082] The complete input sequence of the attention mechanism is represented as The total sequence length It is composed of feature codes from different modalities concatenated. Here, Represents denoising tokens. The sequence length (including the combined text and noise latent variable encoding), while This corresponds to conditional image coding. The sequence length. This represents the sequence data input into the attention mechanism. Here... It is a three-dimensional tensor; B represents the batch size, i.e., the number of data samples processed in one forward propagation. Each sample can be regarded as an independent task or query. S represents the sequence length, which refers to the number of information units contained in each sample. In this context, S is the total length concatenated from the feature codes of different modalities, including the denoised label sequence. and conditional image coding sequence This means that for each sample, its input consists of multiple information sources, which may come from different modalities (such as text, images, etc.). 'd' refers to the dimension of each information unit, that is, the size of the feature vector at each location. This dimension reflects the expressive power of each information unit; a higher dimension usually means stronger expressive power but also comes with higher computational cost.

[0083] (Length of Denoising Label Sequence): This part of the sequence typically contains labels directly relevant to the task, such as the combined text and noise latent variable encoding. They serve as key reference points for the model to learn how to recover or generate a clear image from noise. In diffusion models, the denoising process approximates the true data distribution by progressively removing noise; therefore, these labels are crucial in guiding how the model operates.

[0084] (Conditional Image Encoding Sequence Length): This part of the sequence contains the information encoded from the conditional image. In multimodal applications, in addition to text descriptions, specific visual cues or conditional images may serve as additional inputs to help the model more accurately understand and generate the target content. For example, in conditionally constrained image generation tasks, given a sketch or outline, the model can generate more specific details based on these conditions.

[0085] In summary, the phrase "input sequence of the attention mechanism" encapsulates the basic structure of the input data, emphasizing that the sequence consists of two parts: one part is the labeling related to the denoising process, and the other part is the encoding based on the conditional image. This design allows the model to effectively utilize conditional information to enhance the generation effect while maintaining an understanding of the original data.

[0086] To prevent the introduced LoRa from affecting the pre-trained knowledge of Flux.1-dev, this invention designs a conditionally gated LoRa attention mechanism, which implements a spatial gating function. Conditional tags are selectively activated during attention computation. This design confines learnable adaptive constraints within the conditional subspace while preserving the original parameterization of the denoised tags. The spatial gating function is defined as follows: ;in It is a binary mask that activates only the conditional subspace. This restricts subsequent adaptations to the specified conditional region by maintaining branch independence. This refers to the final conditional sequence generated, which is obtained by transforming the original sequence. With a binary mask Perform element-wise multiplication (Hadamard product, denoted as...) This process effectively activates or preserves [the cells / elements]. The part corresponding to the conditional subspace is masked, while the other parts are masked. Set to zero. Specifically, this is done by using such a binary mask. To selectively activate Certain parts of this approach ensure that subsequent processing steps are adjusted or optimized only within specified conditional regions, without affecting the entire sequence. This method helps maintain the independence between different branches, allowing each branch to develop independently based on its own conditions, while also enabling resources to be concentrated on optimizing feature representations that are critical to the task.

[0087] Given text, noise, and conditional branches Feature input and projection type The standard projection from the pre-trained Flux.1-dev backbone network is defined as: Each of them This represents a pre-trained attention-derived feature embedding that preserves its inherent representational semantics before applying any conditional adaptation.

[0088] For conditional branches, this embodiment uses LoRa for efficient adaptation. The low-rank parameterization formula is: ;in and Projection to rank The reduced subspace (and returned from it). This is the scaling factor. The formula for calculating the modulated projection is expressed as: ;in Controlling the influence of adaptive components. The text and noise branches retain their original pre-trained projections. The final query, key, and value inputs for attention computation are obtained by concatenating the projections of each branch along the sequence dimension: ; Calculate attention using the adaptive projection: ;in It is the dimension of the bond. Rotational Position Embedding (RoPE) can be applied to... and To maintain spatial alignment, the output is broken down into... and Noise reduction components Enter the diffusion decoding path, and It can be used by the auxiliary module according to the training configuration.

[0089] Specifically, noise reduction components Entering the diffusion decoding path: This means the denoising component obtained after the previous steps (such as feature extraction, adaptive adjustment, etc.) The data will be fed into a transparent decoder specifically designed for the "diffusion model." The diffusion model is a generative model that generates data by gradually adding noise and then learning to remove it. The decoding path here refers to a part of this process that attempts to recover a clear signal or image from noisy data. As input, it helps improve the denoising effect and generation quality during the decoding process.

[0090] and This can be used by the auxiliary module based on the training configuration: This section describes another processing result. Its uses. It's also a feature representation obtained through processing, but it doesn't directly enter the diffusion decoding process. Instead, its use depends on the specific training settings or requirements. Here, "auxiliary modules" can be understood as functional units other than those in the main diffusion decoding path; they may be responsible for different tasks, such as classification, regression, or other types of prediction tasks. This means... It can be flexibly applied to different scenarios or tasks, and can be adjusted and utilized according to actual needs.

[0091] Overall, this statement emphasizes the modular approach in system design, where different feature processing results can be applied to different subsequent processing steps or tasks, thereby achieving more efficient and flexible data processing and analysis capabilities.

[0092] By introducing a low-rank parameterization (Lora) adaptive mechanism for conditional branches, the flexibility and efficiency of multimodal input processing are significantly improved. Specifically, for feature inputs including text, noise, and conditional branches... and its projection type Standard projection preserves the inherent representational semantics of the pre-trained model, while conditional branches further enhance task-specific performance through efficient adaptation. A low-rank matrix is ​​used. and The feature embedding is projected into a lower-dimensional subspace and then projected back into the original space, combined with a scaling factor. The adjustments enabled efficient utilization of computing resources and performance optimization. Subsequently, parameters were adjusted... To control the influence of the adaptive components on the original features, targeted enhancement is achieved while preserving the original information. Finally, by concatenating the projection results of each branch and applying an attention mechanism, this method not only strengthens the fusion and interaction of cross-modal features but also maintains spatial alignment through Rotational Position Embedding (RoPE), improving the quality of the model output. The decomposed denoising components... and conditional components These components serve the subsequent diffusion decoding path and auxiliary modules, demonstrating the versatility and adaptability of this method in complex application scenarios. Overall, this method effectively improves the model's ability to process multi-source information while reducing computational costs, thus promoting more accurate and efficient prediction and generation tasks.

[0093] In one embodiment, the above-mentioned automatic optimization of the separation of foreground and background and edge smoothing through a transparency decoder to generate multiple transparent layers with natural transparency transitions includes steps S136 to S1311.

[0094] S136. Perform pixel brightness evaluation, color difference analysis, and saturation detection on the object image layer to obtain multi-dimensional analysis results.

[0095] In this embodiment, the multi-dimensional analysis result refers to the selection of a set of pixels that meet the background characteristics by evaluating the brightness, color difference, and saturation of each pixel in the object image layer.

[0096] In one embodiment, step S136 described above may include steps S1361 to S1363.

[0097] S1361. Calculate the average intensity of the RGB value of each pixel in the object image layer. If the average intensity is within the set brightness range, then determine that the pixel conforms to the background features in terms of brightness.

[0098] First, in step S1361, the average intensity of the RGB values ​​of each pixel in the object image layer is calculated, and it is checked whether this average intensity is within a preset brightness range. If the average intensity of a pixel falls within this specific brightness range, the pixel is considered to meet the background characteristics in terms of brightness, thereby initially filtering out a set of pixels that may belong to the background.

[0099] S1362. For pixels that meet the background characteristics in terms of brightness, calculate the standard deviation between the RGB channels. If the standard deviation is lower than a set threshold, then distinguish the pixel color from the colored area in the foreground to obtain the selected pixels.

[0100] Next, in step S1362, pixels that are considered to match the background characteristics in terms of brightness are further evaluated by measuring color difference through calculating the standard deviation between their RGB channels. If this standard deviation is below a certain set threshold, it indicates that the color variation of these pixels is not significant, and they can be considered as part of the background rather than the colored areas in the foreground, thus further filtering pixels.

[0101] S1363. For the selected pixels, check whether the color saturation does not exceed the saturation threshold. If the color saturation does not exceed the saturation threshold, then the multi-dimensional analysis result is determined to meet the requirements.

[0102] Finally, in step S1363, the pixels that have undergone the above two filtering steps are checked again, this time to see if their color saturation does not exceed a preset saturation threshold. If the color saturation of a pixel does not exceed this threshold, the pixel is confirmed as the final multi-dimensional analysis result that meets the requirements, meaning that it is identified as part of the background rather than a foreground element. The entire process effectively distinguishes between the background and foreground parts of the image by comprehensively judging multiple dimensions (brightness, color difference, and saturation).

[0103] S137. Mark the pixels that meet the requirements of the multi-dimensional analysis results as the background to form an initial background mask.

[0104] Pixels that are identified as meeting background characteristics through multi-dimensional analysis (including brightness, color difference, and saturation) are marked as background, forming an initial background mask. This initial background mask is a basic version used to distinguish foreground and background regions in an image.

[0105] S138. The initial background mask is processed by closing and opening operations to remove holes and noise, resulting in a continuous and smooth background mask.

[0106] In this embodiment, a predefined structuring element is used to perform a closing operation on the initial background mask to fill holes and breaks, and then an opening operation is performed to remove noise and artifacts, thereby optimizing and obtaining a continuous and smooth background mask.

[0107] The initial background mask is further processed using closing and opening operations from mathematical morphology. First, a closing operation is performed on the initial background mask using predefined structuring elements to fill in holes and breaks within the mask, making the background region more complete and continuous. Then, an opening operation is performed to remove any potential noise and artifacts, optimizing the result to a smooth, continuous background mask without obvious holes or redundant noise. This step is crucial for ensuring the high quality of the final generated background mask.

[0108] S139. Convert the continuous smooth background mask into a binary alpha image, and define the foreground area as opaque and the background area as transparent to form the boundary region identifier between the foreground and the background.

[0109] In this embodiment, the background mask is used to characterize the spatial distribution relationship between the foreground and the background, wherein the partitioning relationship between the foreground and the background is clearly defined by setting the foreground area to be opaque and the background area to be transparent.

[0110] Set the foreground region of a continuous, smooth background mask to opaque and the background region to transparent to obtain the boundary region identifier between the foreground and background.

[0111] The continuous smooth background mask obtained after the above processing is converted into a binary alpha image. In this process, the foreground region is set to be completely opaque (alpha value of 1), and the background region is set to be completely transparent (alpha value of 0), thereby clearly defining the boundary between the foreground and background and obtaining boundary region information for subsequent transparency transition processing.

[0112] S1310. Based on the background mask, locate the edge transition region between the foreground and the background, and apply Gaussian filtering only to the corresponding region to smooth the transparency transition, so as to obtain the processed transparency channel.

[0113] These edge transition zones refer to areas where there is a change in transparency between the foreground and background. Applying Gaussian filtering only to these specific areas can effectively smooth the transition of transparency, avoiding the visual abruptness caused by hard edges, thus obtaining a more natural and smooth transparency channel.

[0114] S1311. Combining the processed transparency channel and the RGB channel corresponding to the single input image, generate multiple transparent layers with natural transparency transitions.

[0115] By combining the processed alpha channel with the RGB channels of a single input image, multiple transparent layers with natural transparency transitions are generated. This process not only enhances the overall visual effect of the image but also ensures foreground and background separation while preserving the original image's color and detail, making it particularly suitable for applications requiring fine editing or compositing. Through these steps, from initial background pixel selection to the generation of the final transparent layers, the entire process provides a systematic and effective method for achieving refined image processing.

[0116] In this embodiment, the Flux.1-dev backbone generation model is modulated based on the conditionally gated LoRa attention mechanism, enabling it to generate a corresponding reconstructed object image layer for each detected foreground entity. The generated object layer is denoted as... Each object image is dominated by foreground content and overlaid on a uniform neutral gray background.

[0117] In order to convert the above-mentioned object image layer containing only RGB information into an RGBA layer with a transparency channel, this embodiment proposes a non-parametric transparency decoding method based on pixel color consistency and brightness characteristics to automatically separate the foreground object region and the background region.

[0118] First, the object image layer Each pixel in Perform pixel-by-pixel analysis and identify the background region based on the following three criteria: Brightness analysis: Calculates the average intensity value of the pixel across the three RGB color channels to represent the overall brightness of the pixel. When this brightness value falls within a preset brightness range... At that time, it was considered that the pixel conformed to the characteristics of a neutral gray background in terms of brightness.

[0119] Color consistency check: Further calculate the standard deviation of this pixel across the RGB channels to measure the degree of difference between color channels. When this standard deviation is less than a preset threshold... When the value is close to colorless, it indicates that the pixel is close to colorless and can effectively distinguish it from colored areas in the foreground object that have structural texture but similar brightness.

[0120] Saturation Limit: Calculates the color saturation of a pixel and requires it to not exceed a saturation threshold. This is to ensure that the identified area belongs to a strictly non-colored area.

[0121] A pixel is marked as a background pixel only if all three conditions are met; otherwise, it is marked as a foreground pixel. By performing the above determination process on all pixels, an initial background mask is constructed. This multi-constraint decision mechanism can still reliably identify the background region even when the foreground color is close to gray or the generated result has a slight color cast, thus improving the reliability of background separation.

[0122] Because the generative model may introduce local breaks, holes, or isolated noise near object edges or occluded areas, the initial background mask... The background mask may be discontinuous in its topological structure. Therefore, this embodiment performs a morphological reconstruction operation on the background mask. Specifically, predefined structural elements are selected. First, a closing operation is performed on the background mask to fill small-scale holes and broken areas. Then, an opening operation is performed to remove isolated noise and small artifacts, thereby obtaining a background mask with a more complete topological structure and smoother boundaries. .

[0123] This morphological processing step can effectively improve the stability of the background mask in terms of spatial continuity and regional integrity, providing a reliable basis for subsequent transparency calculations.

[0124] Obtaining the background mask Then, it is first converted into a binary alpha initial image, that is, the foreground area is set to opaque and the background area is set to transparent, forming the initial alpha matte.

[0125] Subsequently, the boundary transition zone between the foreground and background regions is automatically located and defined as the edge region set. A Gaussian filter is applied to the initial alpha matte only for this edge region to create a continuous, smooth transparency transition at the foreground edge. In non-edge regions, the initial alpha value is kept unchanged, thus avoiding unnecessary blurring of the foreground's internal structure or overall shape.

[0126] By employing the aforementioned locally adaptive transparency refinement strategy, jagged edges and stair-step artifacts can be effectively suppressed while maximizing the preservation of the geometric contours and visual consistency of the foreground object. Finally, the RGB channels of the object image layer are fused with the transparency channel after edge-guided smoothing, resulting in the complete RGBA layer output as follows: The RGBA layer can be directly used for subsequent layer compositing, editing, or redrawing operations, enabling controllable layered representation of multiple visual entities in a single image.

[0127] In this embodiment, by combining the conditionally gated LoRa attention mechanism with non-parametric transparent decoding technology, the object image layer containing only RGB information is effectively converted into an RGBA layer with an Alpha channel. This achieves accurate separation and smooth transition between the foreground and background regions, significantly improving the reliability, spatial continuity, and visual consistency of the generated image in background separation. At the same time, it optimizes the processing effect of the foreground object edges, enabling the final output RGBA layer to support subsequent image compositing, editing, or redrawing operations with high quality.

[0128] S140. Output the decomposition results.

[0129] The decomposition results are output to the terminal for display.

[0130] For common issues encountered by existing technologies in object localization, content reconstruction, and alpha channel decoding, please refer to [link to relevant documentation]. Figure 2 The method in this embodiment effectively solves challenges such as occlusion reconstruction, transparency quality, controllability, and system integration by unifying the design of these three levels.

[0131] First, regarding object integrity reconstruction in occluded scenarios, the method in this embodiment provides independent region constraints for each target object and performs overall reconstruction using a finely tuned multimodal large model. This mechanism not only preserves the features of the visible region but also infers the occluded parts, achieving a semantically coherent and structurally complete object layer, avoiding the problem of outputting incomplete objects in traditional methods.

[0132] Secondly, regarding the generation of the alpha channel, this embodiment introduces a color-invariant transparency decoding strategy. By comprehensively utilizing grayscale, saturation, and local statistical characteristics to model and optimize the alpha channel while maintaining the RGB colors, the smoothness of the boundaries and the continuity of the semi-transparency are effectively improved, avoiding jagged edges, background contamination, and color drift, ultimately generating a high-quality RGBA layer.

[0133] Furthermore, in terms of controllability, the method in this embodiment uses the detection results and instance-level visual conditions as the core control signals, introduces a condition-gated LoRa attention mechanism to accurately inject visual conditions, and decouples the text and noise branches for control, reducing the dependence on the quality of prompt words and achieving accurate and stable reconstruction of specific elements.

[0134] Finally, regarding system integration, the method in this embodiment constructs an integrated detection-generation-decoding framework, organically integrating object detection, content generation, and transparent channel extraction. Through conditional propagation and collaborative design, tight coupling between modules is achieved, avoiding multi-stage error accumulation and improving the stability and usability of the entire system.

[0135] The algorithm identifies and locates visual elements from a single input image, then establishes spatial constraints for candidate layers of each visual element through region cropping. Building upon this, a conditionally gated LoRa attention modulation mechanism is introduced. During the attention calculation phase of the generation process, low-rank parameter modulation is enabled only for conditional features related to the target visual element. This achieves fine-grained region-level generation control while maintaining the stability of the pre-trained model, completing structural completion and appearance consistency reconstruction of invisible regions of visual elements. Subsequently, a transparency layer decoding method based on constraints such as pixel brightness range, color consistency, and saturation is used to automatically distinguish foreground and background and generate transparency masks. Combined with smoothing optimization of target boundary regions, the generated foreground image is converted into an RGBA transparency layer with complete structure, clear boundaries, and preserved original color information. This solves the problems of incomplete structure and blurred boundaries when generating transparency layers from single images, supporting subsequent image editing and design applications.

[0136] Extensive experimental verification demonstrates that the method proposed in this embodiment can stably generate transparent layers with complete structure, consistent appearance, and independent editability in various complex scenarios. Experimental results show that this method outperforms existing technologies in terms of occlusion region recovery, boundary quality, and layer usability, proving the feasibility and effectiveness of the method's technical solution and providing reliable technical support for single-image transparent layer generation and its practical application.

[0137] Specifically, the goal of this embodiment is to decompose a single image into multiple transparent RGBA layers. Since foreground elements in the original image are often occluded, traditional matting evaluation metrics relying on complete alpha ground truth are no longer applicable. Therefore, the evaluation focuses on the reconstruction accuracy of visible areas and the generation quality of occluded areas. This embodiment uses Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to evaluate pixel-level and structural-level reconstruction accuracy; it uses the LPIPS metric to measure perceptual feature consistency; and it uses Fréchet Inception Distance (FID) to evaluate the realism of the generated layers at the distribution level. FID is calculated on only the foreground RGB layers (after alpha synthesis) to conform to the general evaluation protocol of existing layered generation methods. These metrics comprehensively evaluate the layer decomposition and occlusion completion capabilities from multiple dimensions.

[0138] The improved multimodal large model (PRISM) was trained using the AdamW optimizer with a learning rate of 0.0001. The weight decay coefficient is 0.0001. A constant learning rate strategy is adopted during training, and a 500-step warm-up is set to improve training stability. To ensure training efficiency, only the parameters of the Lora adapter layer (rank 128) are updated, while all other backbone network parameters are frozen. The model is trained in bf16 mixed precision mode with gradient pruning of maximum norm 1.0. The model is trained for a maximum of 100 epochs, or terminated early upon convergence. During the inference phase, a flow-matching based sampling strategy is adopted, and the Euler scheduler is used for 20 steps of denoising with a guidance scale of 4.0, achieving a good balance between generation quality and inference efficiency.

[0139] Since PRISM structurally requires each visual entity to be modeled as an independent layer, the method in this embodiment further processes and re-divides the PrismLayersPro dataset to meet training and evaluation requirements. During the training phase, 1985 original multi-layer images were used, and they were split into 8321 independent foreground element layers and 1985 corresponding background layers. During the testing phase, the test set consisted of 181 original images, comprising 819 foreground element layers and 181 background layers.

[0140] For each foreground element layer, it is cropped to its locally visible area based on its rectangular bounding box to ensure the accuracy and consistency of the foreground representation during the condition generation process.

[0141] This embodiment compares PRISM with existing representative layer decomposition methods LayerD and LayerDiffuse. In qualitative experiments, LayerD and PRISM directly use the entire image as input; since the LayerDiffuse method only supports single-element input, the input image is first preprocessed using PRISM's layer detection module to extract the corresponding transparent elements as its input. In quantitative evaluation, to address the issue of inconsistent layer counts in LayerD output, this embodiment uniformly uses the layer detection module to extract all foreground elements from the original image and forces LayerD to output a two-layer structure of "foreground + background," thereby ensuring fairness in the evaluation.

[0142] Table 1 shows that PRISM significantly outperforms existing methods in all indicators.

[0143] Table 1. Comparison of PRISM, LayerD, and LayerDiffuse performance on the PrismLayerPro dataset.

[0144] In background layer reconstruction, the method in this embodiment demonstrates significant advantages in terms of structural consistency and distribution realism. PRISM achieves an SSIM of 0.9909, significantly higher than the comparative methods (LayerDiffuse: 0.7436, LayerD: 0.9391), indicating that it approaches the ideal reconstruction state in terms of global structure preservation and detail continuity. Simultaneously, its FID is only 26.17, significantly lower than existing methods, demonstrating that the method in this embodiment can effectively suppress background noise and texture distribution distortion, generating a more stable and natural background layer.

[0145] In the task of reconstructing foreground element layers, the method in this embodiment shows a more significant improvement in object integrity and perceptual consistency. PRISM achieves a PSNR of 30.35 dB, which is significantly better than the comparison methods, indicating that it has higher accuracy in foreground detail recovery and pixel consistency; at the same time, its LPIPS drops to 0.0606, which is significantly lower than existing methods, indicating that the generated foreground elements are more perceptually closer to the real object, effectively reducing structural loss and perceptual bias.

[0146] A comparison of the data from the background layer and the foreground element layer shows that PRISM outperforms the comparison methods in key indicators such as pixel accuracy, structural consistency, perceptual quality, and distribution realism. Figure 4 The visualization results show that LayerD generates an inconsistent number of transparent layers during element transparency decomposition, and most of these layers suffer from incomplete element structures and significant background residue. LayerDiffuse also exhibits severe background noise in its element transparency layers. In contrast, the method presented in this embodiment generates element transparency layers with high consistency with the actual values ​​in terms of structural integrity and background purity, effectively meeting the practical needs of users. This result verifies that the condition-constrained layer generation and transparency decoding mechanism proposed in this embodiment can effectively solve the technical problems of incomplete foreground decoupling, severe boundary noise, and unstable generation quality in complex scenes found in existing methods. Therefore, the experimental results fully demonstrate that the method in this embodiment can achieve high-quality transparent layer reconstruction of multiple visual entities in a single image, with a stable improvement in technical performance.

[0147] Table 2 compares the differences in model reconstruction performance and computational efficiency between the base model Flux.1-dev and different Lora injection structures.

[0148] Table 2 Performance comparison between the Flux.1-dev base model and the variant injected with LoRa.

[0149] The base model has an SSIM of 0.63 and a PSNR of 10.24, indicating significant deficiencies in structure preservation and pixel reconstruction accuracy. Introducing self-attention Lora (self) improves the SSIM to 0.88 and PSNR to 23.42, demonstrating that the efficient parameter adaptation mechanism effectively enhances the model's ability to model image structural features. Further employing causal attention Lora (Causal) increases the SSIM to 0.91 and PSNR to 27.02, demonstrating that adding temporal or dependency constraints to self-attention can further improve structural consistency and reconstruction stability. Using the conditional gating Lora attention mechanism proposed in this embodiment, the SSIM increases to 0.97 and PSNR reaches 31.48, indicating that this structure more effectively regulates the feature injection process under conditional guidance, thereby significantly improving the quality of hierarchical generation and reconstruction.

[0150] In terms of perceived quality, the base model has an FID of 253.61, which drops to 49.80 after introducing self-attention Lora, and further decreases to 36.61 when causal attention Lora is adopted. The conditional gating Lora has an FID of 19.66, indicating that this structure has significant advantages in texture consistency and distribution realism.

[0151] In terms of computational efficiency, conditionally gated LoRa maintains the highest reconstruction performance while achieving an inference time of 6.32s and GFLOPs of 0.52M, which is a better balance compared to other LoRa variants and does not introduce any additional significant computational burden.

[0152] The comparative experiments described above demonstrate that, under the same training and testing conditions, the conditionally gated LoRa structure proposed in this embodiment exhibits stable and quantifiable technical advantages in terms of reconstruction accuracy, structural consistency, perception quality, and computational efficiency.

[0153] Table 3 compares the model's performance and computational efficiency at different input resolutions.

[0154] Table 3 Performance comparison under training with data at different resolutions

[0155] The results show that while the lower resolution (256²) can complete the basic structure reconstruction, the PSNR is 24.83 and SSIM is 0.94, indicating limited pixel accuracy and detail consistency. The perceptual quality (FID) is 103.46, showing a significant gap between the generated result and the true distribution. When the resolution is increased to 512², the PSNR increases to 31.48, the SSIM reaches 0.97, and the FID decreases to 19.66, showing a significant improvement in structural consistency, pixel reconstruction accuracy, and perceptual quality. Meanwhile, the inference time is 7.2 s and GFLOPs are 0.58M, with computational costs within an acceptable range. Further increasing the resolution to 768² or 1024² does not lead to a sustained performance improvement; in fact, some metrics even decrease (e.g., the PSNR at 768² is 27.02 and the FID is 36.61), indicating that excessively high resolution does not bring additional reconstruction advantages but instead increases computational costs.

[0156] The experimental results above show that different input resolution settings directly affect the model's reconstruction accuracy, perceptual quality, and computational efficiency. Among the various resolution conditions tested, the 5122 resolution achieved superior results in terms of structural consistency, pixel reconstruction accuracy, and perceptual quality, while its computational cost remained within an acceptable range.

[0157] Therefore, in this embodiment, the input image resolution is set to 5122 to ensure both the quality of layer reconstruction and computational efficiency, thereby achieving a more balanced technical effect.

[0158] This embodiment aims to address the challenges encountered by existing technologies in processing complex images. First, the method achieves effective separation of foreground elements and background content by performing instance-level precise detection and independent reconstruction of objects in the input image. This process particularly emphasizes maintaining the integrity and editability of the foreground object structure in the presence of occlusion, thus providing a solid foundation for subsequent image processing and editing. Furthermore, to improve the consistency and integrity of the generated results, a conditionally gated LoRa attention mechanism is introduced. This mechanism selectively injects instance-level visual information into the attention layer of the generative model, enabling the system to effectively complete the structure of missing or occluded areas while maintaining the consistency of object appearance. This not only enhances the realism of the generated image but also improves its adaptability to various application scenarios. Finally, considering the requirements for color fidelity in practical applications, a color-invariant transparency layer decoding strategy is developed. This strategy allows for fine-grained modeling of transparency without altering the original RGB information of the generated object, ensuring clear boundaries and natural transparency transitions in the generated RGBA layer. This method not only improves image quality but also meets the needs of downstream editing tasks, such as applications in compositing and augmented reality.

[0159] In summary, the method of this embodiment, by combining advanced detection technology, attention mechanisms, and color management strategies, provides a comprehensive and efficient solution for generating high-quality, editable transparent layers, significantly advancing the development of the field of multimodal image processing.

[0160] In another embodiment, different deep learning architectures can be considered, such as Transformer-based models or convolutional neural network (CNN) variants, as long as they can accurately identify and separate objects in an image at the instance level and effectively reconstruct foreground elements from background content. The key is that the chosen model must have sufficient expressive power to capture complex visual information and be able to adapt to challenges such as occlusion, thereby ensuring high-quality generated results.

[0161] Furthermore, to further optimize performance, other improvement strategies can be explored. For example, reinforcement learning mechanisms can be introduced to allow the model to self-adjust parameters to better adapt to different scenarios; or generative adversarial networks (GANs) can be used to enhance the realism and detail integrity of generated images. Importantly, regardless of the chosen technical path, it is crucial to maintain the algorithm's flexibility and versatility, ensuring it is not limited to the current application scenario but can be extended to other related image processing tasks. In this way, not only can the current desired results be achieved, but a solid foundation for future technological innovation can also be laid.

[0162] The aforementioned detection-guided object-level transparency layer generation method first feeds a single input image into a detection layer model to identify and separate various visual elements, forming a set of object layers with independent bounding boxes. Then, using an improved multimodal large model combined with a conditionally gated LoRa attention mechanism, the object image layer is reconstructed for each object layer based on specific conditions. A transparency decoder optimizes foreground-background separation and edge smoothing, enabling the efficient and accurate extraction of multiple independent layers containing complete object information and high-quality transparent boundaries from a single input image. This effectively addresses the shortcomings of existing technologies in handling occlusion, transparency modeling, and text-driven generation, meeting the needs for independent reconstruction and fine-tuning of image elements. This method not only improves the quality and accuracy of image decomposition but also enhances the natural transition effect of the generated results, providing strong support for advanced image editing.

[0163] Figure 5 This is a schematic block diagram of a system 300 for generating transparent layers at the level of a guided object, provided in an embodiment of the present invention. Figure 5As shown, corresponding to the above-described method for generating transparent layers at the level of a guide object, the present invention also provides a system 300 for generating transparent layers at the level of a guide object. This system 300 includes a unit for executing the above-described method for generating transparent layers at the level of a guide object, and the system can be configured in a desktop computer, tablet computer, laptop computer, or other terminal. Specifically, please refer to... Figure 5 The detection-guided object-level transparent layer generation system 300 includes an acquisition unit 301, a layer detection unit 302, a transparent layer generation unit 303, and an output unit 304.

[0164] The acquisition unit 301 is used to acquire a single input image; the detection layer unit is used to input the single input image into the detection layer model to identify and separate each visual element to form an independent bounding box, thereby obtaining an object layer set; the transparent layer generation unit 303 is used to input the object layer set into an improved multimodal large model to perform multiple editable RGBA transparent layer decomposition to obtain the decomposition result; including: using a conditionally gated LoRa attention mechanism to reconstruct the object image layer based on specific conditions, and automatically optimizing the separation of foreground and background and edge smoothing through a transparent decoder to generate multiple transparent layers with natural transparency transitions; the output unit 304 is used to output the decomposition result.

[0165] In one embodiment, the detection layering unit includes: The feature map generation subunit is used to input the single input image into the detection hierarchical model and generate multi-scale feature maps through the backbone network to form a feature pyramid. The first prediction subunit is used to use the region proposal network to predict multiple candidate regions and confidence scores based on the feature pyramid to obtain a first candidate region. The filtering subunit is used to filter the first candidate region according to the confidence threshold to obtain a second candidate region. The alignment and cropping subunit is used to perform a region alignment operation on the second candidate region and, based on the aligned candidate region, crop the corresponding part from the single input image as an independent object layer, and integrate all object layers into a set to obtain an object layer set, wherein the object layers in the object layer set are all processed according to the timeline.

[0166] In one embodiment, the transparent layer generation unit 303 includes: The system comprises the following subunits: a partitioning subunit, which divides the object layer set into a denoised label sequence and a conditional image encoding sequence to obtain a partitioning result; an activation subunit, which designs a spatial gating function to apply a binary mask to the activation conditional subspace of the partitioning result to obtain an activation result; an adaptive processing subunit, which applies a pre-trained standard projection formula to the activation result to preserve the original parameterization and representation semantics, and uses low-rank parameterization techniques to adaptively process the conditional branches, adjusting the scaling factor to obtain a processing result; a concatenation subunit, which concatenates the query, key, and value projections corresponding to the processing result along the sequence dimension to form the final input result; and an output calculation subunit, which calculates the attention output based on the final input result and applies rotational position embedding to maintain spatial alignment to obtain denoising components and conditional components to reconstruct the object image layer.

[0167] In one embodiment, the transparent layer generation unit 303 includes: The multidimensional analysis subunit is used to perform pixel brightness evaluation, color difference analysis, and saturation detection on the object image layer to obtain multidimensional analysis results; the marking subunit is used to mark pixels that meet the requirements of the multidimensional analysis results as background to form an initial background mask; the optimization subunit is used to process the initial background mask through closing and opening operations to remove holes and noise and obtain a continuous and smooth background mask; the definition subunit is used to convert the continuous and smooth background mask into a binary alpha image and define the foreground transparency and background transparency to obtain the boundary region identifier between the foreground and background; the filtering subunit is used to locate the edge transition region between the foreground and background based on the background mask and apply Gaussian filtering only to the corresponding region to smooth the transparency transition to obtain the processed transparency channel; the layer generation subunit is used to combine the processed transparency channel and the RGB channel corresponding to the single input image to generate multiple transparent layers with natural transparency transitions.

[0168] In one embodiment, the multidimensional analysis subunit includes: The brightness analysis module is used to calculate the average intensity of the RGB values ​​of each pixel in the object image layer. If the average intensity is within a set brightness range, the pixel is determined to meet the background characteristics in terms of brightness. The color analysis module is used to calculate the standard deviation between the RGB channels for pixels that meet the background characteristics in terms of brightness. If the standard deviation is lower than a set threshold, the pixel color is distinguished from the colored area in the foreground to obtain the selected pixels. The saturation analysis module is used to check whether the color saturation of the selected pixels does not exceed the saturation threshold. If the color saturation does not exceed the saturation threshold, the multi-dimensional analysis result is determined to meet the requirements.

[0169] In one embodiment, the optimization subunit is used to perform a closing operation on the initial background mask using predefined structuring elements to fill holes and breaks, and then perform an opening operation to remove noise and artifacts, thereby optimizing to obtain a continuous and smooth background mask.

[0170] In one embodiment, the defined subunit is used to set the foreground region of a continuous smooth background mask to opaque and the background region to transparent, so as to obtain the boundary region identifier between the foreground and the background.

[0171] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned detection guidance object-level transparent layer generation system 300 and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0172] The aforementioned detection-guided object-level transparent layer generation system 300 can be implemented as a computer program, which can, for example... Figure 6 It runs on the computer device shown.

[0173] Please see Figure 6 , Figure 6 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0174] See Figure 6 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0175] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a method for detecting and generating a transparent layer at the bootstrap level.

[0176] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0177] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for detecting and generating transparent layers at the level of a guide object.

[0178] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 6The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0179] The processor 502 is used to run a computer program 5032 stored in the memory to implement all the steps of the detection guide object-level transparent layer generation method.

[0180] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0181] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0182] Therefore, the present invention also provides a storage medium. This storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all the steps of the method for generating a detection-guided object-level transparent layer.

[0183] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0184] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0185] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0186] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0187] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0188] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting the generation of transparent layers at the guide object level, characterized in that, include: Acquire a single input image; The single input image is input into the detection hierarchical model to identify and separate each visual element to form an independent bounding box, thus obtaining an object layer set; The object layer set is input into an improved multimodal large model to decompose multiple editable RGBA transparent layers to obtain the decomposition results; including: reconstructing the object image layer based on specific conditions using a conditionally gated Lora attention mechanism, and automatically optimizing the separation of foreground and background and edge smoothing through a transparency decoder to generate multiple transparent layers with natural transparency transitions; Output the decomposition results; The detection hierarchical model is constructed by building a dataset that is hierarchical to image elements and training the model under the Detectron2 framework. During the training process, element separability and spatial independence are used as the criteria to uniformly label and learn various visual elements in a single input image, so that the model can stably output a set of candidate bounding boxes with comprehensive coverage and reasonable boundaries in complex scenes. The condition-gated LoRa attention mechanism includes parallel conditional branching and structured modulation mechanisms for instance-level conditional modeling. The condition-gated LoRa attention mechanism embeds structured gating constraints into the low-rank adaptive attention computation of the improved multimodal large model through parallel conditional branching. The method of reconstructing an object image layer based on specific conditions using the conditionally gated LoRa attention mechanism includes: The object layer set is divided into a denoising marker sequence and a conditional image coding sequence to obtain the partitioning result; Design a spatial gating function to activate the conditional subspace of the partitioning result using a binary mask, so as to obtain the activation result; The activation results are applied to a pre-trained standard projection formula to preserve the original parameterization and representation semantics, and the conditional branches are adaptively processed using low-rank parameterization techniques to adjust the scaling factor in order to obtain the processing result. The query, key, and value projections corresponding to the processing results are concatenated along the sequence dimension to form the final input result; The attention output is calculated based on the final input result, and rotational position embedding is applied to maintain spatial alignment in order to obtain denoising components and conditional components to reconstruct the object image layer.

2. The method for generating a transparent layer at the level of a guiding object according to claim 1, characterized in that, The step of inputting the single input image into the detection hierarchical model to identify and separate each visual element to form an independent bounding box, thereby obtaining an object layer set, includes: The single input image is input into the detection hierarchical model, and multi-scale feature maps are generated through the backbone network to form a feature pyramid; The region proposal network is used to predict multiple candidate regions and their confidence scores based on the feature pyramid to obtain the first candidate region; The first candidate region is filtered based on a confidence threshold to obtain the second candidate region; Perform a region alignment operation on the second candidate region, and based on the aligned candidate region, crop the corresponding part from the single input image as an independent object layer, integrate all object layers into a set to obtain an object layer set, wherein the object layers in the object layer set are all processed according to the timeline.

3. The method for generating a transparent layer at the level of a guiding object according to claim 1, characterized in that, The process automatically optimizes the separation of foreground and background and smooths edges using a transparency decoder, generating multiple transparent layers with natural transparency transitions, including: Pixel brightness evaluation, color difference analysis, and saturation detection are performed on the object image layer to obtain multi-dimensional analysis results; Pixels that meet the requirements of the multi-dimensional analysis results are marked as background to form an initial background mask; By processing the initial background mask through closing and opening operations, holes and noise are removed to obtain a continuous and smooth background mask; The continuous smooth background mask is converted into a binary alpha image, and the foreground transparency and background transparency are defined to obtain the boundary region identifier between the foreground and the background. Based on the background mask, the edge transition region between the foreground and the background is located, and Gaussian filtering is applied only to the corresponding region to smooth the transparency transition, so as to obtain the processed transparency channel. By combining the processed transparency channel with the RGB channel corresponding to the single input image, multiple transparent layers with natural transparency transitions are generated.

4. The method for generating a transparent layer at the level of a guiding object according to claim 3, characterized in that, The process of evaluating pixel brightness, analyzing color differences, and detecting saturation in the object image layer to obtain multi-dimensional analysis results includes: The average intensity of the RGB value of each pixel in the object image layer is calculated. If the average intensity is within the set brightness range, the pixel is determined to conform to the background features in terms of brightness. For pixels that match the background characteristics in terms of brightness, the standard deviation between the RGB channels is calculated. If the standard deviation is lower than a set threshold, the pixel color is distinguished from the colored area in the foreground to obtain the selected pixels. For the selected pixels, check whether the color saturation does not exceed the saturation threshold. If the color saturation does not exceed the saturation threshold, then the multi-dimensional analysis result is determined to meet the requirements.

5. The method for generating a transparent layer at the level of a guiding object according to claim 4, characterized in that, The process of processing the initial background mask through closing and opening operations to remove holes and noise, resulting in a continuous and smooth background mask, includes: The initial background mask is closed using a predefined structuring element to fill holes and breaks, and then an opening operation is performed to remove noise and artifacts, resulting in a continuous and smooth background mask.

6. A system for detecting and generating transparent layers at the object level, characterized in that, The system uses the method for generating a transparent layer at the level of a guided object as described in any one of claims 1 to 5, including: The acquisition unit is used to acquire a single input image; The detection layer unit is used to input the single input image into the detection layer model to identify and separate each visual element to form an independent bounding box, thereby obtaining an object layer set; A transparent layer generation unit is used to input the object layer set into an improved multimodal large model to decompose multiple editable RGBA transparent layers to obtain decomposition results; including: reconstructing the object image layer based on specific conditions using a conditionally gated Lora attention mechanism, and automatically optimizing the separation of foreground and background and edge smoothing through a transparent decoder to generate multiple transparent layers with natural transparency transitions; The output unit is used to output the decomposition results.

Images