A method, apparatus, device, and storage medium for splitting editable image layers.
By using adaptive alpha masking and background repair technology, the problem of accidental deletion of objects in image editing was solved, and the integrity and visual effect of the layer splitting were improved.
Patent Information
- Application Number
- CN202611304453.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-26
- Publication Date
- 2026-11-10
AI Technical Summary
In image editing, existing techniques often mistakenly delete areas of the object that are close to the background when using solid-color backgrounds for object extraction. This can lead to local missing parts, holes, or abnormal edge colors, affecting the integrity and visual effect of the separated object.
By generating an adaptive alpha mask, combining chromatic aggregator features and color protection information, the main subject area is made transparent to generate a transparent foreground layer. The background is then repaired using a background healing mask to ensure the integrity and visual effect of the split layers.
It improves the integrity of image layers after splitting and the visual effect when recombining them, maintains the independent editing capabilities of the main subject, text, and background, and restores the original visual structure and edge effects.
Smart Images

Figure CN123071479A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and specifically to an editable image layer splitting method, apparatus, device, and storage medium. Background Technology
[0002] In existing image editing processes, for images such as posters, advertisements, and product images that have already been composited into a single bitmap, methods such as object detection, image segmentation, text recognition, and image completion are typically used to extract objects, text, and underlying elements from the image. Object extraction can employ semantic segmentation, saliency detection, or background removal based on a specified color to obtain independent images. Text is extracted using text recognition to determine its content and location, and the original object's location is filled using image completion or generative image processing. After this processing, different visual elements in the original bitmap can be used for subsequent movement, replacement, or content modification.
[0003] In methods that use solid-color backgrounds to assist object extraction, pixel retention is typically determined by the similarity between the background color to be removed and the colors of individual pixels. When the object being extracted contains areas that are close to the set background color, these areas are easily judged as invalid pixels along with the background to be removed. This is especially true for similar-colored textures, reflective areas, and contour transition areas on product surfaces, which can easily result in localized missing pixels, holes, or abnormal edge colors. Such accidental deletions directly affect the integrity of the object after it is separated from the original image, and even if it is subsequently overlaid onto the underlying image, it is difficult to maintain the original visual structure and edge effects. Summary of the Invention
[0004] This invention provides an editable image layer splitting method, apparatus, device, and storage medium, which can improve the integrity of the image layer after splitting and the visual effect when it is subsequently recombined.
[0005] A first aspect of the present invention provides an editable image layer splitting method, the method comprising: A unified canvas is determined based on the acquired source image, and visual structure analysis is performed on the source image to obtain the main subject area, text information, and background description. An intermediate image containing the main subject and a predetermined solid color chroma key background is generated based on the main subject region. An adaptive alpha mask is generated by combining the chroma key features of the intermediate image and the color protection information of the source image. The intermediate image is made transparent based on the adaptive alpha mask to obtain a transparent foreground layer; A background repair mask is generated based on the main subject area and the text area in the text information. The background is repaired in the area covered by the background repair mask according to the background description. The repaired area is then combined with the source image to obtain a background layer. An editable text layer is generated based on the text information, and the transparent foreground layer, the background layer, and the editable text layer are output according to the coordinate system corresponding to the unified canvas.
[0006] In a second aspect of the invention, an editable image layer splitting apparatus is provided, the apparatus being used to perform an editable image layer splitting method as described in any of the above embodiments, the apparatus comprising an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to determine a unified canvas based on the acquired source image, perform visual structure analysis on the source image, and obtain the main subject area, text information, and background description. The processing module is used to generate an intermediate image containing the main subject and a predetermined solid color chroma key background based on the main subject region, and to generate an adaptive alpha mask by combining the chroma key features of the intermediate image and the color protection information of the source image. The processing module is used to perform transparency processing on the intermediate image based on the adaptive alpha mask to obtain a transparent foreground layer; The processing module is used to generate a background repair mask based on the main subject area and the text area in the text information, perform background repair on the area covered by the background repair mask according to the background description, and synthesize the repaired area with the source image to obtain a background layer. The output module is used to generate an editable text layer based on the text information, and output the transparent foreground layer, the background layer, and the editable text layer according to the coordinate system corresponding to the unified canvas.
[0007] In a third aspect of the invention, an electronic device is provided, including a processor and a memory, the memory storing a computer program, wherein the computer program, when executed by the processor, implements an editable image layer splitting method as described in any of the preceding claims.
[0008] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium storing instructions that, when executed, perform an editable image layer splitting method as described in any of the preceding claims. In summary, one or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: The reason why this invention can improve the integrity of image layers after splitting and the visual effect of subsequent recombination is that it does not simply cut the subject, text and background of a single bitmap separately. Instead, it first constrains all layers with a unified canvas and a unified coordinate system, so that the transparent foreground layer, background layer and editable text layer are consistent in spatial position. Then, it uses color protection information in the source image, adaptive alpha mask and joint judgment of boundary connectivity and high confidence chromatic key to avoid the accidental deletion of inherent colors, reflections and edge transition areas in the main subject that are similar to the chromatic key. At the same time, it reduces the number of holes, outline defects and color edge residues in the subject by restoring the inverse chromatic key color of semi-transparent edges and checking for chromatic key residues. For the background, the repair range is limited by the background repair mask, and the generated result is only used in the area covered by the mask, while the original pixels of the source image are retained in the other areas. Thus, the consistency of background texture, lighting and layout can still be maintained after the main subject and text are removed. Finally, the split layers have independent editing capabilities and can better restore the original visual structure and edge effects when re-overlayed. Attached Figure Description
[0009] Figure 1 This is a flowchart illustrating an editable image layer splitting method disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the generation process of an adaptive alpha mask according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a combined background mask and locally restricted compositing process disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of a module of an editable image layer splitting device disclosed in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention.
[0010] Explanation of reference numerals in the attached drawings: 401, acquisition module; 402, processing module; 403, output module; 501, processor; 502, communication bus; 503, user interface; 504, network interface; 505, memory. Detailed Implementation
[0011] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0012] In the description of the embodiments of the present invention, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a specific manner.
[0013] In the description of the embodiments of the present invention, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.
[0014] Existing image layer splitting typically extracts objects, text, and background separately through object detection, image segmentation, text recognition, and image completion to achieve independent editing of each visual element in a single bitmap. However, when using a solid color background to assist in object extraction, the background is usually removed based on the similarity between the pixel color and the background color. When the object itself has textures, reflections, or contour transition areas that are similar in color to the background, it is easy to accidentally delete them, resulting in partial missing parts of the object, holes, and abnormal edge colors, which affects the integrity of the separated object and the visual effect when it is subsequently recombined.
[0015] This invention discloses an editable image layer splitting method applied to a server. The server includes, but is not limited to, electronic devices such as mobile phones, tablets, wearable devices, and PCs (Personal Computers), and can also be a backend server running an editable image layer splitting method. The server can be implemented using a standalone server or a server cluster composed of multiple servers.
[0016] This embodiment discloses a method for splitting editable image layers, referring to... Figure 1 It includes the following steps: S110: Determine a unified canvas based on the acquired source image, perform visual structure analysis on the source image, and obtain the main subject area, text information, and background description.
[0017] S120: Generate an intermediate image containing the main subject and a predetermined solid color chroma key background based on the main subject area, and generate an adaptive alpha mask by combining the chroma key features of the intermediate image and the color protection information of the source image.
[0018] S130: The intermediate image is made transparent based on the adaptive alpha mask to obtain a transparent foreground layer.
[0019] S140: Generate a background repair mask based on the main subject area and the text area in the text information. Perform background repair on the area covered by the background repair mask according to the background description, and then combine the repaired area with the source image to obtain the background layer.
[0020] S150 generates an editable text layer based on the text information, and outputs a transparent foreground layer, a background layer, and an editable text layer according to the coordinate system corresponding to the unified canvas.
[0021] In practice, the source image is a composite bitmap to be split into layers, where the main subject, text, and background are already formed within the same pixel plane. After acquiring the source image, it is decoded, the image orientation is corrected based on EXIF information, and it is converted to RGBA format. The width and height of the actually decoded image are used to determine the unified canvas. The unified canvas is the spatial reference used by the main subject area, text information, and all output layers. Its size is consistent with the orientation-corrected source image, and the position of each area is represented according to the pixel coordinates corresponding to this unified canvas.
[0022] Visual structure analysis is performed on the source images after the above processing. Visual structure analysis refers to identifying the main visual objects, text, and background according to the semantics and spatial distribution of different visual content in the image, and forming a structured analysis result with clear positional relationships. During processing, people, goods, main animals, or large visual objects with independent movement, scaling, or replacement value in the source image are identified, and their corresponding subject identifiers, subject descriptions, pixel coordinates, and layer order are obtained. At the same time, visible text in the source image is identified, and the text content, text position, width, height, and corresponding font attributes and layer order are obtained. For image content other than the main visual objects and text, its texture, gradient, lighting, spatial relationships, and decorative features are extracted to form a background description. The background description is used to characterize the underlying image content in the source image that does not contain the main subject and text.
[0023] The identified main subject areas are then coordinate-corrected according to a unified canvas. Coordinates exceeding the canvas boundaries are confined to the canvas's boundaries, and invalid areas with zero width or height after correction are deleted. Next, each main subject area is filtered based on its proportion of the unified canvas area. Icons or embellishments with an area proportion below a preset threshold and lacking independent editing value are not considered main subjects; the remaining areas are determined as the main subject areas. The main subject area refers to the spatial extent of the main subject within the unified canvas, used to identify the positions for subsequent subject cropping and foreground layer generation.
[0024] Text information is data obtained by structurally recording the text content and its spatial and display attributes in the source image. It includes at least the text content, position, width, height, and layer order, and may also include font, font size, font weight, color, line spacing, character spacing, and rotation angle. The position of the text information is recorded according to the coordinates corresponding to the unified canvas, ensuring that the text content corresponds to its original layout in the source image. After the above processing, the main subject area, text information, and background description are all established with the unified canvas, respectively representing the main subject, text, and background content in the source image.
[0025] In one possible implementation, refer to Figure 2 Based on the source image reference region and the intermediate chroma key image, pixels similar to the predetermined chroma key are extracted from the source image reference region to form a similar color protection set, which is used to exclude the original similar color regions of the subject itself. At the same time, boundary-connected chroma key regions connected to the image boundary and globally high-confidence chroma key regions that meet the high-confidence chroma key conditions are determined from the intermediate chroma key image. The two are combined to exclude the similar color protection set, resulting in a hard removal set. Then, based on the hard removal set, its neighborhood is determined, and pixels that meet the local feathering conditions are identified in the neighborhood to form soft removal regions. The alpha value of each pixel is determined by combining the hard removal set and the soft removal regions to generate an alpha mask.
[0026] Specifically, when processing the main subject area, a reference area is formed by extending outwards from the outer boundary of the main subject area according to its width and height. The reference area is a region cropped from the source image that contains the main subject and a small amount of surrounding image content, used to retain the contextual information needed by the generative image editing model to recognize the main subject. The portion of the reference area that extends beyond the uniform canvas boundary is directly cropped, ensuring that the reference area always falls within the effective pixel range of the source image. Subsequently, the image content corresponding to the reference area is cropped and processed by the generative image editing model. The generative image editing model is an image processing model capable of regenerating local image content according to specified editing requirements. This step requires it to retain the main subject and its actual components, remove the original background, text, other objects, and non-subject visual content, while maintaining the main subject's structure, color, texture, proportion, and perspective essentially unchanged. The area outside the main subject is filled with a uniform, predetermined solid color key, thus obtaining an intermediate image. The predetermined solid color key is a pre-specified solid color used to distinguish the main subject from the generated background, and its color remains consistent throughout the subject processing.
[0027] After obtaining the intermediate image, the color of each pixel is determined relative to a predetermined pure color key. Color key distance indicates the proximity between the current pixel's color and the predetermined pure color key; the closer the pixel's color is to the predetermined pure color key, the smaller its color key distance. Channel advantage indicates whether the current pixel possesses the color channel characteristics corresponding to the predetermined pure color key. It can be used in conjunction with color key distance to avoid judging the background solely based on the similarity of color values. The same determination method is applied to the reference area of the source image to identify pixels that originally have colors similar to the predetermined pure color key, forming color protection information. This color protection information records similar color regions inherent to the main subject, such as the subject's surface color, reflective color, or texture color, ensuring that these regions are not directly deleted during subsequent color key removal due to color similarity. The formed color protection information can be appropriately extended to cover the edge pixels of similar color regions.
[0028] In the intermediate image, candidate pixels that are close to the predetermined pure color chromaticity are first identified based on chromaticity distance. Then, a connectivity search is performed from the outer boundary of the intermediate image inwards, identifying regions that can be continuously connected to the outer boundary through candidate pixels as boundary connected regions. Boundary connected regions are chromaticity regions that are continuously connected to the chromaticity background of the intermediate image, primarily used to identify the external pure color background normally generated by the generative image editing model. Simultaneously, a judgment is made across all pixels in the intermediate image, combining chromaticity distance and channel advantages, identifying pixels with both high chromaticity similarity and obvious chromaticity channel characteristics as high-confidence chromaticity regions. High-confidence chromaticity regions are regions that are not limited by whether they are connected to the image boundary and can be reliably identified as the predetermined pure color chromaticity, thus covering closed chromaticity blocks or residual color bands around the main subject that are not connected to the outer boundary.
[0029] After merging the boundary connected region and the high-confidence chromatic key region, pixels that need to be retained are excluded based on color protection information, resulting in the hard-removed region. The hard-removed region is a pixel region that can be determined to belong to the generated background with a relatively high degree of certainty, and its corresponding pixels are directly set to completely transparent in subsequent transparency processing. In order to clean up the small chromatic key pixels remaining around the hard-removed region, the hard-removed region can be extended within a limited range, but the pixels obtained from the extension still need to meet the chromatic key similarity condition and must not belong to the pixels corresponding to the color protection information. In this process, the boundary connected region is responsible for identifying the external continuous background, the high-confidence chromatic key region is used to supplement closed or isolated chromatic key regions, and the color protection information restricts the two types of chromatic key determinations from intruding into the original similar color parts of the main subject.
[0030] The soft removal region is further defined outside the hard removal region. The soft removal region is a transitional pixel area located at the boundary between the main subject and the chroma key background, which cannot be directly identified as complete background. It mainly includes anti-aliased pixels, semi-transparent edges, and pixels where the subject edge blends with the chroma key background. During processing, the neighborhood of the hard removal region is first determined. Pixels within this neighborhood that meet the more lenient chroma key criteria are searched, while pixels corresponding to color protection information are excluded. Pixels meeting the criteria are then identified as soft removal regions. Feathering is a color transition condition used to determine whether edge pixels enter the soft removal region. Its judgment range is wider than that of the hard removal region, allowing the transparency to gradually change at the edge of the main subject, rather than forming abrupt boundaries.
[0031] The alpha value of each pixel is determined based on the hard and soft removal regions. The alpha value represents the transparency of the corresponding pixel; completely transparent background pixels use the lowest alpha value, pixels preserved within the main subject use the highest alpha value, and pixels in the soft removal region have their alpha values determined by their chroma key distance, falling between the two. After generating the alpha values for all pixels, the alpha distribution at the edges of the main subject is slightly reduced and smoothed to decrease the thin color bands formed by the chroma key background at the anti-aliased edges, thus generating an adaptive alpha mask. The adaptive alpha mask records the transparency of each pixel in the intermediate image. Its generation is constrained by chroma key distance, channel dominance, boundary connectivity, and color protection information, and is subsequently used to perform transparency processing on the intermediate image.
[0032] In one possible implementation, when making the intermediate image transparent, an adaptive alpha mask is mapped to the intermediate image at the same pixel position. The hard-removed region corresponds to pixels belonging to a predetermined solid color chroma key background, while the soft-removed region corresponds to transition pixels between the main subject edge and the predetermined solid color chroma key background. Pixels within the hard-removed region are set to full transparency, while pixels within the soft-removed region have continuously varying alpha values based on their chroma key distance. Pixels not included in either the hard or soft-removed regions remain opaque, thus preserving the content within the main subject and creating a continuous transparent transition at the main subject edge.
[0033] Alpha value is a numerical value used to represent the transparency of a pixel. The lower the alpha value, the more transparent the corresponding pixel; the higher the alpha value, the less transparent the corresponding pixel. After determining the alpha value of each pixel, the main subject edges of the adaptive alpha mask are subjected to small-scale shrinkage and smoothing. This allows the fine chroma key edges formed by anti-aliasing to gradually transition to the transparent area, reducing the residual chroma key colors at the main subject outline. Anti-aliased edges are transitional pixel areas at the main subject outline formed by the mixing of the subject color and the background color; their color is usually not a single subject color or a predetermined pure chroma key.
[0034] For pixels in a semi-transparent state within the adaptive alpha mask, inverse chromatic key color restoration is performed on the corresponding positions in the intermediate image. Inverse chromatic key color restoration involves subtracting the background color component mixed in with a predetermined pure chromatic key from the current color of the semi-transparent pixel based on the alpha value, in order to restore edge colors that are closer to the original color of the main subject. During processing, the influence ratio of the predetermined pure chromatic key on the pixel color is determined based on the transparency of the current pixel, and the corresponding color channel is corrected accordingly. The corrected color is limited to the effective color range. After this processing, when the main subject is superimposed on other backgrounds, the colored edges formed by the original predetermined pure chromatic key at the semi-transparent contours can be reduced.
[0035] After completing the inverse chroma key color restoration, chroma key residue detection is performed on the transparent visible area. The visible area is the pixel area where the alpha value is not fully transparent and will still participate in the image display; chroma key residue refers to pixels in the visible area that still have obvious predefined pure color chroma key color characteristics. The transparency result is judged based on the amount of chroma key residue in the visible area. When the chroma key residue exceeds the allowable range, a secondary cleaning constrained by color protection information is performed on the corresponding area, and inverse chroma key color restoration is performed again to avoid deleting similar colors of the main subject along with the residual chroma keys.
[0036] After secondary cleaning, the corrected alpha value is written to the alpha channel of the intermediate image, while retaining the main subject color after the inverse chroma key color restoration, forming a foreground image in RGBA format. The alpha channel is used in RGBA images to record the transparency of each pixel; the RGBA format adds an alpha channel in addition to the red, green, and blue color channels. The foreground image is then configured according to the position and size corresponding to the main subject area, maintaining a coordinate relationship consistent with the unified canvas, thus obtaining a transparent foreground layer.
[0037] In one possible implementation, refer to Figure 3First, the main subject area and text area are determined, and then the main subject area and text area are expanded respectively. The expanded areas are merged to form a background repair mask. Then, based on the background repair mask, the main subject and text locations within its coverage area are repaired, generating candidate repair images. In the local compositing stage, only pixels in the candidate repair images that are within the coverage area of the background repair mask are allowed to participate in the compositing. Pixels outside the coverage area of the background repair mask continue to use the original pixels at the corresponding positions in the source image, so that the unrepaired areas remain unchanged, and finally the background layer after removing the main subject and text is obtained.
[0038] Specifically, when processing the background repair mask, a mask with the same width and height as the unified canvas is first created, ensuring a one-to-one correspondence between the pixel positions in the mask and the source image. The mask marks the spatial range within which subsequent background repair is permitted; its covered area indicates the location where the main subject or text needs to be removed and the background content regenerated, while the original content of the source image is preserved outside the covered area. Based on the aforementioned visual structure analysis, the main subject area and the text area within the text information are expanded. The subject safety margin is an expansion range reserved around the main subject area, used to cover the projection, reflection, rim light, and edge deviations caused by the subject's positioning adjacent to the main subject. The text safety margin is an expansion range reserved around the text area, used to cover the strokes, shadows, and glowing content outside the text strokes. The subject safety margin and text safety margin are set separately, and the expanded areas are all limited to the unified canvas area. All expanded areas are then merged and written into the mask to obtain the background repair mask. The boundaries of the expanded areas can be rounded to reduce the obvious blocky transitions formed by regular rectangular boundaries after background repair.
[0039] After obtaining the background inpainting mask, the source image, the background inpainting mask, and the background description formed by the aforementioned visual structure analysis are used together for background inpainting. The background description is a structured representation of the background content in the source image that does not contain the main subject or text, including background texture, perspective, gradients, lighting, and decorative elements, used to constrain the generated content of the area to be filled. When processed by the generative image editing model, the background inpainting mask defines the area to be repaired, and the background texture, perspective, gradients, lighting, and decorative elements that connect with the surrounding area are restored based on the background description. Simultaneously, the main subject, text, and other foreground content are restricted from being retained or added to the repair area, resulting in a generated image. The generated image is a candidate background image formed after the generative image editing model completes the background inpainting; its size and coordinate position correspond to the unified canvas.
[0040] The generated image does not directly replace the source image entirely; instead, the compositing area is determined based on the background inpainting mask. The compositing area is the region where the generated image is allowed to participate in the formation of the final background layer, corresponding to the coverage area of the background inpainting mask. The boundaries of the compositing area are feathered. Feathering is a process that gradually changes the blending degree of the compositing area's edges from the inside out, used to reduce the appearance of obvious seams between the generated content and the original background of the source image. Feathering only works near the edges of the compositing area; the inner areas far from the boundaries still use the background-inpainted content.
[0041] After feathering the edges, the generated image and the source image are locally composited according to the positions corresponding to the background repair mask. Local compositing is a process that only allows the generated image to replace or blend corresponding pixels of the source image within the composite area; pixels outside the composite area retain their original pixels at the corresponding positions in the source image. For pixels at the feathered boundaries, a transition is created between the generated and source images based on the boundary gradient, ensuring a continuous connection between the repaired content and the original background. The composited image serves as the background layer, thus replacing the main subject and text with the repaired background content, while areas not included in the background repair mask retain their original source image pixels.
[0042] Furthermore, when creating an editable text layer based on text information, the text content, position, width, height, and layer order recorded in the text information are read and structurally saved according to the coordinate relationship of a unified canvas. Text information is data recorded from the content, spatial position, and display attributes of visible text in the source image. In addition to text content and position, it may also include font, font size, font weight, color, alignment, line spacing, character spacing, and rotation angle. An editable text layer is a layer formed by converting the text in the source image from its original pixel content into independently modifiable text data. The text content and display attributes can be edited separately, instead of being saved as fixed pixels in a background layer. For multiple identified texts, their respective positions, sizes, and layer orders are preserved, ensuring that the original arrangement is maintained when the texts are rearranged.
[0043] The transparent foreground layer corresponds to the main subject separated from the source image. Each main subject is represented as an RGBA format image, and its position, width, height, and layer order within the unified canvas are recorded. The transparent foreground layer is an image layer whose area outside the main subject has transparent properties and can be moved or replaced independently. Its transparency is determined by the aforementioned adaptive alpha mask. The background layer is the bottom layer image without main subjects or text, formed after background restoration. Its canvas size is consistent with the unified canvas, and it is set at the bottom of all layers.
[0044] When outputting layers, a common coordinate system is established using a unified canvas. This coordinate system serves as a reference for determining the spatial positions of transparent foreground layers, background layers, and editable text layers within the same canvas. The position coordinates of each layer are recorded according to the width, height, and origin of the unified canvas. The unified canvas is a common layer size determined by the actual decoded width and height of the source image. Subsequent layers use this size as a reference for position and size, allowing the main subject, text, and background to be recombine according to their original spatial relationships.
[0045] For transparent foreground layers smaller than the unified canvas, it's not necessary to expand the main subject image to the same size as the unified canvas. Instead, record the subject's position, width, and height, and position it accordingly on the unified canvas. Editable text layers are similarly configured according to the position and size specified in the text information, while background layers maintain the same full size as the unified canvas. Each layer can also save its layer order to determine the overlapping relationships between different layers during recombination.
[0046] After organizing the layers, output one or more transparent foreground layers, one background layer, and one editable text layer, and associate each layer with the same unified canvas and coordinate system. The output information can also record the product identifier, generation method, rollback status, and rollback reason for each layer. The rollback status indicates whether the corresponding layer used a backup processing result due to generation anomalies, and the rollback reason records the specific anomaly type that triggered the backup processing. The resulting layers can be recombine according to layer order, while retaining the ability to independently move, scale, or replace the main subject and independently modify the text content.
[0047] Furthermore, when generative image editing models participate in layer splitting, the model output has a certain degree of uncertainty. Besides the possibility of failing to generate a predetermined pure color chromaticity background, forming locally closed chromaticity regions, outputting a completely transparent image, or failing to process a single main subject, for main subjects with transmissive properties such as glass, transparent packaging, and semi-transparent decorations, the problem of closed chromaticity regions being confused with truly transparent areas can easily arise. After the generative image editing model removes the original background and replaces it with a predetermined pure color chromaticity background, the predetermined pure color chromaticity may be displayed through the transparent areas of the main subject, forming closed chromaticity regions inside the main subject's outline that are not connected to the external background. Simultaneously, chromaticity remnants generated by abnormal model processing may also exhibit the same closed shape. Existing methods for background determination based on boundary connected regions and high-confidence chromaticity regions can identify regular chromaticity backgrounds connected to image boundaries and remove high-confidence chromaticity regions not connected to boundaries, but cannot determine solely based on connectivity whether closed chromaticity regions belong to the background normally seen through transparent materials or chromaticity remnants generated by abnormal model processing. If all closed chromaticity regions within the main subject are uniformly classified as hard-removed areas and set to full transparency, the transparent areas that originally possessed some transmission, refraction, or reflection characteristics will be directly hollowed out, forming transparent holes that do not conform to the actual structure of the main subject. If they are uniformly retained to avoid damaging the transparent structure, abnormal chromaticity will remain in the transparent foreground layer, and the chromaticity colors will reappear when the transparent foreground layer is overlaid on other backgrounds. Therefore, in this type of scenario, it is necessary to simultaneously utilize the transparent areas of the main subject in the source image and their transparency information to match the position and transparency features of the closed chromaticity regions. While maintaining the true transparent structure, closed chromaticity regions that are inconsistent with the true transparent features are treated as abnormal chromaticity.
[0048] In one possible implementation, the original pixels of the source image are first preserved within the main subject area, and a local analysis range is established according to the coordinate relationship corresponding to a unified canvas. Contours within the main subject area are extracted, and connected component analysis is performed on contours that form complete closed boundaries. Small holes, noise, and scattered closed contours formed by compression artifacts are removed to obtain closed regions. A closed region refers to a region whose boundary is continuously closed within the main subject area and whose interior forms an independent pixel range. This region may correspond to the part of transparent material where the background can be observed, or it may correspond to closed textures or solid holes inherent in the main subject itself. During processing, it is not directly identified as a transparent region based on its closed shape, but rather the image structure inside and outside the closed region is used for further judgment.
[0049] For each closed region, several analysis paths are extended from its boundary outwards from the main subject region. Local pixel blocks are read from within the closed region, near the main subject boundary, and in the outer background. The outer background refers to the range of background pixels located outside the main subject region and not occupied by text or other main subject regions. If an analysis path intersects with a text region or other main subject region, that path is discarded to avoid comparing other foreground content as background. For the remaining analysis paths, local edge direction, grayscale gradient, texture changes, and color distribution are extracted to determine whether the content displayed in the closed region can continuously extend from the outer background into the main subject.
[0050] Texture continuity is used to characterize whether the texture arrangement in the outer background continues to appear in a similar direction and spatial period within a closed region after passing the main subject boundary. To determine texture continuity, texture features of local pixel blocks on both sides of the analysis path can be compared. If lines, grains, stripes, or other texture structures in the outer background still have corresponding arrangement directions and relative positions within the closed region, the path is considered to have texture continuity. For backgrounds with repeating textures, multiple adjacent analysis paths can be combined for judgment to avoid misjudgment caused by accidental similarity of a single local texture.
[0051] Contour continuity is used to characterize whether edges or linear structures in the outer background maintain a traceable extension relationship after entering the main subject area. For example, when there are continuous straight lines, arcs, or object contours in the background, the position and direction of the contour on both sides of the main subject are determined, and then it is determined whether there are contour segments inside the closed area that connect with the contours on both sides and have continuous directional changes. If the contours on both sides can form a continuous path through the contour segments inside the closed area, it indicates that the closed area may display background structures that can be seen through transparent materials; if the contour is completely interrupted after entering the boundary of the main subject, or if the direction of the internal contour does not correspond to the outer background, the condition for the closed area to be identified as a transparent area is reduced.
[0052] Color change relationships are used to characterize whether the color shift, brightness attenuation, or saturation change of the background color after passing through the main transparent material has continuity. During processing, the color changes of pixels inside the closed region and the background pixels outside are compared along the same analysis path. When the color change trend of the outer background can continue to the closed region, only showing a stable decrease in brightness, color shift, or weakened contrast overall, the color change can be considered to conform to the transmission characteristics of a transparent material. If the color inside the closed region does not have a corresponding change relationship with the outer background, and is mainly composed of stable solid colors or independent textures, it is not considered a transparent region.
[0053] The closed regions are determined by combining texture continuity, contour continuity, and color variation. When the same closed region shows continuous extension of the outer background structure across multiple analysis paths, and the interior of the closed region primarily exhibits color attenuation, refraction shift, or localized reflection relative to the outer background, rather than forming an independent solid texture, the closed region is defined as a transparent region. A transparent region refers to an area that is part of the main subject but allows background content to be displayed through its material; it remains part of the main subject area and is not removed from it.
[0054] After identifying the transparent area, the background reference area is determined based on the aforementioned effective analysis path. The background reference area is a pixel range selected from the outer background that corresponds to the visible background content within the transparent area. It is used to estimate the impact of the transparent material on the background color and structure. During determination, the search proceeds outward along the direction of high texture or contour continuity within the transparent area, selecting local areas outside the main subject area that are not obscured by text areas, other main subject areas, or obvious foreground decorations. For a single transparent area, multiple background reference areas can be established at different locations, and then filtered according to the degree of correspondence between each background reference area and the internal structure of the transparent area.
[0055] Texture preservation indicates the degree to which texture features in the background reference area are retained after passing through the transparent area. During processing, the transparent area is divided into several local regions, and similar or identical texture structures are searched in the corresponding background reference areas. Texture direction, local edge density, and texture contrast are compared. When the background texture remains relatively clear in the transparent area, it indicates that the translucency at that location is relatively high; when the background texture is only weakly recognizable, it indicates that the main subject material strongly occludes the background content.
[0056] Contour continuity indicates the degree to which a contour in the background reference area can maintain continuous extension after passing through a transparent region. It is determined by comparing the directional changes, positional offsets, and traceable lengths of the contour before and after entering the transparent region, focusing on contours that can cross the transparent region. Slight refraction caused by glass or transparent packaging allows for a certain positional offset of the contour within the transparent region, as long as the overall contour still corresponds to the background reference area, thus preserving its continuity.
[0057] Based on this, transparency information is determined. Transparency information represents the degree of transparency of background content at different locations within the transparent area, and is stored in association with the transparent area according to the pixel positions corresponding to the unified canvas. For locations in the background reference area where textures and outlines can be displayed relatively completely in the transparent area and where color changes are minimal, a higher degree of transparency is determined; for locations where the background structure is significantly weakened, while the main subject's own reflection, material color, or surface texture is more obvious, a lower degree of transparency is determined. A single degree of transparency is not required within the transparent area; continuously varying transparency information can be formed according to local pixel ranges, thereby preserving the differences between locations such as glass edges, packaging corners, and reflective strips and the central transparent area.
[0058] Before generating the intermediate image, the transparent areas and transparency information are transformed to a reference area corresponding to the main subject area using a unified canvas coordinate system. When processing the main subject, the generative image editing model retains the original structure and material of the main subject in non-transparent areas, while preserving the boundaries, reflections, refractions, and material colors of transparent areas. Simultaneously, the pre-defined solid color chromaticity background generated outside the main subject is extended behind the transparent areas. In the resulting intermediate image, pre-defined solid color chromaticity may appear within the transparent areas, but these chromaticity keys are processed differently from the chromaticity background outside the main subject; that is, the chromaticity keys within the transparent areas need to be further evaluated in conjunction with the transparency information.
[0059] When performing chroma key feature detection on the intermediate image, the detection range is first divided according to the transparent areas. For pixels outside the transparent areas, it is further determined whether they belong to the chroma key to be removed based on chroma key distance, channel dominance, boundary connectivity, and color protection information. For pixels within the transparent areas, they are not directly included in the hard removal area because their color is close to the predetermined pure color chroma key. Instead, the transparency information corresponding to the same position in the source image is read, and it is determined whether the position distribution of the current chroma key pixel matches the transparent area.
[0060] When a closed chromatic key region appears within a transparent region, its position is matched with that of the transparent region. If the closed chromatic key region is located within the transparent region, its boundary range roughly corresponds to the transmission range of the transparent region, and the corresponding position has a high degree of transmission in the source image, then the closed chromatic key region is considered to mainly originate from the background of a predetermined pure color chromatic key displayed through the transparent material. It is not directly removed as ordinary chromatic key residue, but its corresponding alpha value is determined according to the transparency information. If the closed chromatic key region extends beyond the transparent region, or the corresponding position does not have background transmission characteristics in the source image, then the mismatched portion is treated as an abnormal chromatic key, and its alpha value is determined by combining chromatic key characteristics and color protection information.
[0061] For cases where a closed chroma key region falls only partially within a transparent region, it is segmented along the boundary of the transparent region. The portion within the transparent region that matches the transparency information retains continuous alpha values according to the transparency information; the portion exceeding the transparent region is further determined based on chroma key features. If the hard removal condition is met, it is set to full transparency; if it is located in the neighborhood of the hard removal region and meets the feathering condition, a transitional alpha value is formed using a soft removal method. This avoids uniformly clearing the entire closed chroma key region and also prevents the overall retention of abnormal chroma keys even if some pixels correspond to truly transparent materials.
[0062] After the above processing, the alpha value corresponding to the transparent area, the alpha value determined based on the chromatic affinities outside the transparent area, and the color protection information are jointly written into the adaptive alpha mask. For pixels in the source image that are inherently similar in color to the main subject, the color protection information continues to prevent them from being misidentified as chromatic affinities background. For predetermined pure color chromatic affinities that normally show through within the transparent area, their transparency is controlled by the transparency information. For closed chromatic affinities that do not match the transparent area, they are processed according to the conventional chromatic affinities removal rules. Thus, the transparent areas and transparency information identified in the source image can be passed all the way to the chromatic affinities judgment and alpha value determination stage of the intermediate image.
[0063] In one possible implementation, when processing the main subject, the coordinate range of the main subject area within a unified canvas is used as a reference. Preset distances are then extended outwards both horizontally and vertically. The portion extending beyond the unified canvas is truncated, thus forming a reference area. The reference area continues to use the same coordinate reference as the source image and retains a certain range of original background content around the main subject, enabling the generative image editing model to identify the main subject by combining the context near its outline. The transparent areas identified by the aforementioned visual structure analysis are recorded using unified canvas coordinates. Therefore, coordinate offsets can be performed based on the upper left corner position of the reference area, transforming the transparent areas to the corresponding local coordinates of the reference area. If the reference area is scaled proportionally before being sent to the generative image editing model, the coordinates of the transparent areas are simultaneously transformed according to the same scaling relationship. After the model completes processing and restores the original size, the transparent areas are then restored to the coordinate positions corresponding to the reference area.
[0064] After cropping the image content corresponding to the reference region from the source image, a generative image editing model is used to process the cropped image. During processing, the main subject and its real components are preserved, and constraints are placed on the original structure, color, texture, proportion, and viewpoint of the main subject. Simultaneously, the original background, text, icons, other objects, and lighting effects not belonging to the main subject are removed from the reference region, and the area outside the main subject is set to a uniform, predetermined solid color chromaticity background. For transparent areas already mapped to the reference region, they are not directly deleted as ordinary backgrounds; instead, the contours, refractions, reflections, and color changes formed by the transparent material are preserved, ensuring that the predetermined solid color chromaticity background appears at the transmission position corresponding to the transparent area. After the model returns the result, it is restored according to the original size of the reference region to obtain an intermediate image, ensuring that the intermediate image, reference region, and transparent region correspond in pixel position.
[0065] Subsequently, chromatic key feature detection is performed on the intermediate image. For each pixel in the intermediate image, its chromatic key distance and channel dominance are determined. Chromatic key distance is used to determine how close a pixel's color is to a predetermined pure color chromatic key, and channel dominance is used to further determine whether it possesses the color channel features corresponding to the predetermined pure color chromatic key. Pixels that simultaneously meet the preset chromatic key distance and channel dominance conditions are marked as high-confidence chromatic key pixels, and then merged according to the spatial adjacency relationship between pixels to form high-confidence chromatic key regions. For cases where the main subject itself contains pink, purple, or other colors similar to the predetermined pure color chromatic key, color protection information continues to participate in the screening, protecting similar color pixels in the source image that originally belonged to the main subject, preventing these pixels from being classified as chromatic keys to be deleted simply because they meet the color conditions.
[0066] When performing boundary connectivity detection on high-confidence chroma key regions, the search begins from the image boundaries surrounding the central image. Regions that can be continuously reached from the image boundary through adjacent high-confidence chroma key pixels are identified as boundary connected regions. In practice, flooding filling or connected component search can be used, maintaining consistent pixel adjacency rules throughout the search process. Solid-color chroma key backgrounds generated by generative image editing models outside the main subject are usually connected to the image boundary and thus enter the boundary connected regions; however, chroma key pixels surrounded by the main subject's outline will not connect to the image boundary through high-confidence chroma key pixel paths.
[0067] After determining the boundary connected regions, the remaining parts of the high-confidence chroma key regions are divided into connected components. The portions not included in the boundary connected regions but forming continuous pixel regions are identified as closed chroma key regions. Not all closed chroma key regions are directly deleted as anomalous chroma keys because they may have two origins: one is chroma key remnants formed by incomplete cleanup by the generative image editing model; the other is chroma key regions formed when a predetermined pure color chroma key background appears inside the main subject after passing through transparent areas such as glass or transparent packaging. Both types of regions may appear as high-confidence chroma key regions that are not connected to the image boundaries in the intermediate image; therefore, the aforementioned transparent regions and transparency information are used for further differentiation.
[0068] During position matching, the pixel coordinates of each closed chroma key region are spatially compared with the mapped transparent region. For closed chroma key regions that do not overlap with the transparent region at all, they are determined not to belong to the background that is normally visible through the transparent material and are identified as hard-removed regions. For closed chroma key regions that only partially enter the transparent region, they are divided along the boundary of the transparent region; the portion outside the transparent region is identified as a hard-removed region, while the portion inside the transparent region is further judged according to the transparency information. This approach does not uniformly provide deletion results for a single connected component, but rather allows the same closed chroma key region to form different processing parts based on its positional relationship with the transparent region.
[0069] For closed chromatic affinities falling within the transparent region, the corresponding alpha value is determined based on the transparency information associated with the transparent region in the source image. During processing, the transparency information corresponding to each pixel in the closed chromatic affinity region is searched according to the established coordinate relationship between the pixels in the intermediate image and the pixels in the source image. If the corresponding position belongs to a part with obvious background transmission characteristics in the source image, the pixel is not set to full transparency, but rather the corresponding partial transparency state is retained according to the transparency information; the transparency information indicates that the main material has a higher degree of occlusion, and the retention level is increased accordingly, while the transparency level remains high for positions with obvious background transmission. In this way, the alpha value inside the transparent region changes continuously with the original transmission change of the transparent material, rather than becoming completely transparent simply because a high-confidence chromatic affinity appears in the intermediate image.
[0070] For pixels located within a transparent area but whose chromaticity distribution is significantly inconsistent with the transparency information, anomaly detection is still required. Specifically, the coverage of a closed chromaticity region within the transparent area can be compared with the transmission range represented by the transparency information. If a continuous chromaticity portion extends to a location that originally lacked background transmission characteristics, or remains continuous even after crossing the boundary of the transparent area, the corresponding mismatched portion is reclassified into the hard removal area. This avoids the erroneous retention of randomly generated chromaticity fragments within a transparent subject by generative image editing models simply because they happen to be located near a transparent area.
[0071] After identifying the hard-removed region, the corresponding pixels within it are set to full transparency. For portions of the transparent region that have undergone position matching and match the transparency information, the previously determined alpha value is used. Pixels near the edge of the hard-removed region but not directly identified as complete background can continue to form a continuous transparent transition using existing soft-removed processing methods. Color protection information is still used as an exclusion condition at this stage; if a pixel belongs to a similar color region present in the main subject in the source image, it will not be directly set to full transparency simply because the chromaticity distance in the intermediate image is small. Existing processing uses a joint determination method for the relationship between boundary connected chromaticity, high-confidence chromaticity, color protection, and local soft edges, which can continue to serve as the basis for the above-mentioned region processing.
[0072] After determining the alpha values for various pixels, an adaptive alpha mask is created based on the pixel arrangement of the intermediate image. Hard-removed areas provide fully transparent pixels corresponding to the background, transparent areas provide partially transparent pixels determined by transparency information, soft-removed areas provide continuous alpha values at the edges of the main subject, and color protection information restricts the incorrect transparency of inherent similar colors of the main subject. Pixels of the main subject not covered by the above areas remain opaque. The final adaptive alpha mask maintains the same size and pixel coordinates as the intermediate image, allowing subsequent transparency processing to directly write alpha values at the corresponding positions, while retaining abnormally closed chroma keys and chroma keys that are normally visible through transparent materials in different processing paths.
[0073] In one possible implementation, when making the intermediate image transparent, the size and pixel coordinates of the intermediate image and the adaptive alpha mask are first kept consistent, so that each alpha value in the adaptive alpha mask corresponds to the same pixel position in the intermediate image. The hard-removed region, transparent region, closed chroma key region, and transparency information determined in the previous step are read and processed separately for each region. For pixels in the hard-removed region that are not in the transparent region, this part can be identified as a predetermined solid color chroma key background generated by the generative image editing model; therefore, the corresponding alpha value is set to full transparency, so that the predetermined solid color chroma key background no longer participates in the display of the transparent foreground layer. For pixels that are not in the hard-removed region but belong to the main subject entity, they remain opaque, thereby preserving the main subject's own color, texture, and structure.
[0074] A certain neighborhood is defined around the hard-removed region. For pixels within this neighborhood, the aforementioned chroma key distance is read, and pixels close to a predetermined pure color chroma key but insufficient to be directly identified as background are selected according to feathering conditions, forming a soft-removed region. The soft-removed region is mainly located between the outer contour of the main subject and the predetermined pure color chroma key background; its pixels often contain both the subject color and the background color. For the soft-removed region, full transparency is not used; instead, continuous alpha values are determined based on the chroma key distance of each pixel. The closer a pixel is to the predetermined pure color chroma key, the higher its transparency; as the pixel color gradually approaches the main subject's own color, the corresponding transparency gradually decreases. Pixels corresponding to color protection information are still excluded when determining continuous alpha values to avoid incorrect feathering of similar colors originally present at the edge of the main subject due to their location in the neighborhood of the hard-removed region. Existing processing chains also restrict low-confidence chroma key pixels from entering the soft-removed region within the neighborhood of the hard-removed region before forming continuous alpha changes.
[0075] For pixels simultaneously located within both transparent and hard-removed regions, the fully transparent result corresponding to the hard-removed region is not directly adopted. Instead, the position matching result between the closed chroma key region and the transparent region formed in the previous step is read. This matching result continues to use the result obtained from the aforementioned processing, i.e., determining whether the closed chroma key region corresponds in spatial location and coverage to a transparent region with background transmission characteristics in the source image. If the corresponding pixel belongs to the positionally matched portion, it is considered that the predetermined pure color chroma key appearing at that location may be formed by the predetermined pure color chroma key background passing through the transparent material of the main subject. Instead of directly deleting it as a normal background pixel, the transparency level at the corresponding location is obtained from the transparency information, and the Alpha value of that pixel is re-determined accordingly.
[0076] Transparency information is associated with transparent areas using a unified canvas coordinate system. Therefore, when processing intermediate images, the corresponding transparency information can be obtained based on the coordinate correspondence between the intermediate image and the reference area. For locations where transparency information indicates significant background transmission, the corresponding pixels maintain a high degree of transparency but do not become completely transparent. For locations where transparency information indicates a high proportion of reflection, color, or surface structure of the transparent material itself, the corresponding pixels retain more of the main subject content. After this processing, an alpha distribution that varies with the material's transmission state can be formed within the transparent area. The glass edges, refractive positions, and localized reflective positions will not have the exact same level of transparency as the center of the glass.
[0077] If a pixel is located within a transparent area, but the position matching result of its closed chromatic affinities does not match with the transparent area, then its hard removal state is maintained. Specifically, the closed chromatic affinities can be separated according to the position matching boundary formed in the previous step. The portion within the matching range has its alpha value re-determined based on the transparency information, while the portion outside the matching range or corresponding to a non-transparent position in the source image remains fully transparent. In this way, even if the same closed chromatic affinities region contains both real background transmissive portions and abnormal chromatic affinities, they can be processed separately within each pixel, rather than being retained or deleted entirely simply because they belong to the same connected region.
[0078] After configuring the alpha values as described above, inverse chroma key color restoration is performed on semi-transparent pixels with alpha values between full transparency and full opacity. These semi-transparent pixels include not only soft-removed areas on the main subject's outline but also pixels in transparent areas that have achieved partial transparency based on transparency information. Since the intermediate image is generated by a generative image editing model against a predetermined solid color chroma key background, the color currently stored in the semi-transparent pixels may simultaneously contain color components of both the main subject's color and the predetermined solid color chroma key. During processing, the component of the predetermined solid color chroma key in the current pixel's color is estimated based on the corresponding alpha value. This color component is then subtracted from the current pixel, and the processing results for each color channel are effectively limited to a specific range, thereby restoring the foreground color that the main subject actually participates in display. This processing is consistent with the inverse chroma key color restoration in the original chroma key processing chain, reducing the likelihood of chroma key color outlines reappearing after transparent foregrounds are superimposed on other backgrounds.
[0079] For transparent areas, color restoration using inverse chromatic affinities preserves the color shifts, reflections, and refractions inherent in the transparent material itself, rather than uniformly restoring all colors in the transparent area that are close to the predetermined pure chromatic affinities to the main body color. The processing is still based on transparency information: if the transparency information indicates that the current location primarily reflects background transmission, the focus is on eliminating the predetermined pure chromatic affinities introduced during the generation of the intermediate image; if the transparency information indicates that the current location has a distinct main material color, then the corresponding material color components are retained. This avoids treating the original translucent glass as a uniformly colored solid area after inverse chromatic affinity color restoration.
[0080] After completing the inverse chroma key color restoration, chroma key residue checks are performed on pixels in the intermediate image with alpha values greater than those in the fully transparent state. For visible pixels outside the transparent areas, the presence of significant predefined pure color chroma key residue is further determined based on chroma key distance and channel advantage. If the residue meets the preset cleanup conditions, the corresponding residue area is expanded to a limited extent and cleaned again under the constraints of color protection information, and the inverse chroma key color restoration is performed again. Existing processing methods also re-detect high-saturation chroma key residue in the visible areas and perform secondary cleanup when it exceeds the allowable range.
[0081] For chroma key remnants within transparent areas, a corresponding check of transparency information is added. Transparency information at the same location in the source image is read using uniform coordinates, and the current alpha value is compared to the original transparency level at that location. If the transparency information indicates that the current location is partially transparent, but the current alpha value has been processed to be fully transparent or nearly fully transparent, and this abnormal state occurs continuously in a localized area, then the aforementioned hard removal or chroma key cleanup is deemed to have over-processed the transparent area, and the corresponding pixel is restored to the alpha value determined according to the transparency information. Conversely, when the transparency information indicates that there is no significant background transmission at the current location, but continuous predetermined pure color chroma key pixels are still retained in the intermediate image after transparency, this part is identified as an abnormal chroma key remnant, and the alpha value is reset using the hard removal method. Through this check, transparency information not only participates in the previous closed chroma key determination but also continues to be used for correcting the results after transparency.
[0082] Near the boundaries of transparent areas, it is also necessary to check for abrupt changes in the alpha value. If there is a significant jump between the alpha value determined according to the transparency information within the transparent area and the adjacent main body, a limited range of smoothing can be performed within the boundaries of the transparent area, allowing the transparency to gradually change with the edge structure of the transparent material. This smoothing range does not cross the already determined aberration chromaticity boundaries, avoiding the re-mixing of background pixels that should be fully transparent into the transparent area. At the same time, small-scale shrinkage and smoothing can continue to be performed on the adaptive alpha mask at the outer contour of the main body to clean up the fine color edges caused by anti-aliasing.
[0083] After the above verification and correction, the final Alpha value is written to the Alpha channel of the corresponding pixel in the intermediate image, while retaining the color channel data after the inverse chroma key color restoration, forming the RGBA image corresponding to the main subject. If the intermediate image was enlarged before being sent to the generative image editing model due to model processing size requirements, the processed RGBA image is restored to the size corresponding to the reference area according to the original ratio, while maintaining the correspondence between the transparent areas and the main subject outline with the original coordinate positions. Existing processing methods require that the model be re-placed into a transparent canvas of the original subject outline size after returning to avoid changes in the final position of the subject.
[0084] Finally, based on the position of the main subject area on the unified canvas, the processed RGBA image is positioned accordingly, and the position, width, height, and layer order of the main subject area are recorded to form a transparent foreground layer. For cases where the source image contains multiple main subjects, the above transparency process is performed on each main subject separately, forming a transparent foreground layer for each main subject. All transparent foreground layers are saved using the same unified canvas and coordinate system. In this way, the closed chromaticity region, transparent region, and transparency information determined in the previous step can continuously participate in hard removal, soft removal, inverse chromaticity color recovery, and chromaticity residue verification. Ultimately, the transparent foreground layer removes abnormal chromaticity backgrounds while retaining the inherent transparent structure of the transparent material itself.
[0085] In one possible implementation, when processing the background restoration mask, a single-channel mask with the same width and height as the unified canvas is first created, ensuring that the position of each pixel in the mask matches the corresponding position in the source image. The boundary coordinates of the main subject area and the text area are read separately. The main subject area is expanded outward according to the subject safety margin, and the text area is expanded outward according to the text safety margin. The subject safety margin is used to cover any residual shadows, reflections, rim lights, and region recognition deviations outside the main subject outline. The text safety margin is used to cover the strokes, shadows, and glow effects around the text strokes. When the expanded area exceeds the unified canvas, the excess portion is cropped to the unified canvas boundary. The edges of each expanded area can be rounded. All expanded areas are then merged and written into the single-channel mask to form the background restoration mask. The background restoration mask is a spatial mask used to define the range of background reconstruction. The covered positions indicate the positions where background content is allowed to be regenerated, while the remaining positions retain the original pixels of the source image. The existing processing method also expands and merges the main subject area and text area separately, and uses differentiated safety margins to cover the subject projection, text outline and adjacent visual effects.
[0086] For a main subject containing transparent areas, when expanding the main subject area, the transparent areas are not removed from the background repair mask. Instead, the transparent areas and the solid parts of the main subject are both within the coverage of the background repair mask. Although the transparent areas can display some background content, their pixels still contain reflections, refractions, color shifts, and surface textures formed by the transparent material itself, and cannot be directly retained as the final background layer. If only the solid parts of the main subject are removed while the transparent areas are retained, glass outlines, packaging reflections, or refraction marks are likely to remain in the background after removing the main subject. Therefore, when generating the background layer, background repair is still performed on the corresponding positions of the transparent areas, but before repair, the visible background content is used to restore the background structure before it was obscured by the main subject.
[0087] When extracting background structure information from the transparent areas of the source image, pixels with effective background transmission characteristics are first selected based on the aforementioned transparency information. Background structure information refers to the spatial structural features that can be identified from the transparent areas and used to constrain background restoration, including the extension direction of background textures, contour direction, line positions, gradient directions, brightness variations, and local color change trends. For locations where the transparency information indicates a high degree of background transmission, more visible background features are retained; for locations with high reflectivity, refraction, or a high proportion of the main material's color, the participation of these locations in background structure extraction is reduced to avoid mistaking main features such as glass highlights or packaging color casts for background structure.
[0088] In specific processing, the local textures and contours within the transparent area can be tracked along the continuous background direction obtained when determining the transparent area. If a background line enters the transparent area from one side of the main subject and an edge in a similar direction can be continuously detected within the transparent area, extending from the other side of the main subject, then the extension position and direction of the line within the transparent area are recorded. For textured backgrounds, the direction of texture arrangement, interval changes, and local intensity relationships that can still be observed within the transparent area are recorded. For gradient or large-area smooth backgrounds, the main directions of color and brightness changes are recorded. During the extraction process, the reliability of each location is limited by transparency information; the more obvious the background transmission, the higher the degree of preservation of the corresponding background structure information.
[0089] For transparent areas with significant refraction, perfect pixel alignment between the outline within the transparent area and the outer background is not required. The overall continuity trend can be determined based on the corresponding background outlines on both sides of the transparent area. Small-scale positional shifts within the transparent area are treated as refraction changes caused by the transparent material, and restored to a structure continuous with the outer background when forming background structure information. Isolated bright spots, localized color bands, or highlight edges that cannot establish a correspondence with the outer background of the transparent area are not included in the background structure information to avoid retaining the transparent material characteristics of the main subject after background restoration.
[0090] The background description continues to utilize the results of the aforementioned visual structure analysis, describing the overall content of the background that does not contain the main subject or text. This includes the background texture type, perspective, gradients, lighting conditions, and decorative elements. The background structure information differs from the background description. The background description provides overall constraints on what background content should be generated for the entire area to be repaired, while the background structure information provides the local positional relationships and continuous structures that can already be observed within the transparent areas. During background repair, both are associated with the source image and the background repair mask, allowing the generative image editing model to reconstruct the background within the coverage area of the background repair mask. Existing background repair methods already require the recovery of background texture, perspective, gradients, lighting, and decorative elements based on the background repair mask and background description, while avoiding the regeneration of the main subject or text.
[0091] When processing transparent areas, generative image editing models prioritize continuing existing textures, contours, or gradients outside the transparent area based on background structure information. For example, if a refracted background contour can be observed inside the transparent area, its continuous direction is restored based on the contour positions on both sides of the transparent area and the background structure information, without retaining the refraction offset caused by the transparent material. For background textures visible through the transparent area, they are completed according to the texture direction and density of adjacent unoccluded backgrounds. For locations lacking effective background structure information, a corresponding background is generated based on the background description and the source image content around the background inpainting mask. In this way, usable background information in the transparent area participates in background restoration, but the visual features generated by the transparent material itself are not retained as background content.
[0092] For the text areas and the non-transparent parts of the main subject, since the true background under the occlusion position is usually not directly obtainable from the source image, content continuation can be mainly based on the background description and the unoccluded background near the edge of the background inpainting mask. If multiple areas covered by the background inpainting mask overlap, they are treated as the same continuous inpainting range to avoid different textures or lighting conditions at the boundary after the main subject area and text area are generated separately. After the generative image editing model completes the processing, it obtains a generated image that corresponds to the unified canvas. The generated image is a candidate image for background reconstruction within the coverage area of the background inpainting mask, where the content that can be modified is still limited by the background inpainting mask.
[0093] After obtaining the generated image, instead of directly replacing the source image as a whole, the background inpainting mask continues to control the final compositing. The area covered by the background inpainting mask is designated as the area that can be modified; pixels at the corresponding positions in the generated image are read within this area. Outside the area covered by the background inpainting mask, pixels at the corresponding positions in the source image are read directly. The original processing method also limits the generated image to participate in compositing only within the area of the background inpainting mask, ensuring that the source image pixels outside the background inpainting mask remain unchanged.
[0094] At the boundary of the background inpainting mask, a limited transition range can be set inwards to gradually connect the generated image and the source image. This process does not expand outwards from the background inpainting mask, avoiding color or texture changes caused by the generative image editing model from affecting areas that originally did not need repair. The generated image's background inpainting result is used in areas far from the boundary of the background inpainting mask, while areas within the boundary transition range smoothly connect the generated image and the source image, maintaining the source image pixels outside the background inpainting mask.
[0095] After compositing, a continuity check is performed on the original location of the transparent area, comparing the repaired content at that location with the surrounding unobstructed background in terms of outline direction, texture direction, and color changes. If the corresponding location of the transparent area still retains the refractive outline, reflective bands, or local material colors of the original main subject, this residual location is incorporated into the background repair mask for re-local background repair; if the repaired content can now seamlessly connect with the background outside the transparent area, the current composite result is retained. The final background layer has the same size as the unified canvas, where the original main subject area, transparent area, and text area are replaced by the reconstructed background content, while the remaining areas continue to use the original pixels of the source image.
[0096] This embodiment also discloses an editable image layer splitting device, referring to... Figure 4 The device includes an acquisition module 401, a processing module 402, and an output module 403. It is used to execute any of the editable image layer splitting methods described above, wherein: The acquisition module 401 is used to determine a unified canvas based on the acquired source image, perform visual structure analysis on the source image, and obtain the main subject area, text information, and background description. Processing module 402 is used to generate an intermediate image containing the main subject and a predetermined solid color chroma key background based on the main subject area, and to generate an adaptive alpha mask by combining the chroma key features of the intermediate image and the color protection information of the source image. Processing module 402 is used to perform transparency processing on the intermediate image based on the adaptive alpha mask to obtain a transparent foreground layer; Processing module 402 is used to generate a background repair mask based on the main subject area and the text area in the text information, perform background repair on the area covered by the background repair mask according to the background description, and synthesize the repaired area with the source image to obtain a background layer. Output module 403 is used to generate an editable text layer based on text information, and output a transparent foreground layer, a background layer, and an editable text layer according to the coordinate system corresponding to the unified canvas.
[0097] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0098] This embodiment also discloses an electronic device, as shown in the reference. Figure 5 The electronic device may include: at least one processor 501, at least one communication bus 502, user interface 503, network interface 504, and at least one memory 505.
[0099] The communication bus 502 is used to enable communication between these components.
[0100] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.
[0101] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0102] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.
[0103] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory may include a non-transitory computer-readable storage medium. The memory 505 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. As a computer storage medium, the memory 505 may include an operating system, a network communication module, a user interface 503 module, and an application program for an editable image layer splitting method.
[0104] In the electronic device, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 501 can be used to call an application program stored in the memory 505 that provides an editable image layer splitting method. When executed by one or more processors 501, the electronic device performs one or more methods as described in the above embodiments.
[0105] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.
[0106] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0107] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.
[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0110] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 505 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned memory 505 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.
[0111] The present invention also discloses a non-transitory computer-readable storage medium storing instructions. When executed by one or more processors 501, these instructions cause an electronic device to perform one or more methods as described in the above embodiments.
[0112] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A method for splitting editable image layers, characterized in that, The method includes: A unified canvas is determined based on the acquired source image, and visual structure analysis is performed on the source image to obtain the main subject area, text information, and background description. An intermediate image containing the main subject and a predetermined solid color chroma key background is generated based on the main subject region. An adaptive alpha mask is generated by combining the chroma key features of the intermediate image and the color protection information of the source image. The intermediate image is made transparent based on the adaptive alpha mask to obtain a transparent foreground layer; A background repair mask is generated based on the main subject area and the text area in the text information. The background is repaired in the area covered by the background repair mask according to the background description. The repaired area is then combined with the source image to obtain a background layer. An editable text layer is generated based on the text information, and the transparent foreground layer, the background layer, and the editable text layer are output according to the coordinate system corresponding to the unified canvas.
2. The method for splitting editable image layers according to claim 1, characterized in that, The step of generating an intermediate image containing the main subject and a predetermined solid color chroma key background based on the main subject region, and generating an adaptive alpha mask by combining the chroma key features of the intermediate image and the color protection information of the source image, specifically includes: A reference region is formed by expanding outward from the main subject area as a reference. The image located in the reference region of the source image is cropped, and the main subject is retained by a generative image editing model while the area outside the main subject is set as a predetermined solid color background to obtain the intermediate image. The chromaticity distance and channel advantage of the pixels in the intermediate image and the reference area relative to the predetermined pure color chromaticity are determined respectively, and the color protection information is determined based on the pixels in the reference area that satisfy the chromaticity similarity condition; Based on the chroma key distance of the intermediate image, a boundary connected region connected to the image boundary is determined, and a high-confidence chroma key region is determined based on the chroma key distance and the channel advantage. The boundary connected region and the high-confidence chroma key region are merged and the pixels corresponding to the color protection information are excluded to obtain the hard removal region. Within the neighborhood of the hard removal region, a soft removal region is determined based on the chroma key distance that satisfies the feathering condition and does not belong to the pixel corresponding to the color protection information. An alpha value is determined based on the hard removal region and the soft removal region to generate the adaptive alpha mask.
3. The method for splitting editable image layers according to claim 1, characterized in that, The process of generating a background repair mask based on the main subject area and the text area in the text information, performing background repair on the area covered by the background repair mask according to the background description, and compositing the repaired area with the source image to obtain a background layer specifically includes: Create a mask with the same size as the unified canvas, expand the main subject area and the text area according to the subject safety margin and the text safety margin respectively, and merge the expanded main subject area and the text area to obtain the background repair mask; Based on the source image, the background inpainting mask, and the background description, a generative image editing model is used to perform background inpainting on the area covered by the background inpainting mask to obtain a generated image. The composite area of the generated image is determined based on the background repair mask, the boundary of the composite area is feathered, and the generated image is composited with the source image within the composite area to obtain the background layer.
4. The method for splitting editable image layers according to claim 1, characterized in that, The step of performing visual structure analysis on the source image to obtain the main subject area, text information, and background description specifically includes: The main subject region is determined based on the main visual object in the source image. A closed region is extracted from the main subject region. A transparent region is determined from the closed region based on the texture continuity, contour continuity, and color change relationship between the closed region and the background outside the main subject region. Along the continuous direction between the transparent area and the outer background, a background reference area corresponding to the transparent area is determined from the source image, and the transparency information of the transparent area is determined based on the degree of texture preservation, contour continuity and color change between the transparent area and the background reference area; The transparent area and the transparency information are associated with the main subject area. When generating the intermediate image, the transparent area is preserved, and the predetermined solid color background is displayed through the transparent area. The chroma key features in the intermediate image are partitioned based on the transparent region. For chroma key pixels located within the transparent region and matching the transparency information, an alpha value is determined based on the transparency information. For chroma key pixels located outside the transparent region or not matching the transparency information, an alpha value is determined based on the chroma key features and the color protection information, thereby generating the adaptive alpha mask.
5. The method for splitting editable image layers according to claim 4, characterized in that, The method further includes: A reference region is determined based on the main subject region, the transparent region is mapped to the reference region, the main subject is preserved through a generative image editing model, and the region outside the main subject is set as the predetermined solid color chromaticity background to obtain the intermediate image. Based on the chromatic key distance and channel advantage of the intermediate image, a high-confidence chromatic key region is determined. Boundary connectivity detection is performed on the high-confidence chromatic key region to obtain a boundary connected region. Then, a closed chromatic key region that is not connected to the boundary connected region is determined from the high-confidence chromatic key region. The closed chroma key region is matched with the transparent region in terms of position. The closed chroma key region located outside the transparent region is identified as a hard removal region. For the closed chroma key region located within the transparent region, the Alpha value is determined based on the transparency information. The adaptive alpha mask is generated by combining the hard-removed area, the alpha value corresponding to the transparent area, and the color protection information.
6. The method for splitting editable image layers according to claim 5, characterized in that, The step of processing the intermediate image to be transparent based on the adaptive alpha mask to obtain a transparent foreground layer specifically includes: According to the adaptive alpha mask, pixels outside the transparent region in the hard removal region are set to full transparency, and a soft removal region is determined in the neighborhood of the hard removal region according to the chroma key distance. Continuous alpha values are determined for the pixels in the soft removal region. For pixels that are simultaneously located in the transparent region and the hard-removed region, they are distinguished according to the position matching result of the closed chroma key region and the transparent region. The matching pixels are assigned an Alpha value according to the transparency information, while the non-matching pixels are kept completely transparent. The chroma key colors of the semi-transparent pixels in the intermediate image are restored by inverting the chroma key based on the alpha value, and the chroma key residues are checked by combining the transparent area and the transparency information, and the alpha values in the transparent area that do not match the transparency information are corrected. The corrected Alpha value is written into the intermediate image, and the transparent foreground layer is generated according to the position corresponding to the main subject area.
7. The method for splitting editable image layers according to claim 6, characterized in that, The process of generating a background repair mask based on the main subject area and the text area in the text information, performing background repair on the area covered by the background repair mask according to the background description, and compositing the repaired area with the source image to obtain a background layer specifically includes: The main subject area and the text area are expanded respectively, and the expanded main subject area and the text area are merged to obtain the background repair mask, and the transparent area is located within the coverage of the background repair mask; Based on the transparency information, background structure information is extracted from the transparent area of the source image. Combining the background structure information and the background description, background repair is performed on the area covered by the background repair mask to obtain the generated image. The generated image and the source image are composited according to the background repair mask, wherein the area covered by the background repair mask uses the pixels corresponding to the generated image, and the area outside the area covered by the background repair mask uses the pixels corresponding to the source image, thus obtaining the background layer.
8. An editable image layer splitting device, characterized in that, The apparatus is used to perform an editable image layer splitting method as described in any one of claims 1-7, the apparatus comprising an acquisition module, a processing module, and an output module, wherein: The acquisition module is used to determine a unified canvas based on the acquired source image, perform visual structure analysis on the source image, and obtain the main subject area, text information, and background description. The processing module is used to generate an intermediate image containing the main subject and a predetermined solid color chroma key background based on the main subject region, and to generate an adaptive alpha mask by combining the chroma key features of the intermediate image and the color protection information of the source image. The processing module is used to perform transparency processing on the intermediate image based on the adaptive alpha mask to obtain a transparent foreground layer; The processing module is used to generate a background repair mask based on the main subject area and the text area in the text information, perform background repair on the area covered by the background repair mask according to the background description, and synthesize the repaired area with the source image to obtain a background layer. The output module is used to generate an editable text layer based on the text information, and output the transparent foreground layer, the background layer, and the editable text layer according to the coordinate system corresponding to the unified canvas.
9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, wherein the computer program, when executed by the processor, implements an editable image layer splitting method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform an editable image layer splitting method as described in any one of claims 1-7.
