A material replacement method for positioning an application area

Abstract

The application discloses a material replacement method for smear area positioning, relates to the technical field of image processing, and aims to solve the problem that when multiple materials jointly form a current display image in layered image editing, the display space smear area and the actual material to be replaced and the action area thereof are difficult to accurately correspond. According to the current synthesis relationship, the materials actually participating in formation and the source positions in the smear range of each display position are determined, and a replacement range is formed in the space of each material itself; the image content to be replaced is obtained by taking the display space as a common reference and is written into the corresponding material, and the actually modified source positions are remapped to the display space, the actual influence range is determined and is checked with the position range actually formed by at least one material in the smear range, so that the material object mismatch and the replacement range dislocation are reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for replacing materials by locating a smeared area. Background Technology

[0002] Image material replacement falls under the field of image processing technology and is commonly used in image editing, visual content creation, and interactive material adjustment. In these applications, users typically specify the local area to be processed in the currently displayed image through smearing, selection, or other interactive operations. Image segmentation, content extraction, position alignment, local replacement, or fusion processing are then performed based on the determined area. Existing technologies can utilize user smearing trajectories to form regions of interest or masks, thereby limiting the scope of subsequent image processing. For example, Chinese invention patent application publication number CN106898005A, "A method, apparatus, and terminal for interactive image segmentation," discloses a processing method that forms marked areas and regions of interest based on smearing trajectories on the original image and generates a mask map for interactive image segmentation.

[0003] For local image content replacement, existing technologies can also complete region alignment and replacement based on the feature points, positional relationships, and segmentation results of the target and replacement regions. For example, Chinese invention patent application publication number CN107507216A, "Method, Apparatus, and Storage Medium for Replacing Local Regions in an Image," discloses obtaining the feature points and positional information of corresponding regions in the reference image and the replacement image, performing foreground segmentation and geometric deformation on the relevant regions, and then implementing local region replacement and fusion processing. Therefore, existing technologies can already achieve the determination of user interaction areas and the positioning, alignment, and local replacement between a single image region and the replacement content.

[0004] However, in layered image editing scenarios, the currently displayed image may be formed by multiple editable image content materials. When multiple materials overlap, partially occlude, or are transparently overlaid at the same display position, or are limited by cropping and material masking, the underlying material object and its effective image range may not be the same at different display positions within the same smeared area formed by the user in the currently displayed image. In this case, the smeared area in the display space can indicate the visible position that the user wants to process, but it is difficult to directly determine the actual material object to be replaced and its corresponding range of action based solely on the display area itself. If the display area is still directly used as the basis for local replacement, it may result in the replacement being applied to the wrong material, layer, or material range, and omissions, errors, or inconsistencies in local content may occur when the material position changes, the occlusion relationship changes, the masking changes, or the material is recombined. Therefore, under the condition that multiple materials jointly form the current displayed image, how to accurately correspond the smeared area in the display space to the actual material object to be replaced and its effective range of action remains a technical problem that needs to be further solved in this type of interactive material replacement process. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method for material replacement based on the location of the smeared area, thereby solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for replacing materials by smearing area positioning, comprising: S1. Multiple materials with editable image content are combined according to the composition relationship including spatial mapping, sampling, masking, alpha and source-over hierarchy to form the current display image, and the display space smearing range formed by the user is obtained; S2. Based on the composition relationship, determine at least one material actually involved in the formation of each display position within the smearing range and at least one source position read from each material when forming that position, and establish the correspondence between display position, material and source position; S3. Determine the actual location formed by at least one material within the smearing area, and form a replacement range in the space of each material according to the corresponding and identical spatial mapping and sampling relationship; S4. Determine the image content to be replaced using the display space as a common reference, map the positions of each material source to the display space, and obtain the image content to be replaced according to the mapped positions; S5. Write the corresponding content to be replaced within the replacement range of each material, while keeping the composition relationship unchanged; S6. Remap the actual modified source location to the display space according to the composition relationship, determine the actual range of influence, and verify it with the actual location range formed by at least one material within the smeared area.

[0007] Furthermore, when determining the material that actually participates in forming the display position, the materials are processed sequentially from top to bottom, and the actual contribution is determined by the product of the effective Alpha of the current material at the display position and the remaining transmittance that has not yet been covered by the material above. After processing the current material, the product of the remaining transmittance and 1 minus the effective Alpha is used as the updated remaining transmittance, and the updated remaining transmittance is used for the next layer of material, retaining only the material with an actual contribution greater than 0. The effective alpha is determined by the material alpha corresponding to the current display position and the material mask coverage. The effective alpha outside the cropping range is 0.

[0008] Furthermore, when determining the source position, the display position is reverse-mapped to the material's own space to obtain continuous source coordinates. Bilinear sampling is used to read the four source pixels around the continuous source coordinates. The normalized sampling weight is determined based on the horizontal and vertical distances. The same source pixels and sampling weights are used to read the color, alpha, and material mask.

[0009] Furthermore, when each material forms a replacement range in its own space, the source pixels corresponding to the materials that actually participate in forming the smear position are collected, and the center of each source pixel is mapped to the display space in the current spatial mapping direction. Bilinear sampling is used to read the coverage of the smear mask in the display space, and a grayscale mask representing the replacement range of the corresponding material is formed based on the coverage.

[0010] Furthermore, using the smallest axis-aligned bounding rectangle of the actual position range formed by at least one material within the smearing area as the target range, the material to be replaced is scaled to cover the target range while maintaining the aspect ratio, so that the centers of the two overlap and the portion of the scaled material to be replaced that exceeds the target range is cut off; When a source pixel corresponding to the same source location participates in sampling at multiple display locations, only one content write is performed, and the image content to be replaced is determined by the position of the source pixel center mapped to the display space.

[0011] Furthermore, from the formation of the smearing area to the completion of the writing of each material content and the verification of the actual impact area, the spatial mapping, cropping, material masking, alpha and hierarchical order in the composition relationship remain unchanged; When any of the spatial mapping, cropping, material masking, alpha, or layer order changes, the established correspondence between display position, material, and source position is no longer used. Instead, the actual material, source position, and replacement range that participate in forming the display position are re-determined based on the changed compositing relationship.

[0012] Furthermore, a grayscale mask of 0 to 255 is used to represent the replacement range of each material. When the grayscale value is 255, the color to be replaced is written. When the grayscale value is 1 to 254, the original color and the color to be replaced are linearly mixed according to the ratio of the grayscale value to 255. When the grayscale value is 0, the original color is retained. Write to maintain the alpha, mask, crop, spatial mapping, and layer order of the material.

[0013] Furthermore, when determining the actual impact range, the grayscale mask is returned to the display space according to the current spatial mapping and bilinear sampling. The materials are processed sequentially from top to bottom. The product of the grayscale mask sampling coverage of the current material, the effective alpha, and the remaining transmittance that has not yet been covered by the upper material is accumulated. The accumulated result is quantized from 0 to 255 to reach a display position with at least one grayscale level to form the actual impact range. The display position compares the actual area of ​​influence with the area of ​​position actually formed by at least one material within the smeared area. Positions where the latter exists but the former does not are defined as uncovered positions, and positions where the former exists but the latter does not are defined as extended positions.

[0014] Furthermore, for the edge of the actual impact range, the grayscale value obtained by quantifying the cumulative results is compared with the corresponding grayscale value of the mask applied to the display space; If any of the following situations exist: uncovered position, extended position, or grayscale difference exceeding one grayscale level, read the material and source pixels of the corresponding display position, find the display position within the scope of this smearing inspection that uses each source pixel as sampling support, multiply the smearing mask coverage of each display position by the sampling weight of the corresponding source pixel, sum them and divide by the total weight, quantize the resulting value to 0 to 255 and update the corresponding grayscale mask, rewrite the relevant source pixels and determine the local actual influence range.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. By determining the actual materials and source positions involved in forming each display position within the smearing area based on the current compositing relationship, and forming a replacement range in the space of each material itself, the display space is used as a common reference to determine the image content to be replaced. This allows the visible content formed by multiple materials within the same smearing area to correspond to the actual underlying material and its position of action, thereby reducing material object mismatch and replacement range misalignment when replacing only based on the display area or a single layer.

[0016] 2. After the content is written to the underlying material, the actual modified source position is remapped to the display space according to the spatial mapping, sampling, masking, alpha and hierarchical relationship consistent with the source. The actual impact range is then checked against the smear position range with actual replaceable material sources. This allows the actual effect of the underlying modification on the current displayed image to be directly verified, and it can identify the non-correspondence of uncovered positions, extension positions and sampling edges.

[0017] 3. By maintaining consistency in the composition relationship used for source determination and underlying writing during the same smearing process, and re-determining the correspondence between display position, material and source position when space mapping, cropping, material masking, alpha or hierarchical relationship changes, the smearing input is always interpreted according to the actual current display state, avoiding the continued incorrect use of existing correspondence after the composition state changes, thereby maintaining consistency in the technical relationship between display space positioning, underlying material writing and subsequent recompositing. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a method for replacing materials by locating a smeared area. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example: Combined with Appendix Figure 1 This embodiment provides a method for replacing materials by locating a smeared area, including: S1. Multiple materials with editable image content are combined according to the composition relationship including spatial mapping, sampling, masking, alpha and source-over hierarchy to form the current display image, and the display space smearing range formed by the user is obtained; S2. Based on the composition relationship, determine at least one material actually involved in the formation of each display position within the smearing range and at least one source position read from each material when forming that position, and establish the correspondence between display position, material and source position; S3. Determine the actual location formed by at least one material within the smearing area, and form a replacement range in the space of each material according to the corresponding and identical spatial mapping and sampling relationship; S4. Determine the image content to be replaced using the display space as a common reference, map the positions of each material source to the display space, and obtain the image content to be replaced according to the mapped positions; S5. Write the corresponding content to be replaced within the replacement range of each material, while keeping the composition relationship unchanged; S6. Remap the actual modified source location to the display space according to the composition relationship, determine the actual range of influence, and verify it with the actual location range formed by at least one material within the smeared area.

[0021] This implementation uses a layered image editing program running on an image editing terminal as the implementation vehicle. The image editing terminal employs a general-purpose processor capable of performing two-dimensional image compositing, image sampling, and pixel content writing. The editing program maintains multiple raster materials that store image content that can be directly modified, and generates the currently displayed image according to the spatial position, scaling and rotation status, image sampling method, cropping range, material mask, alpha transparency, and bottom-up hierarchical order of each material in the current canvas. The material mask is used to limit which positions of the material itself are allowed to participate in the display, and the alpha is used to indicate the degree of coverage of the material pixels on the current display result. Independently stored material masks only serve as visibility constraints for the corresponding materials and are not used as material objects to be written into the image content; data objects that do not store image content that can be directly modified do not participate in this low-level content writing.

[0022] To ensure a consistent spatial framework throughout the processing, the display coordinate system of the currently displayed image is defined with the top-left corner of the canvas as the origin. Horizontally, the x-coordinate increases to the right, and vertically, the y-coordinate increases downwards. Each material's own image coordinate system uses the same directional definition. The screen position during user drawing is uniformly represented as display coordinates after conversion from the editor's existing window-to-canvas mapping. Window scaling, scrollbars, and interface borders are no longer considered in subsequent material source determination. The spatial mapping relationship of each material stores the translation, scaling, and rotation relationships between its own image coordinates and display coordinates. When a material is cropped, the cropped area is included as part of the effective display area in the processing.

[0023] In this embodiment, the compositing relationship employs a standard layer-by-layer Source-Over Alpha compositing method. The effective alpha of each pixel is defined by its own alpha and the mask coverage of other pixels at the same location; the effective alpha for locations outside the cropping range is 0. During image formation, the same display position is processed layer by layer from the topmost layer downwards: first, the effective alpha of the current pixel at that position is calculated; then, the remaining transmittance not yet covered by the upper layers is used to define the proportion of the current pixel that can be included in the final display result. After the current pixel is processed, the remaining transmittance is passed to the next layer based on the uncovered portion of the pixel. This allows for the determination of which editable image content elements actually contribute to the same display position without relying on object names or manual selection.

[0024] Image sampling employs bilinear sampling as the primary implementation method. After the display position is reverse-mapped into the space of a certain material, if the resulting position falls between the centers of integer pixels, the four adjacent source pixels around that consecutive position are read, and four sampling weights are determined based on the horizontal and vertical distances from the consecutive position to the centers of the four source pixels; the closer the source pixel is, the greater its weight, and the sum of the four weights is 1 after normalization. Material color, alpha, and material masking all use the same geometric mapping and sampling aperture to ensure that the color formation position, transparency judgment position, and mask judgment position are consistent. Specific image size, number of materials, and brush size do not constitute fixed conditions for the method to succeed; the dimensions and values ​​given below are used to illustrate how the same processing rule is executed continuously.

[0025] When an image editing program displays the currently displayed image, it can simultaneously obtain the order of the materials used to generate the image, the spatial mapping parameters of each material, the cropping range, the material masking data, and the alpha data. Users can use a mouse, stylus, or touch input to paint on the currently displayed image, with the input device continuously generating display coordinate points. The first display coordinate point is recorded at the start of the painting process, and subsequent display coordinate points are recorded according to the order of input device events during the movement. The current trajectory acquisition terminates when the painting ends. Adjacent coordinate points are connected by straight line segments, and the brush center continuously covers the display pixels along these straight line segments. For example, with a circular brush, if the shortest distance from the center of a display pixel to the trajectory line segment is not greater than the brush radius, that display pixel is considered part of the current painting coverage area.

[0026] To preserve partial coverage of the brush edges, the displayed smear area is represented using a grayscale mask of the same size as the currently displayed image. Display pixels completely uncovered by the brush are recorded as 0, and those completely within brush coverage are recorded as 255. When the brush outline passes through a display pixel, the grayscale value is converted to a value between 1 and 254 based on the proportion of that pixel's area covered by the brush geometry. The editor's built-in anti-aliasing rasterization process directly provides this coverage ratio, so no additional empirical threshold needs to be set. The grayscale value only indicates the spatial coverage of this smear and does not indicate whether a particular material should be replaced.

[0027] For example, if the current canvas is 1920×1080 pixels, the user uses a circular brush with a diameter of 24 pixels to continuously move from display coordinates (720, 420) to (980, 660). The input device generates several coordinate points on this trajectory. The editing program connects adjacent coordinate points in event sequence and rasterizes the coverage area of ​​the circular brush. The smear mask value for pixels inside the trajectory is 255, and edge pixels obtain corresponding grayscale values ​​based on the degree of partial coverage. Subsequent source analysis only traverses display pixels with mask values ​​greater than 0 and adjacent 1-pixel boundaries that may be affected by bilinear sampling, without performing irrelevant calculations pixel by pixel on the entire 1920×1080 image.

[0028] The compositing parameters used to interpret this smear are taken from the same set of material states that generated the image currently seen by the user. For each editable image content material participating in the display, the processing program reads the image size, translation position in the canvas, scaling ratio, rotation angle, cropping range, material mask, and alpha data, and also reads the compositing order of the material from top to bottom. The parameters are not newly established management information, but rather image processing parameters already used by the image editing program when rendering the current screen. The entire smear source analysis, source material content writing, and actual impact range verification all use this set of compositing parameters with the same spatial and transparency caliber.

[0029] From the completion of the smearing process until the end of the underlying content writing and scope verification, the spatial mapping, cropping, material masking, alpha, and layer order involved in this process remain consistent with those when the currently displayed image was generated. The editing program allows normal scaling, moving, or material layer adjustments to continue after this processing is complete. If the image editing program receives an edit operation that changes the compositing parameters before the current processing is finished, the existing correspondence between the current display position and the source material position will not be used for the new compositing state. The processing program will redetermine the corresponding source and source material replacement range based on the actual new compositing parameters. This process is only used to ensure that the same smear is always interpreted according to the compositing caliber consistent with the user's view.

[0030] For each display position in the display space mask where the grayscale value is greater than 0, the image editing program checks the editable image content material in the top-to-bottom hierarchical order of the current material. For a given material, the target display position is first converted to its own image coordinates based on the translation, scaling, and rotation relationships of the material. The reverse conversion is performed in the reverse order of the spatial transformation when generating the current display image: first, the translation amount of the material in the display space is subtracted; then, the coordinates before rotation are restored according to the opposite rotation direction; and finally, the continuous coordinates corresponding to the material's own size are restored according to the horizontal and vertical scaling ratios. If the obtained continuous coordinates fall outside the range of the material image or outside the current cropping range, there are no readable source pixels for that material at the target display position, and the current material does not participate in the subsequent source determination for that display position.

[0031] When continuous coordinates are within the effective image range of the source material, the source sampling position is obtained using the same bilinear sampling relationship as when the currently displayed image was generated. Taking the continuous source coordinates (482.25, 316.60) as an example, the actual source pixel positions read are (482, 316), (483, 316), (482, 317), and (483, 317). Since the horizontal decimal part is 0.25 and the vertical decimal part is 0.60, the sampling weights of the four source pixels are 0.30, 0.10, 0.45, and 0.15, respectively. The sum of the four sampling weights is 1. The color components, alpha, and material mask value at the four positions are weighted according to this weight, thus obtaining the sampled color, sampled alpha, and sampled mask coverage corresponding to the current display position. When the continuous coordinates fall exactly at the center of an integer pixel, the corresponding source pixel weight is 1, and the weights of the other source pixels are 0.

[0032] The effective alpha of the source image is determined by a combination of the sampling alpha and the sampling mask coverage. Taking 8-bit data as an example, both the source pixel alpha and the mask are in the range of 0 to 255, and after reading, they are normalized to 0 to 1 respectively. When the current source image's sampling alpha is 204 and the sampling mask is 128, the normalized values ​​are approximately 0.800 and 0.502, respectively, and the effective alpha is approximately 0.402. When the sampling mask is 0, the source image alpha is 0, or the target display position is outside the cropping range, the effective alpha is 0, and the source image does not contribute any actual image content at that display position.

[0033] The actual contribution of the same display position is determined layer by layer according to the Source-Over composition order. The processing program sets the remaining transmittance to 1 before checking the top layer material. The actual contribution of the current material to the final display position is equal to the product of the material's effective alpha and the current remaining transmittance. After the current material is processed, the product of the current remaining transmittance and 1 minus the material's effective alpha is used as the updated remaining transmittance, and this updated remaining transmittance is applied to the next layer material. Materials with an actual contribution of 0 are not included in the material source for that display position; materials with an actual contribution greater than 0 retain their material identifier, form the actual source sampling position read for that display position, and the sampling weights of each source sampling position. This judgment is directly based on the transparent composition data used for image rendering, and does not consider whether the material is on the top layer, the material name, manual selection, or simple rectangular intersection as judgment conditions.

[0034] For example, a certain display area is covered by an upper semi-transparent texture, a middle semi-transparent decorative texture, and a bottom background texture. The effective alpha of the upper semi-transparent texture is 0.402, and there is no covering material above it; its actual contribution is 0.402, and the remaining transmittance becomes 0.598. The effective alpha of the middle semi-transparent decorative texture is 0.600, and its actual contribution is 0.598 multiplied by 0.600, approximately 0.359, resulting in a remaining transmittance of approximately 0.239. The effective alpha of the bottom background texture is 1, and its actual contribution is approximately 0.239. Therefore, this display area retains three actual contributing material sources simultaneously, instead of treating only the top semi-transparent texture as the sole processing object. If the effective alpha of the upper material at this display area is 1, the remaining transmittance becomes 0. Although the material below it geometrically covers this area, it no longer actually participates in the formation of this display pixel, and the corresponding lower material does not enter the source of this display area.

[0035] After each smear display location is analyzed layer by layer, the image editing program retains the display location, the actual contributing material identifier, the corresponding continuous source coordinates, the actual source sampling position and sampling weight, and the actual contribution of the material. The saved content is temporary data required for the current image processing and does not require a separate database or specific table structure. The display location is subsequently used to uniformly determine the sampling position of the material to be replaced, the material identifier is used to distinguish the underlying material to be actually written to, and the source sampling position and sampling weight are used to form the replacement range of each material's own space and to ultimately map the underlying modifications back to the display space.

[0036] When no editable image content contributes a non-zero amount to the smeared area, that area does not correspond to a source image. For example, if the smear path passes through a completely transparent area, or if the displayed area is only covered by an interface auxiliary object whose image content cannot be directly modified, and there is no directly writable image content at the underlying layer, then although that area is still within the user's brush coverage, it is not considered the actual location for replacing the underlying image. This distinction maintains a clear boundary between the displayed smear area and the display area that actually has a replaceable image source.

[0037] After mapping the displayed locations to the actual source locations of the contributing material, the image editing program extracts display locations from the display space smear mask that simultaneously meet two conditions: first, the display space smear mask value is greater than 0; second, at least one actual contributing editable image content material has been identified. Locations meeting both conditions constitute the range of smear locations with actual replaceable material sources for this operation. Other locations in the display space smear mask remain in the user-interactive display but are not written to underlying objects that lack replaceable sources.

[0038] For each actual contributing element, the processing program aggregates all display locations containing that element's source and the actual source sampling locations read by these display locations within the element. The aggregated results are used to define the local area within the source element where the replacement mask needs to be calculated. Since bilinear sampling may cause a display location to simultaneously read four adjacent source pixels, the local area contains four source sampling locations; when multiple adjacent display locations reference the same source pixel, that source pixel is only retained once in the local area. This processing ensures that the replacement range of the source element is based on the source pixels that participated in the display during actual rendering, rather than directly inferring it from the complete rectangular area of ​​the element or the bounding rectangle of the display smear.

[0039] For each source pixel center obtained from the aggregation, the processing program forward transforms the source pixel center to the display space according to the current spatial mapping relationship of the material. The forward transformation adopts the same scaling, rotation, and translation order as the current display image generation to obtain the continuous positions of the source pixel center in the display space. Subsequently, the smear coverage corresponding to the continuous position is read on the display space smear mask according to bilinear sampling. When the continuous position falls outside the smear coverage area, the source material replacement mask value is 0; when the continuous position falls in the fully smeared area, the source material replacement mask value is 255; when it falls on the brush edge, a grayscale value between 1 and 254 is obtained according to the bilinear sampling result of the display space smear mask.

[0040] Whether a source pixel remains within the replacement range is also constrained by the actual source of contribution. If a source pixel is located in the smeared area after forward mapping, but the material containing that source pixel is completely obscured by the upper opaque material at the corresponding display position, then that source pixel does not appear in the actual source at the corresponding display position, and the source material replacement mask remains 0. If the same source pixel participates in forming multiple display positions within the smeared range through the current scaling and rotation relationships, its actual role is checked using the source sampling support relationships of that source pixel in these display positions. The first determination of the source material replacement mask uses the display mask sample value after forward mapping of the source pixel center as the source mask value. If the actual influence range check finds that the boundary does not correspond, the boundary mask value is re-formed based on all display sampling relationships in which the source pixel participates.

[0041] For example, a texture with an original size of 1200×900 pixels is scaled to 900×675 pixels and rotated 12° in the canvas. After the display position is reverse-mapped to the continuous coordinates of the texture, the smearing area involves sampling positions around the source image with horizontal coordinates of 460 to 690 and vertical coordinates of 280 to 520. The processor only generates replacement masks in these actual sampling positions and adjacent pixels required for bilinear sampling, without changing the rest of the source image. The center of a pixel located at source coordinates (482, 317) is forward-mapped to approximately display coordinates (804.6, 493.2). When the sample value of the display space smear mask at this continuous position is 255, the initial value of the replacement mask for this source pixel is 255; if it is forward-mapped to the edge of the smear brush and the sample value is 96, then the initial value of the corresponding source mask is 96.

[0042] When the source material itself has irregular masking, the source material replacement mask does not replace the original material mask. The original material mask continues to determine the visible range of the material in the display composition, and the source material replacement mask only limits which source image content locations to be replaced are written. When the original material mask corresponding to a source pixel is 0, the material does not make a valid contribution to the relevant display location, and the source pixel will not receive a replacement effect due to display smearing geometry coverage. Thus, both conditions must be met simultaneously: "the location painted by the user" and "which underlying image content actually forms that location".

[0043] After the above processing is completed, each contributing material yields a grayscale replacement mask with the same size as its own image or corresponding to a local processing area. A mask value of 0 indicates that the source location does not contain the image content to be replaced, 255 indicates that the source location is completely filled with the image content to be replaced, and 1 to 254 indicate that the source location performs an edge transition between the original image content and the image content to be replaced according to the corresponding coverage ratio. The source material replacement mask corresponds one-to-one with the material identifier and continues to be used together with the corresponding data of the display position and the source sampling position for unified spatial sampling of the content to be replaced.

[0044] The image to be replaced is an image material that the user has already selected and whose pixel content can be directly read. To avoid different underlying materials starting from different positions in the image to be replaced, the image editing program determines a unique spatial correspondence between the image to be replaced and the smeared position range with an actual source of replaceable material in the display space. The processing program obtains the smallest axis-aligned bounding rectangle of the smeared position range, and uses this bounding rectangle as the target range of the image to be replaced in the display space. The image to be replaced is scaled using a coverage method that maintains the original aspect ratio, ensuring that the width and height of the scaled image are not less than the width and height of the target range, and that the center of the scaled image coincides with the center of the target range; any part exceeding the target range is cropped from all sides. This processing forms a defined display space sampling position for image to be replaced with different aspect ratios according to the same rule.

[0045] For example, the bounding rectangle of the area to be replaced, which has a smeared area with an actual replaceable material source, is 320 pixels wide and 240 pixels high. The material to be replaced is 1024×1024 pixels. While maintaining the aspect ratio to cover the target area, the material to be replaced is scaled to 320×320 pixels, using the 320-pixel target width as a constraint. Then, 40 pixels are cropped from the top and bottom, making the remaining 320×240 pixels consistent with the target area. The center position of the target area therefore corresponds to the center position of the material to be replaced; the horizontal left boundary of the target area corresponds to the left boundary of the cropped material to be replaced, and the horizontal right boundary corresponds to the right boundary of the cropped material to be replaced. No correspondence is established for the content to be replaced outside the target area.

[0046] For each display position within the range of available replacement materials, the processing program determines the corresponding continuous coordinates within the replaced material that has undergone uniform scaling and centering cropping, based on the horizontal and vertical distances of that position relative to the top-left corner of the target range. Bilinear sampling is then used to obtain the content of the image to be replaced. When the display position falls within the center of the target range, the content at the center of the cropped area of ​​the replaced material is read. When the display position moves one display pixel to the right, its corresponding position in the replaced material moves to the right by the corresponding distance according to a uniform scaling ratio. This display space is calculated only once and does not change with the number of underlying materials.

[0047] After the display position and the content to be replaced are determined, the processing program reads the established correspondence between the display position, the actual contributing material, and the source sampling position, and determines the source position participating in this replacement in each actual contributing material. For multiple actual contributing materials corresponding to the same display position, each material uses the display space where that display position is located as a common reference for the content to be replaced. When writing source pixels, the processing program forward converts the center of each source pixel to the display space according to the spatial mapping relationship of its respective material, and reads the color of the material to be replaced at the consecutive display positions according to the aforementioned unique display space sampling relationship, which is used as the unique color to be written for that source pixel. When the same source pixel participates in bilinear sampling of multiple adjacent display positions, different colors to be written are not repeatedly assigned for different display positions; the display position obtained by forward mapping of the center of the source pixel is still used to determine its image content to be replaced. Thus, although different underlying materials are located in different source image coordinate systems, they all establish a correspondence for the content to be replaced through the same display space.

[0048] For example, the display position (820, 510) is actually formed by both the upper semi-transparent texture material and the lower background material. The display position corresponds to the source sampling position in the two materials that participated in its formation. When writing to the lower layer, the processing program obtains the pixel center of the source pixel to be written in the two materials, converts each source pixel center to the display space according to the spatial mapping relationship of its respective material, and reads the color of the material to be replaced at the obtained continuous display position according to the unified display space relationship. Although the source coordinates of the two materials are different, the content to be replaced is determined according to the same display space reference; the original alpha of the two materials remains unchanged, so when recombining, they still participate in the formation of the current display image according to the original transparent overlay relationship.

[0049] The content to be replaced mapping only processes source locations where the source material's replacement effect mask value is greater than 0. If a source location, although within the image's range, does not actually participate in the formation of the smeared area, or if the corresponding source material's replacement effect mask value is 0, no content will be created for that source location in this write operation. This rule ensures that the underlying write range is simultaneously limited by both the actual display source and the user's smearing space.

[0050] The image editing program performs content writing on the image data of each contributing material. Each material reads its own source material replacement mask and the image content to be replaced corresponding to that source location. When the source material replacement mask value is 255, the color content of the corresponding source pixel is written as the replacement color; when the mask value is between 1 and 254, the mask value is divided by 255 to obtain the coverage ratio of the content to be replaced, and the original source pixel color and the replacement color are linearly blended using this ratio and written back; when the mask value is 0, the original color content is preserved. The writing only changes the color content channel used for display; the material's own alpha data, material mask, cropping range, spatial mapping, and layer order remain consistent with those when the current display image was formed.

[0051] Taking 8-bit RGBA source material as an example, the original RGB value of a source pixel is (80, 120, 160), and the color to be replaced is (200, 100, 60). The mask value for the source material replacement is 128. 128 corresponds to a write coverage ratio of approximately 0.502. The processing program mixes the red, green, and blue color channels at the same ratio to obtain approximately (140, 110, 110) and writes it to the source image; the original alpha value of the source pixel remains unchanged. The internal position of the source material replacement mask with a value of 255 is directly written with (200, 100, 60), while the position with a mask of 0 remains (80, 120, 160). The edge source pixels thus maintain a continuous transition with the edges of the display space smear.

[0052] When multiple contributing materials correspond to the same display position, each material modifies its own underlying color content, and the colors to be written are all sampled with a unified display space as the reference. Taking the example of a display position formed by an upper-layer semi-transparent texture material, a middle-layer semi-transparent decorative material, and an underlying background material, if all three materials have a non-zero actual contribution at that display position, the pixel center of the source pixel to be written in each of the three materials is mapped forward to the display space, and the corresponding color to be replaced is read at its respective resulting display position. Then, the writing is completed according to the replacement mask of its respective source material. The original alpha and material mask remain unchanged, so when the three materials are recombined, the final display pixels are formed according to the original transparent overlay relationship.

[0053] The underlying data written directly affects the pixel data of the source material read by the image editing program during subsequent recompositing, rather than simply overlaying a temporary image onto the current screen buffer or the current composite screenshot. When source material is moved, the canvas is redrawn, or the interface is refreshed, the editing program reads the modified source image content and continues to form the display image according to the original spatial mapping, cropping, masking, alpha, and hierarchical relationships. Therefore, the current display result and the underlying source material used in subsequent recompositing maintain the same data source.

[0054] When a single source pixel participates in the formation of multiple display positions under bilinear sampling conditions, the source pixel performs only one actual color write. The color to be written to the source pixel is determined by the corresponding position of its pixel center in the display space, and the write coverage is determined by the mask used to replace the corresponding source material. The influence of this source pixel on multiple display pixels is not repeatedly written in the source image, but is naturally propagated to adjacent display positions by bilinear sampling during recombination. The actual influence range is checked to see if the propagation range is consistent with the position that needs to be covered in this smearing.

[0055] After the underlying material has been written, the image editing program does not use whether the RGB values ​​change before and after the replacement as the basis for judging the actual impact range. Some colors in the material to be replaced may happen to be the same as the original material colors. If only the color difference is compared, the underlying writing that has actually been completed will be mistakenly judged as unprocessed. The actual impact range is determined based on the source material replacement mask and the composition relationship of the same image. That is, it checks which written source material positions can actually affect which display positions under the current spatial mapping, bilinear sampling, material masking, alpha, and hierarchical relationship.

[0056] The processing procedure uses the area surrounding the smeared location with a valid source of replaceable material as the inspection range, and adds one display pixel around it as a bilinear sampling boundary. For each display location within the inspection range, the same inverse spatial mapping and bilinear sampling method as used in generating the current image are employed to read the source material replacement mask value for each modified material. If the source replacement mask sample value of a material at the target display location is greater than 0, and the material has a non-zero valid alpha at that display location and is not completely occluded by the upper layer material, then the modification of that material has a real impact on that display location.

[0057] When multiple modified materials jointly affect the same display position, the actual degree of influence is calculated according to the hierarchical order consistent with color source-over composition. The processing program traverses from the top layer downwards, using the source replacement mask sampling coverage of the current material, the effective alpha of the material, and the remaining transmittance not yet covered by the upper layer to form the modification influence of the current material, and accumulates the modification influence of each contributing material. When the accumulated modification influence reaches at least one gray level after quantization from 0 to 255, the display position is recorded as part of the actual influence range. One gray level corresponds to the minimum non-zero coverage that the 8-bit mask used in this embodiment can represent, and this boundary is determined by the mask quantization resolution.

[0058] Smearing locations with actual replaceable source material are determined using a clear binary caliber: the display space smear mask is greater than 0, and at least one editable image content material has been identified as having a non-zero actual contribution at that display location. The processing procedure compares the smearing location range with the actual influence range for each display location. Locations that exist within the smearing location range but not within the actual influence range are recorded as uncovered locations; locations that do not exist outside the smearing location range but exist within the actual influence range are recorded as extension locations; when both exist simultaneously or neither exists, the binary interaction relationship of that display location is consistent. When further checking using 8-bit edge coverage, if the coverage grayscale obtained by remapping the source modification differs from the corresponding grayscale of the display space smear mask by no more than one grayscale level, it is considered consistent within the quantization error range.

[0059] In the main implementation process where the current compositing relationship remains unchanged, since the source material replacement mask is formed using the same spatial mapping and sampling relationship as the display generation, and the display position and the source sampling position use the same corresponding aperture, most internal positions can directly obtain a consistent action relationship. Differences may mainly occur at rotation, non-integer scaling, and brush anti-aliasing edges. For edge differences, the processing program uses the already saved source sampling support and sampling weights to reconstruct the replacement mask values ​​of the relevant source pixels, without correcting by arbitrarily adding or deleting the entire source region.

[0060] The edge re-formation rule is as follows: For a given uncovered or extended location, read the actual contributing material corresponding to that display location and all source sampling locations within that material that participated in forming that display location. For each source sampling location, continue searching for all display locations within the current smearing inspection range that use that source sampling location as sampling support. Multiply the smearing mask coverage of these display locations by the bilinear sampling weight of that source sampling location in the corresponding display location, sum the results, and divide by the sum of the sampling weights to obtain the new replacement mask coverage of that source sampling location. The obtained coverage is quantized in the range of 0 to 255 and written back to the corresponding source material replacement mask. This rule uses the existing display smearing coverage and source sampling weights to determine the edge value, without introducing a new objective function or empirical scoring.

[0061] After the mask is updated following the replacement of related source materials, the processing program only re-writes the corresponding replacement color to the affected source pixels and re-obtains the actual local impact range according to the same spatial mapping, sampling, masking, alpha, and hierarchical relationships. If the binary range is consistent and the edge coverage grayscale difference does not exceed one grayscale level, it is confirmed that this material replacement corresponds to the smearing position range with an actual replaceable material source. If the current compositing relationship has not changed, but after processing according to the above edge reconstruction rules, there are still uncovered or extended positions, or the edge coverage grayscale difference still exceeds one grayscale level, then this verification fails, and it is not confirmed that the current underlying modification corresponds to the smearing position range with an actual replaceable material source. If during the inspection it is found that the spatial mapping, cropping, material masking, alpha, or layer order used to generate the current display image has changed, the existing correspondence between the display position and the source material position will no longer be used as the basis for processing the current display state. The processing program will use the changed actual compositing parameters to redetermine the actual contributing material, source sampling position, and corresponding source material replacement mask for the display position, and then perform low-level writing and actual impact range verification according to the display space corresponding to the same material to be replaced.

[0062] After ensuring that the binary range is consistent and the edge coverage grayscale difference does not exceed one grayscale level, the image editing program uses the modified underlying materials as the current image data source and regenerates the currently displayed image according to the current compositing relationship. The final technical output of this process is the current edited image, which has actually written the corresponding underlying material image content, and whose actual influence range in the display space corresponds to the range of the smeared position with replaceable material sources.

[0063] The following describes the actual operation of the above rules using a sequential processing procedure involving three editable image content materials. The current display canvas size is 1920×1080 pixels. The bottom background material is 1920×1080 pixels, with a spatial scaling ratio of 1, an alpha of 255, and no additional masking. The middle decorative material is 1600×1200 pixels, scaled by 0.75 and shifted 260 pixels to the right and 120 pixels down during display. The overall alpha of the material is 153, and it also has an 8-bit grayscale mask in some areas. The top texture material is 1200×900 pixels, scaled by 0.75 and rotated 12° during display. Its local alpha and the material mask together form a semi-transparent overlay. The three materials participate in the Source-Over composition in the order of top texture material, middle decorative material, and bottom background material. The above dimensions and values ​​are only used to demonstrate the process.

[0064] The user continuously paints on a visual area in the currently displayed image, using a brush with a diameter of 24 pixels. The brush path extends from display coordinates (720, 420) to (980, 660). The image editing program creates a painting mask in the display space based on the brush path and the brush radius. The coverage within the painting is 255, and the edges are partially covered with grayscale based on the actual pixel coverage area. The painted area overlaps with the display areas of three other materials, but this does not automatically mean that all three materials should be modified.

[0065] For the display position (820, 510), the program first checks the upper-layer texture. It performs a reverse calculation based on the current translation, scaling, and 12° rotation of the texture to obtain the continuous source coordinates (482.25, 316.60). Bilinear sampling reads four source pixels: (482, 316), (483, 316), (482, 317), and (483, 317), with sampling weights of 0.30, 0.10, 0.45, and 0.15, respectively. The alpha and mask of the upper-layer texture are read with the same weights. If the sampling alpha is 204 and the sampling mask is 128, the effective alpha is approximately 0.402. Since the upper-layer texture is at the top and its initial remaining transmittance is 1, its actual contribution is approximately 0.402. Therefore, the display position (820, 510) records the upper-layer texture and the aforementioned four source sampling positions.

[0066] The mid-layer decorative material undergoes a source-same-source check at the display position (820, 510). After reverse conversion, this display position falls within the effective image range of the mid-layer decorative material and does not exceed the cropping boundary. The sampled material has an Alpha of 153, a mask of 255, and an effective Alpha of 0.600. The upper-layer texture material already occupies approximately 0.402 of the display coverage, leaving approximately 0.598 of the remaining transmittance transferred to the mid-layer decorative material. The actual contribution of the mid-layer decorative material to the final display result is approximately 0.359. This contribution is greater than 0, and the display position (820, 510) simultaneously records both the mid-layer decorative material and its corresponding source sampling position.

[0067] The bottom background material has an Alpha of 255, and the display position (820, 510) directly corresponds to a sample from the same location in its source image. After processing, the remaining transmittance of the upper texture material and the middle decorative material is approximately 0.239, and the bottom background material contributes approximately 0.239 due to its effective Alpha of 1. Therefore, the display position (820, 510) has 3 actual image content sources. If the mask value of the upper texture material at the adjacent display position (822, 512) is 255 and the sampling Alpha is 255, then the effective Alpha of the upper texture material is 1, and the remaining transmittance directly becomes 0. Although the middle decorative material and the bottom background material geometrically cover this position at (822, 512), they are not recorded as actual contribution sources for this display position.

[0068] The image editing program iterates through the display locations where the mask value is greater than 0, following the method described above. Some smeared locations are formed solely by the underlying background material, while others are formed by a combination of mid-layer decorative material and the underlying background material. Areas where transparency overlaps may even be formed by all three materials. The final display location corresponds to the source material based on the actual alpha, mask, and layer relationships, rather than assigning the entire smeared area to a single layer at once.

[0069] For upper-layer texture materials, the program gathers all display positions and corresponding source sampling positions that record the material as the actual source of contribution. The source pixel (482, 317) participates in the sampling of the display position (820, 510), and may also participate in bilinear sampling of adjacent display positions. The program maps the center of this source pixel to the display space using a scaling of 0.75, a rotation of 12°, and a translation relationship, obtaining continuous display coordinates, and then applies a mask in the display space to read the coverage amount. If the position is inside the mask, the source material replacement mask value is 255; if the mapped position falls on the brush edge and a display mask sampling value of 96 is obtained, the source material replacement mask is correspondingly 96. The middle-layer decorative materials and the bottom-layer background materials each form their own source material replacement masks according to their spatial mapping and actual source relationship.

[0070] If a small, completely transparent blank area exists within the smearing area, and no editable image content material contributes a non-zero value to this blank area, then that corresponding display position will not enter the actual underlying replacement area. This blank area does not need to be deleted from the smearing mask; the program simply prevents the corresponding source position from being generated when forming the replacement area for each source material. This preserves the user's original brush shape while avoiding the creation of fictitious objects in locations without actual source material.

[0071] This study uses a 1024×1024 pixel texture image to be replaced. The smallest axis-aligned bounding rectangle of the actual replaceable source smear display position is 320×240 pixels. The program scales the 1024×1024 image to be replaced proportionally to 320×320 pixels, and crops 40 pixels from the top and bottom of the scaled image, using the 320×240 pixel cropped area to cover the target bounding rectangle. The upper left corner of the target bounding rectangle corresponds to the upper left corner of the cropped area, and the center of the target bounding rectangle corresponds to the center of the cropped area of ​​the image to be replaced. Each actual replacement display position is uniquely determined by its relative position within the 320×240 target range, which defines the sampling position of the image to be replaced.

[0072] For the display position (820, 510), the program has already determined the upper-layer texture material, middle-layer decorative material, and lower-layer background material corresponding to this position, along with their source sampling positions. When writing the actual source pixels, the pixel center of each material to be written is mapped forward to the display space according to its respective spatial mapping relationship. The image content to be replaced, determined according to a unified display space relationship, is then read at the resulting consecutive display positions. Although the upper-layer texture material, middle-layer decorative material, and lower-layer background material have different source coordinates and spatial transformation relationships, they all use their positions in the same display space as the sampling reference for the content to be replaced. The position of the image to be replaced is not recalculated from the upper left corner of their respective replacement range. Therefore, the replacement texture at the transparent boundary of the material will not change position due to the different origins of the source material coordinates.

[0073] When the replacement mask value of a source pixel in the upper-layer texture material is 255, the program directly writes the corresponding RGB to be replaced into the source pixel while preserving the original alpha. When the mask value is 96, 96 is divided by 255 to obtain a coverage ratio of approximately 0.376. Approximately 37.6% of the RGB to be replaced is mixed with approximately 62.4% of the original RGB before being written back, while the original alpha and the original mask remain unchanged. The same writing rules are applied to the middle-layer decorative materials and the bottom-layer background materials according to their respective source replacement masks. If the same source pixel is sampled by multiple display positions, it is only written once in the source image, and subsequent display images automatically read the updated source pixel according to the original bilinear sampling.

[0074] After the bottom layer writing is complete, the program checks the smeared outer range and its surrounding 1-pixel boundary. The display position (820, 510) re-reads the source replacement masks of the three modified materials in reverse order, and calculates whether the current modification can actually enter the display position based on the effective alpha and residual transmission relationship of each material. If at least one of the three materials has a non-zero modification impact, the display position falls within the actual influence range; the display position must also be within the original smear mask and have at least one actual replaceable material source, maintaining a consistent binary interaction. For positions inside the brush, the source mask sampling coverage and the display smear coverage are typically close to 255.

[0075] Assuming the display smear mask value at the brush edge display position (718, 418) is 84, but the modified coverage obtained from the initial source mask remapping is converted to 80, the difference is 4 8-bit gray levels, exceeding the 1 gray level quantization tolerance used in this implementation. The program reads all source sampling positions (718, 418) in the relevant material and continues to search for the display positions within the current inspection range formed by these source sampling positions. For a certain source pixel, the smear mask value of each relevant display position is multiplied by the bilinear sampling weight of the source pixel in the corresponding display position, the sum of the products is divided by the sum of the corresponding sampling weights, and the resulting value is quantized to 0 to 255 and used as the new replacement effect mask value for the source pixel. After the relevant source pixels are rewritten, the actual display influence range is recalculated only for that local edge.

[0076] If the actual modified coverage at the recalculated display position (718, 418) is 83, which differs from the target smear mask value of 84 by one gray level, then the correspondence is determined according to the 8-bit mask quantization resolution; the same check is also performed on adjacent positions. This edge correction does not change the display space correspondence of the material to be replaced, does not change the actual contributing material set, and does not change the original space mapping, alpha, material mask, and hierarchical relationship. It only uses the original display smear coverage and the already determined source sampling weights to re-determine the edge source mask coverage.

[0077] After this processing is confirmed, the canvas is redrawn by directly reading the modified upper-layer texture, middle-layer decorative, and lower-layer background materials. If the user subsequently moves the upper-layer texture material, the new display result is generated according to the modified source image data and the new spatial position of that material; previously replaced content moves along with the actual materials and does not remain in its original screen position. If the user changes the material masking or layer order, the new composite result continues to use the modified lower-layer image content as input. This continuous behavior stems from the fact that the replacement is directly written to the source material data, rather than a one-time overwrite of the previous screen image.

[0078] When the source material is only translated, the display position is mapped back to the source material space by simply subtracting the translation amount. The source sampling position usually maintains a one-to-one or approximately one-to-one relationship with the display pixels. When the source material is scaled or rotated at a non-integer multiple, the back mapping yields continuous source coordinates, and bilinear sampling generates multiple source sampling positions. Regardless of the complexity of the spatial transformation, the determination of the source of the display position uses the same spatial mapping relationship generated when the current display image was generated. The formation of the source material replacement effect range and the verification of the final actual impact range also follow the same relationship. Therefore, spatial transformation only changes the specific sampling position and does not change the object correspondence in the processing chain.

[0079] When the source material is cropped, the source continuous coordinates obtained by reverse mapping are first checked to see if they are within the size range of the source image, and then checked if the position is allowed to be displayed under the current cropping conditions. If the cropping conditions are not met, the effective alpha of the source material at the target display position is directly treated as 0, and its color is not read as the actual display source. When the source material mask has grayscale edges, the mask grayscale and the source alpha jointly determine the effective alpha; a mask value of 128 does not directly indicate that the source material should be replaced by 50%, but rather that the transparency coverage basis of the source material in the current display pixel formation has changed accordingly. The actual source material replacement coverage is still determined by mapping the display smear mask to the source material through the same spatial sampling relationship.

[0080] When the upper layer of material at a display location is semi-transparent, the lower layer materials may still contribute to the overall image quality. Instead of the simplified rule of "stopping the search downwards as soon as the upper layer pixel is not transparent," the processing algorithm calculates layer by layer based on the remaining transmittance. Only when a layer's effective alpha reaches complete opacity, reducing the remaining transmittance to 0, will the material below it cease to be a source for that display location. This layer-by-layer processing maintains the same accuracy even when multiple semi-transparent materials are stacked; as the number of materials increases, only the number of layers traversed for the same display location increases, without introducing new judgment types.

[0081] When the overall alpha of a given source image is less than 255, but the alpha of its source pixels exhibits local variations, the sampled alpha at the current display position is first obtained by bilinear sampling of the local alpha of the source pixels. This sampled alpha is then combined with the overall alpha of the source image and the mask coverage of the source image to calculate the effective alpha used for display compositing. If the editing program has already incorporated the overall alpha into the final alpha of the source image pixels, it directly reads the effective alpha before compositing, without repeating the multiplication. The judgment is always based on the amount of transparency coverage used when rendering the currently displayed image, avoiding discrepancies between the calculation methods in the manual and the editor's actual display methods.

[0082] When the source material replacement mask is formed, the center of the source pixel is mapped forward to the display space and the smearing mask is read as the first determination method. When more than one 8-bit gray level difference appears at the bilinear sampling edge, the edge source mask value is recalculated using the existing support relationship between the display position and the source sampling position. This secondary processing only checks for the actual non-corresponding boundaries found within the display area and does not traverse the entire source image unrelated to the smearing range. This allows for handling sampling coverage diffusion caused by rotation and non-integer scaling while maintaining the same technical mechanism.

[0083] The display space sampling relationship of the material to be replaced remains unchanged within a single smearing process. After the bounding rectangle of the target display position is formed, the material to be replaced completes its display space positioning according to fixed proportional coverage and centering cropping rules. Regardless of the layer, scaling ratio, or rotation angle of the source material, its source position is first mapped to the current display coordinates, and then the content to be replaced is read from that fixed display space. The image size of the source material itself only affects the source pixel density and sampling position, and does not change the geometric continuity of the content to be replaced in the final display space.

[0084] When performing the second smoothing operation, the image editing program regenerates the current display image using the modified underlying image from the first smoothing as the current content. The second smoothing generates a new display space smoothing mask, and the source of the display position is redefined based on the actual spatial mapping, sampling, masking, alpha, and hierarchy relationships at the time of the second smoothing. The display position and source sampling position used in the first smoothing are not used as the default source relationship for the second smoothing. This continuous operation mode does not require the establishment of an additional state machine; each smoothing operation obtains input from the current actual display image and its underlying image composition relationships.

[0085] When the second application overlaps with the area replaced in the first application, the second processing reads the color of the source material that has already been written in the first application. The content to be replaced in the second application still establishes a unified display space correspondence according to the display position range of the actual replaceable material source in the second application, and is written at the source position of the current actual contributing material. The first writing does not change the alpha, mask, and layer relationship of the relevant material, so the source determination can still be performed directly according to the existing compositing data; when the user actively changes the spatial position, mask, or layer of the material between the two processing steps, the second source determination naturally uses the changed current compositing relationship.

[0086] The image editing program identifies replaceable material objects based on whether the object saves directly modifiable raster image content and participates in the current Source-Over Alpha composition. Independent material masks only serve as visibility constraints for their corresponding materials and do not individually receive the color to be replaced; other data objects that do not save directly modifiable image content are not included in source location determination and underlying content writing. Object types and composition data can be directly obtained from the current image generation process of the image editing program.

[0087] The processing scope of this implementation mainly focuses on the current smear and its sampling-affected neighborhood. When there are many source materials, materials that cannot possibly cover the smeared position can be excluded first by utilizing the spatial intersection relationship between the current display bounding area of ​​each material and the smearing bounding area. Then, the remaining materials are judged pixel-by-pixel based on the actual alpha and the mask. This spatial intersection is only used to reduce irrelevant calculations and does not replace the actual display contribution judgment; as long as the display area of ​​the material intersects with the smearing area, whether it becomes an actual replacement source is still determined by the source sampling, alpha, mask, and upper layer transmission relationship.

[0088] The core inputs of this implementation method all come from data that the image editing program can obtain during the current screen generation and user interaction. The display space smear coordinates are derived from input events; spatial mapping, sampling method, cropping, material masking, alpha, and layer order are derived from the current image composition; the source sampling position is obtained by inversely converting the display coordinates according to the current spatial mapping and using the current sampling method; the image content to be replaced comes from readable image materials already selected by the user; the source material replacement mask is formed by the display smear mask, the actual contribution source, and the same spatial sampling relationship. The above data does not depend on external training data, unknown model output, or human experience scoring.

[0089] The sole primary criterion for determining whether a display location has a viable source material is: the display space mask value must be greater than 0, and at least one source material containing modifiable image content must have a greater than 0 actual source-over compositing contribution at that location. Whether the actual contribution of the source material is greater than 0 is directly calculated from the source sampling alpha, the source material mask, the cropping validity, and the remaining transmittance of the upper layer. Masking geometry coverage, source material rectangular coverage, or the source material being currently selected are not considered as independent criteria for confirming the actual source.

[0090] The source material replacement effect range adopts a unique master formation rule: a local source pixel candidate range is established only for materials that have actually participated in the formation of the target display position and their source sampling positions; the display continuous position of the candidate source pixel center is obtained by mapping the current material space in the forward direction, and the display space smear mask coverage is read using bilinear sampling consistent with the display generation as the initial value of the source replacement mask; when the actual influence range verification finds that the edges do not correspond, the edge mask value is then weighted and calculated using all display sampling positions and corresponding sampling weights in which the source pixel participated. This rule also provides the determination path for ordinary internal positions and sampling edge positions.

[0091] The content to be replaced from multiple source materials uses a single common reference: the material to be replaced is first proportionally covered and centered within the bounding rectangle of the actual replaceable display position, and each display position is uniquely associated with the continuous coordinates of the material to be replaced by its display coordinates. The source material position is not directly sampled from the material to be replaced using its own coordinates; instead, its display coordinates are first determined according to its current spatial mapping, and then the image content to be replaced is obtained using these display coordinates. This rule ensures that multiple materials with the same display position but different underlying source coordinates still obtain replacement content under the same spatial reference.

[0092] The actual impact range is determined using a single master check rule: the source replacement mask for each modified source material is returned to the display space according to the current spatial mapping and bilinear sampling, and together with the corresponding material's effective alpha and layer residual transmittance, it is determined whether the underlying modification actually affects the target display position. Under an 8-bit mask, when the cumulative modification impact reaches at least one gray level, the display position is recorded as part of the actual impact range; the target display position is determined by a display smear mask greater than 0 and the existence of at least one actual replaceable material source. After the binary range is consistent, the edge coverage gray level is then checked for differences by quantizing one gray level.

[0093] Spatial orientation and coordinate system remain consistent throughout the processing. Both the display space and the material's own space use the top-left corner as the origin, with the positive x-axis pointing to the right and the positive y-axis pointing downwards. For material rotation, the current rendering direction used by the editing program is adopted; during reverse mapping, the opposite direction is restored. All source coordinates, display coordinates, and sampling coordinates of the material to be replaced use the pixel center as the geometric reference. If a specific processing program internally uses other origins or rotation directions, these should be uniformly converted to the above-mentioned coordinates when input into this method, and then subsequent processing should be completed according to the same rules.

[0094] The above description is merely a specific embodiment of the present invention, used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Without departing from the technical concept of the present invention, any equivalent substitutions, reasonable modifications, or combinations made by those skilled in the art based on the disclosure of this specification should be determined according to the claims to determine whether they fall within the scope of protection of the present invention.

Claims

1. A method for replacing materials by locating a smeared area, characterized in that, include: S1. Multiple materials with editable image content are combined according to the composition relationship including spatial mapping, sampling, masking, alpha and source-over hierarchy to form the current display image, and the display space smearing range formed by the user is obtained; S2. Based on the composition relationship, determine at least one material actually involved in the formation of each display position within the smearing range and at least one source position read from each material when forming that position, and establish the correspondence between display position, material and source position; S3. Determine the actual location formed by at least one material within the smearing area, and form a replacement range in the space of each material according to the corresponding and identical spatial mapping and sampling relationship; S4. Determine the image content to be replaced using the display space as a common reference, map the positions of each material source to the display space, and obtain the image content to be replaced according to the mapped positions; S5. Write the corresponding content to be replaced within the replacement range of each material, while keeping the composition relationship unchanged; S6. Remap the actual modified source location to the display space according to the composition relationship, determine the actual range of influence, and verify it with the actual location range formed by at least one material within the smeared area.

2. The method for replacing materials by locating a smeared area according to claim 1, characterized in that, When determining the material that actually participates in forming the display position, the materials are processed sequentially from top to bottom. The actual contribution is determined by the product of the effective Alpha of the current material at the display position and the remaining transmittance that has not yet been covered by the material above. After processing the current material, the product of the remaining transmittance and 1 minus the effective Alpha is used as the updated remaining transmittance, and the updated remaining transmittance is used for the next layer of material, retaining only the material with an actual contribution greater than 0. The effective alpha is determined by the material alpha corresponding to the current display position and the material mask coverage. The effective alpha outside the cropping range is 0.

3. The method for replacing materials by locating a smeared area according to claim 2, characterized in that, When determining the source position, the display position is reverse-mapped to the material's own space to obtain continuous source coordinates. Bilinear sampling is used to read the four source pixels around the continuous source coordinates. The normalized sampling weight is determined based on the horizontal and vertical distances. The same source pixels and sampling weights are used to read the color, alpha, and material mask.

4. The method for replacing materials by locating a smeared area according to claim 3, characterized in that, When each material forms a replacement range in its own space, the source pixels corresponding to the materials that actually participate in forming the smear position are collected, and the center of each source pixel is mapped to the display space in the positive direction according to the current space mapping. Bilinear sampling is used to read the coverage of the smear mask in the display space, and a grayscale mask representing the replacement range of the corresponding material is formed based on the coverage.

5. The method for replacing materials by locating a smeared area according to claim 1, characterized in that, The target range is defined by the smallest axis-aligned bounding rectangle of the actual position range formed by at least one material within the smearing area. The material to be replaced is scaled to cover the target range while maintaining the aspect ratio, so that the centers of the two are aligned and the part of the scaled material to be replaced that exceeds the target range is cut off. When a source pixel corresponding to the same source location participates in sampling at multiple display locations, only one content write is performed, and the image content to be replaced is determined by the position of the source pixel center mapped to the display space.

6. The method for replacing materials by locating a smeared area according to claim 1, characterized in that, From the formation of the smearing area to the completion of the writing of each material content and the verification of the actual impact area, the spatial mapping, cropping, material masking, alpha and hierarchical order in the composition relationship remain unchanged; When any of the spatial mapping, cropping, material masking, alpha, or layer order changes, the established correspondence between display position, material, and source position is no longer used. Instead, the actual material, source position, and replacement range that participate in forming the display position are re-determined based on the changed compositing relationship.

7. The method for replacing materials by locating a smeared area according to claim 1, characterized in that, The replacement range of each material is represented by a grayscale mask from 0 to 255. When the grayscale value is 255, the color to be replaced is written. When the grayscale value is 1 to 254, the original color and the color to be replaced are linearly mixed according to the ratio of the grayscale value to 255. When the grayscale value is 0, the original color is retained. Write to maintain the alpha, mask, crop, spatial mapping, and layer order of the material.

8. The method for replacing materials by locating a smeared area according to claim 4, characterized in that, When determining the actual range of influence, the grayscale mask is returned to the display space according to the current spatial mapping and bilinear sampling. The material is processed sequentially from top to bottom. The product of the grayscale mask sampling coverage of the current material, the effective alpha, and the remaining transmittance that has not yet been covered by the upper material is accumulated. The accumulated result is quantized from 0 to 255 to reach a display position with at least one grayscale level to form the actual range of influence. The display position compares the actual area of ​​influence with the area of ​​position actually formed by at least one material within the smeared area. Positions where the latter exists but the former does not are defined as uncovered positions, and positions where the former exists but the latter does not are defined as extended positions.

9. The method for replacing materials by locating a smeared area according to claim 8, characterized in that, For the edge of the actual impact range, the grayscale of the cumulative result is compared with the corresponding grayscale of the mask applied to the display space; If any of the following situations exist: uncovered position, extended position, or grayscale difference exceeding one grayscale level, read the material and source pixels of the corresponding display position, find the display position within the scope of this smearing inspection that uses each source pixel as sampling support, multiply the smearing mask coverage of each display position by the sampling weight of the corresponding source pixel, sum them and divide by the total weight, quantize the resulting value to 0 to 255 and update the corresponding grayscale mask, rewrite the relevant source pixels and determine the local actual influence range.

Images