Image editing method and apparatus, device, and storage medium

Abstract

Provided in the embodiments of the present disclosure are an image editing method and apparatus, a device, and a storage medium. The method comprises: in an editing interface of a target image, receiving a selection operation of a user on a first special effect object of a preset type; in response to the selection operation on the first special effect object, rendering on a first image layer corresponding to the target image a second image layer corresponding to the first special effect object, and rendering on the second image layer a third image layer corresponding to at least one main object in the target image; and, on the basis of an editing operation or a confirmation operation of the user on the first special effect object, determining an edited or confirmed first special effect object in the second image layer to obtain an edited target image, the edited target image comprising a first image layer, a second image layer and a third image layer. In this way, the present disclosure achieves sandwiched presentation of special effect objects, and allows users to freely edit the special effect objects of sandwiched layers, thereby meeting diversified special effect editing requirements of users.

Description

Method, apparatus, device and storage medium for image editing

[0001] This application claims priority to the Chinese invention patent application entitled “Method, device, apparatus and storage medium for image editing” and application number 202311161804.5, filed on September 8, 2023. The entire contents of that application are incorporated herein by reference. Technical Field

[0002] Example embodiments of the present disclosure relate generally to information processing, and more particularly, to methods, apparatuses, devices, and computer-readable storage media for image editing. Background Art

[0003] Currently, more and more applications are designed to provide users with various services. For example, users can create, browse, comment on, and forward various types of content within the application, including various media such as videos, images, image collections, and audio. Furthermore, these applications support user interaction, making it easier for users to capture and create content. They can also provide users with special effects during content creation. For example, users can add special effects elements such as animals, scenery, and objects to their created content.

[0004] Summary of the Invention

[0005] In a first aspect of the present disclosure, a method for image editing is provided. The method comprises: receiving a user selection operation for a first special effect object of a predetermined type in an editing interface of a target image; in response to the selection operation for the first special effect object, rendering a second layer corresponding to the first special effect object on a first layer corresponding to the target image, and rendering a third layer corresponding to at least one main object in the target image on the second layer; and based on the user's editing operation or confirmation operation on the first special effect object, determining the edited or confirmed first special effect object in the second layer to obtain an edited target image, wherein the edited target image includes the first layer, the second layer, and the third layer.

[0006] In a second aspect of the present disclosure, a device for image editing is provided. The device includes: a selection receiving module configured to receive a user's selection operation of a predetermined type of first special effect object in an editing interface of a target image; a rendering module configured to, in response to the selection operation of the first special effect object, render a second layer corresponding to the first special effect object on a first layer corresponding to the target image, and render a third layer corresponding to at least one main object in the target image on the second layer; and an image determination module configured to determine the edited or confirmed first special effect object in the second layer based on the user's editing operation or confirmation operation on the first special effect object, to obtain an edited target image, wherein the edited target image includes the first layer, the second layer, and the third layer.

[0007] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform the method according to the first aspect of the present disclosure.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to perform the method according to the first aspect of the present disclosure.

[0009] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent hereinafter with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG2 shows a flowchart of a process of image editing according to some embodiments of the present disclosure;

[0013] 3A to 3D illustrate schematic diagrams of example editing interfaces according to some embodiments of the present disclosure;

[0014] 4A and 4B are schematic diagrams showing example editing interfaces in which a target image includes multiple subject objects according to some embodiments of the present disclosure;

[0015] [Corrected 09.10.2024 according to Rule 91] FIG. 5A to FIG. 5C are schematic diagrams showing example editing interfaces in which a target image includes a plurality of special effect objects according to some embodiments of the present disclosure;

[0016] FIG6 shows a schematic diagram of a process of rendering a layer according to some embodiments of the present disclosure;

[0017] 7A to 7C are schematic diagrams showing example images for subject segmentation according to some embodiments of the present disclosure;

[0018] FIG8 shows a schematic structural block diagram of an apparatus for image editing according to some embodiments of the present disclosure; and

[0019] FIG9 shows a block diagram of an electronic device that can be used to implement some embodiments of the present disclosure. DETAILED DESCRIPTION

[0020] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0021] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The term "some embodiments" should be understood as "at least some embodiments." Other explicit and implicit definitions may be included below.

[0022] The term "in response to" indicates that a corresponding event occurs or a condition is satisfied. It will be understood that the timing of executing a subsequent action executed in response to the event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is satisfied. In some cases, the subsequent action may be executed immediately when the event occurs or the condition is satisfied; in other cases, the subsequent action may be executed some time after the event occurs or the condition is satisfied.

[0023] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0024] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0025] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to the electronic device, application, server or storage medium and other software or hardware that performs the operation of the technical solution of the present disclosure based on the prompt message.

[0026] As an optional but non-limiting implementation, in response to receiving a user's active request, a prompt message may be sent to the user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0028] 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In the example environment 100, an application 120 is installed in a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or a device attached to the terminal device 110.

[0029] In some embodiments, application 120 may be a content sharing application that can provide various services related to media content items to user 140, including browsing, commenting, forwarding, creation (e.g., shooting and / or editing), publishing, etc. In some embodiments, application 120 may be a content editing application, such as an image and / or video editing application.

[0030] In the environment 100 of Figure 1, if the application 120 is in an active state, the terminal device 110 can present the interface 150 of the application 120. The interface 150 may include various pages that the application 120 can provide, such as a content presentation page, a content creation page, a content publishing page, a message page, a personal homepage, and the like. The application 120 can provide content creation functions, including shooting, uploading, editing and / or publishing media content items. In some implementations, the application 120 can provide a content browsing function to browse various types of content published in the application 120. In some implementations, the application 120 can also provide interactive functions, including interacting with other users (for example, adding friends) and interacting with various types of content (including liking, collecting, commenting on content, etc.).

[0031] In some embodiments, the terminal device 110 communicates with the server 130 to enable the supply of services to the application 120. The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.). The server 130 can be various types of computing systems / servers that can provide computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0032] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0033] As mentioned above, users may want to add some elements to the content they create, such as text, various graphics, etc. For example, users can edit images. Users can use text props to add text information to images or use other sticker-type special effects to add image elements to images. Usually, such text or image elements are directly superimposed on the top layer of the user's image for presentation. Such image editing effects are single and may block key areas of the image (such as the main object of the image). In some editing scenarios, it is expected to obtain more presentation methods of sticker-type special effects.

[0034] To this end, embodiments of the present disclosure provide an improved image editing solution. According to this solution, a user selection operation for a predetermined type of special effects object is received in an image editing interface. In response to this selection operation, a layer corresponding to the special effects object is rendered on a layer corresponding to the image, and a layer corresponding to at least one main object in the image is rendered on the layer corresponding to the special effects object. Based on the user's editing operation or confirmation operation on the special effects object, the special effects object is determined to obtain an edited image. The edited image includes the layer corresponding to the image, the layer corresponding to the special effects object, and the layer corresponding to at least one main object.

[0035] The disclosed image editing solution allows for the addition of special effects objects to an edited image, creating a sandwiched presentation of these objects, yielding a variety of rich, three-dimensional visual effects. Furthermore, it allows users to freely edit the sandwiched special effects objects, meeting their diverse special effects editing needs.

[0036] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0037] Figure 2 illustrates a flowchart of an image editing process 200 according to some embodiments of the present disclosure. Process 200 may be implemented on a terminal device 110. For ease of discussion, process 200 will be described with reference to environment 100 in Figure 1. It should be noted that the operations performed by the terminal device 110 described above, as well as those performed by the terminal device 110 described later, may be performed by applications installed on the terminal device 110.

[0038] In block 210 , the terminal device 110 receives a user selection operation for a first special effect object of a predetermined type in an editing interface of a target image.

[0039] "Special effects object" refers to a selectable visual object under the special effects editing function provided in image editing. In this article, the special effects editing function, also known as the template function or the prop function, refers to a function that can at least add a specific visual effect to the image, such as adding static or dynamic objects to the picture, providing interactive effects, changing the color contrast of the picture, character makeup, etc. In some examples, the special effects object may include a dynamic special effects object, such as a dynamic object with animation effects. The special effects object may include a static special effects object, such as the addition of a static object. In some embodiments, the special effects object may include augmented reality (AR) special effects to provide an AR effect.

[0040] In some embodiments, the target image to be edited can be a single image or a video image within a video. For example, during video editing, a user can be supported to edit one or more video images within a video. During image editing, a user can be allowed to request the use of a specific special effects object or prop object.

[0041] In an embodiment of the present disclosure, a user is provided with a selection of a predetermined type of special effect object during the image editing process. As described in detail below, the predetermined type of special effect object is added between the main object and the background of the target image. This type of special effect object is sometimes also referred to as a sandwich effect.

[0042] In some embodiments, the terminal device 110 may present an operation control associated with selecting a special effect object in the editing interface of the image. Figures 3A to 3D show schematic diagrams of example editing interfaces according to some embodiments of the present disclosure. As shown in Figure 3A, at least a special effect selection control 301 is presented in the editing interface 300A of the target image 310. In response to detecting a selection operation on the special effect selection control 301, the terminal device 110 may present at least one type of special effect object. Exemplarily, at least one type of special effect object may include, but is not limited to, a special effect object that can be presented on the top layer of the target image, a special effect object that can be presented on the bottom layer of the target image, a special effect object that can be presented between the bottom layer of the target image and the top layer of the main object, and the like.

[0043] In the discussion of the embodiments of this disclosure, it is assumed that a user chooses to add a predetermined type of special effect object to a target image, namely, a special effect object that can be rendered between the bottom layer of the target image and the top layer of the main object. The first special effect object can be, for example, text, an image, an icon, an animated image, or any other suitable object, which is not limited in this disclosure. It should be understood that other types of special effect objects can also be configured in the image editing interface according to editing needs, such as special effect objects rendered at the top layer.

[0044] The terminal device 110 can then present the first special effects object in association with the target image in response to the selection operation of the first special effects object of the predetermined type. Specifically, referring to Figure 2, in box 220, the terminal device 110 renders the second layer corresponding to the first special effects object on the first layer corresponding to the target image in response to the selection operation of the first special effects object of the predetermined type, and renders the third layer corresponding to at least one main object in the target image on the second layer. That is to say, the first special effects object selected by the user is rendered in the middle layer, on which the layer corresponding to the main object of the target image is also rendered. In this way, the added special effects object can automatically avoid occluding the specific image subject during sandwich presentation, and can also present a sandwich and three-dimensional special effects effect that is different from the "sticker" presented on the general top layer, making the special effects object and the image appear more integrated.

[0045] As shown in FIG3B , in editing interface 300B, in response to a user selecting special effect object 320, terminal device 110 renders the layer (i.e., the second layer) corresponding to special effect object 320 (i.e., the first special effect object) on the layer (i.e., the first layer) corresponding to target image 310. The visual effect of editing interface 300B is that special effect object 320 is superimposed on target image 310.

[0046] Furthermore, the terminal device 110 may also determine at least one main object in the target image. The at least one main object here may be at least one object in the foreground area of ​​the target image. The terminal device 110 may determine the layer corresponding to the at least one main object as the third layer. The terminal device 110 may achieve sandwich presentation of the special effect object by rendering the third layer on the second layer corresponding to the first special effect object. As shown in FIG3C , in the editing interface 300C, after the terminal device 110 renders the layer corresponding to the special effect object 320 on the layer corresponding to the target image 310, it may also render the layer (i.e., the third layer) corresponding to the main object 315 (e.g., the water cup shown in the figure) in the target image 310 on the layer corresponding to the special effect object 320.

[0047] It's important to note that the position, size, and outline of the at least one subject object in the third layer are rendered identically to those of the at least one subject object in the first layer. That is, after the third layer is rendered on top of the second layer, the at least one subject object in the third layer completely overlaps with the at least one subject object in the first layer.

[0048] In some embodiments, to ensure a consistent user experience when editing images, the process of rendering the second layer on top of the first layer, and the third layer on top of the second layer, is invisible to the user. That is, after rendering is complete, editing interface 300C is presented to the user, rather than editing interface 300B. From the user's perspective, after selecting a first special effect object of a predetermined type, editing interface 300C is directly displayed, as shown in FIG3C , directly presenting the sandwich effect of the special effect object. In some embodiments, depending on the application, the rendering process can also be presented to the user layer by layer.

[0049] In some embodiments, the at least one subject object corresponding to the third layer may be, for example, all subject objects in the target image. Specifically, the at least one subject object may be one or more subject objects automatically identified from the target image.

[0050] In some embodiments, at least one subject object corresponding to the third layer may be at least part of a plurality of subject objects in the target image. Specifically, one or more subject objects may be selected by the user from the subject recognition results and rendered on the top layer. In some embodiments, if multiple candidate subject objects are identified from the target image, the terminal device 110 may present subject identifiers corresponding to each of the multiple candidate subject objects in the editing interface. The subject identifier corresponding to each subject object includes at least one of the following: contour identification information for defining the subject object from the target image, or an object indicator presented at a position corresponding to the subject object in the target image.

[0051] Terminal device 110 may receive a user's selection of a subject identifier for at least one subject object from among a plurality of candidate subject objects and render a third layer corresponding to the selected at least one subject object on the second layer. For example, terminal device 110 may determine that a selection of the corresponding subject object has been detected in response to detecting a selection operation for the subject identifier and / or a selection operation for the subject object. Terminal device 110 may then determine that the layer corresponding to the selected at least one subject object is the third layer and render the third layer on the second layer.

[0052] Figures 4A and 4B show schematic diagrams of example editing interfaces in which a target image includes multiple subject objects according to some embodiments of the present disclosure. As shown in Figure 4A, when a target image 410 includes multiple candidate subject objects (subject object 411 and subject object 412), the terminal device 110 can present the subject identification corresponding to each of the multiple candidate subject objects in the editing interface 400A. Specifically, the terminal device 110 can associate the outline identification information 413 and the object indicator 414 of the subject object 411 with the subject object 411 in the editing interface 400A. The terminal device 110 can associate the outline identification information 415 and the object indicator 416 of the subject object 412 with the subject object 412 in the editing interface 400A. The outline identification information (including the outline identification information 413 and the outline identification information 414) can be used to finely represent the outline of the corresponding subject object, or to define each subject object from the target image using, for example, a bounding box. Object indicators 414 and 416 may include, for example, respective anchor points and labels (e.g., labels "Subject 1" and "Subject 2" as shown in the figure). In some embodiments, terminal device 110 displays only the outline identification information corresponding to the subject object, only the object indicator corresponding to the subject object, or both the outline identification information and the object indicator.

[0053] Furthermore, in response to receiving a selection operation on subject object 412, a selection operation on outline representation information 415, and / or a selection operation on object indicator 416, terminal device 110 determines that a selection operation on subject object 412 has been received. Terminal device 110 may determine the layer corresponding to subject object 412 as the third layer.

[0054] As shown in Figure 4B , in editing interface 400B, the layer corresponding to target image 410 is the first layer, the layer corresponding to the first special effect object (e.g., the multiple clouds shown in Figure 4B ) is the second layer, and the layer corresponding to subject object 412 is the third layer. Terminal device 110 renders the second layer on top of the first layer, and then renders the third layer on top of the second layer. As a result, subject object 412 is rendered on top, preventing it from being obscured by the first special effect object, while subject object 411 may be obscured by the first special effect object (depending on the placement of the first special effect object). In this way, the user can choose which subject objects to present on top of the target image (i.e., which subject objects are not obscured by the first special effect object and which are obscured by the first special effect object). This allows for sandwiched presentation of special effect objects while allowing users to select the subject objects to be presented on top of the image, meeting diverse special effect editing needs and further enhancing the user experience with special effects. It should be understood that when selecting a subject object, the user can select only a single subject object placed on the top layer, or they can select multiple subject objects.

[0055] The above describes an example of a user selecting a special effect object of a predetermined type. In some embodiments, the user can also select multiple special effect objects of the predetermined type. Specifically, the terminal device 110 can also receive a user selection of a second special effect object of the predetermined type. In response to the selection of the second special effect object, the terminal device 110 can render a fourth layer corresponding to the second special effect object on the first layer corresponding to the target image. The fourth layer is below the third layer. The fourth layer corresponding to the second special effect object can be above or below the second layer corresponding to the first special effect object, but the fourth layer is always between the first layer corresponding to the target image and the third layer corresponding to the main object. In some embodiments, the terminal device 110 can sort the different special effect objects in the order of user selection, from top to bottom or from bottom to top. The first selected special effect object can be at the bottom or top of the special effect object layer. It should be noted that regardless of the order of the layers corresponding to the special effect objects, all special effect objects of the predetermined type are always sandwiched between the first and third layers.

[0056] At block 230, the terminal device 110 determines the edited or confirmed first special effect object in the second layer based on the user's editing operation or confirmation operation on the first special effect object, and obtains an edited target image. The edited target image includes the first layer, the second layer, and the third layer. The editing operation here may include, for example, at least one of a position movement operation, a direction movement operation, and a scaling operation.

[0057] In some embodiments, after rendering the second layer on the first layer and then rendering the third layer on the second layer, the terminal device 110 can generate a target image including a sandwich special effect. The terminal device 110 can also adjust the first special effect object in the second layer based on a user's editing operation on the first special effect object. Specifically, in response to receiving a user's position movement operation on the first special effect object, the terminal device 110 can adjust the display position of the first special effect object in the second layer. In response to receiving a user's direction movement operation on the first special effect object, the terminal device 110 can adjust the display direction of the first special effect object in the second layer. In response to receiving a user's zoom operation on the first special effect object, the terminal device 110 can adjust the display size of the first special effect object in the second layer. As shown in FIG3D , in response to receiving an editing operation on special effect object 320, the terminal device 110 adjusts the display position, display direction, and display size of special effect object 320 in its corresponding layer.

[0058] Similarly, when the user selects multiple special effects objects of a predetermined type, the terminal device 110 can determine the edited or confirmed special effects objects in the layers corresponding to the multiple special effects objects based on the user's editing operations or confirmation operations on the multiple special effects objects. The edited target image includes multiple layers corresponding to the multiple feature objects. Exemplarily, if the user also selects a second special effects object, the terminal device 110 can also determine the edited or confirmed second special effects object in the fourth layer based on the user's editing operations or confirmation operations on the second special effects object. The edited target image also includes a fourth layer. It should be noted that, in some embodiments, the editing operation for the second special effects object is independent of the editing operation for the first special effects object, that is, the editing operations for different special effects objects are independent of each other and do not affect each other.

[0059] [Corrected 09.10.2024 in accordance with Rule 91] Figures 5A to 5C illustrate schematic diagrams of example editing interfaces in which a target image includes multiple special effect objects, according to some embodiments of the present disclosure. As shown in Figure 5A, the target image 510 in editing interface 500A includes a first layer where the target image 510 resides, a second layer corresponding to the special effect object 520, and a third layer corresponding to the main object 515. As shown in Figure 5B, in response to receiving a user selection operation for a special effect object 530 (i.e., a second special effect object), the terminal device 110 may further render a fourth layer corresponding to the special effect object 530 between the first and third layers to present the target image 510 shown in editing interface 500B. As shown in Figure 5C, the terminal device 110 may further adjust the special effect object 520 in the second layer and the special effect object 530 in the fourth layer in response to the respective editing operations on the special effect object 520 and the special effect object 530 to present the target image 510 shown in editing interface 500C.

[0060] It should be understood that although the first special effect object and the second special effect object are discussed above, in the actual editing process, the user can add more special effect objects of the predetermined type to the target image as needed and flexibly edit these special effect objects as needed. In some embodiments, in response to a confirmation operation on at least one special effect object, the terminal device 110 can generate a target image including a first layer, at least one layer of objects of at least one feature object, and a third layer.

[0061] In this way, while achieving sandwich presentation of special effect objects, users can be allowed to freely edit the special effect objects in the sandwich layer, making the display of special effect objects more flexible.

[0062] The user interaction at the terminal device 110 is described above in conjunction with Figures 2 to 5C. In some embodiments, in order to render the layer corresponding to the subject object, the terminal device 110 obtains subject segmentation information of the target image, and the subject segmentation information at least indicates the position of each of the at least one subject object in the target image. The terminal device 110 can extract at least one subject object from the target image based on the subject segmentation information, and then render a third layer corresponding to the at least one subject object on the second layer (and possibly the fourth layer). In some embodiments, the subject segmentation information can be obtained by the terminal device 110 after processing the target image through a subject segmentation algorithm. In some embodiments, the terminal device 110 can upload the target image to other devices, such as a remote server 130, to determine the subject segmentation object, and receive the subject segmentation information from the server 130.

[0063] The following describes the interaction process between the terminal device 110 and the server 130 with reference to FIG. 6 .

[0064] FIG6 shows a schematic diagram of a process 600 of rendering a layer according to some embodiments of the present disclosure.

[0065] At block 610, in response to receiving a user's selection of a predetermined type of special effects object, the terminal device 110 generates an algorithm request 615 for requesting an algorithm corresponding to the predetermined type of special effects object. The predetermined type of special effects object herein may also be referred to as a sandwich template, for example. The algorithm corresponding to the algorithm request 615 may be, for example, a body segmentation algorithm.

[0066] In some embodiments, the terminal device 110 may send the generated algorithm request 615 to the server 130 (also referred to as a service device) and obtain an algorithm corresponding to the predetermined type of special effect object from the server 130. The terminal device 110 may perform image processing on the target image based on the algorithm to obtain a final target image including the sandwich special effect.

[0067] The terminal device 110 may send the target image together with the algorithm request 615 to the server 130 so that the target image can be processed by the server 130. In some embodiments, the terminal device 110 may send the target image to the server 130 for processing after obtaining user authorization.

[0068] In block 620 , the terminal device 110 renders a second layer corresponding to the selected special effect object on the first layer corresponding to the target image. The second layer may also be referred to as an intermediate layer.

[0069] At block 630 , server 130 receives algorithm request 615 and the target image. Server 130 may perform corresponding image processing on the target image based on the algorithm corresponding to algorithm request 615. For example, server 130 may identify the subject in the target image based on a subject segmentation algorithm.

[0070] At block 640 , the server 130 may invoke a subject segmentation task to segment the subject in the target image to determine subject segmentation information of the target image. The subject segmentation information may include, for example, information identifying the location of each subject object in the target image.

[0071] Figures 7A to 7C show schematic diagrams of example images for subject segmentation according to some embodiments of the present disclosure. As shown in Figure 7A, the target image 700A includes a subject object 701 (i.e., the water cup in the figure). The terminal device 110 provides the target image 700A to the server 130. The server 130 can call the subject segmentation task and use the subject segmentation algorithm to identify the subject segmentation information corresponding to the target image 700A. The subject segmentation information can be, for example, as shown in Figure 7B. The subject segmentation information can, for example, include a subject segmentation map 700B corresponding to the size of the target image. Each subject object has a subject segmentation map, and the pixel value in the subject segmentation map is 0 or 1, 1 indicates that the corresponding pixel position in the target object belongs to the corresponding subject object, and 0 indicates that it does not belong. The server 130 can send the subject segmentation map 700B shown in Figure 7B to the terminal device 110.

[0072] In block 650, the terminal device 110 receives the subject segmentation information sent by the server 130 and renders a third layer corresponding to the subject object of the target image on the second layer (this may also be referred to as a subject information restoration process). Exemplarily, the terminal device 110 obtains a subject segmentation map 700B as shown in FIG. 7B from the server 130 and determines the width, height, starting point information, and the like of the subject object based on the outline information 702 of the subject object indicated by the subject segmentation map 700B. The terminal device 110 may generate a third layer 700C corresponding to the subject object as shown in FIG. 7C based on the subject segmentation map 700B and the target image 700A. The third layer 700C only includes the image corresponding to the subject object 701. The terminal device 110 may then render the third layer 700C onto the second layer corresponding to the special effect object.

[0073] It can be understood that the main body splitting process described with respect to the server in FIG. 6 can also be implemented locally on the terminal device 110 .

[0074] In summary, the image editing solution disclosed herein enables the sandwich presentation of special effects objects and allows users to freely edit the special effects objects within the sandwich layer, meeting their diverse special effects editing needs. Furthermore, when adding special effects objects to an image, the added special effects objects can be automatically prevented from obstructing the main image information.

[0075] 8 shows a schematic structural block diagram of an apparatus 800 for image editing according to certain embodiments of the present disclosure. Apparatus 800 may be implemented as or included in terminal device 110. Each module / component in apparatus 800 may be implemented by hardware, software, firmware, or any combination thereof.

[0076] As shown in Figure 8, the device 800 includes an operation receiving module 810, which is configured to receive a user's selection operation of a predetermined type of first special effect object in the editing interface of the target image. The device 800 also includes a rendering module 820, which is configured to render a second layer corresponding to the first special effect object on the first layer corresponding to the target image in response to the selection operation of the first special effect object, and render a third layer corresponding to at least one main object in the target image on the second layer. The device 800 also includes an image determination module 830, which is configured to determine the edited or confirmed first special effect object in the second layer based on the user's editing operation or confirmation operation on the first special effect object, and obtain an edited target image, wherein the edited target image includes the first layer, the second layer, and the third layer.

[0077] In some embodiments, the rendering module 820 includes: an identification presentation module, configured to present the subject identifications corresponding to the multiple candidate subject objects in the editing interface if multiple candidate subject objects are identified from the target image; a selection receiving module, configured to receive a user's selection of the subject identification of at least one subject object among the multiple candidate subject objects; and a first rendering module, configured to render a third layer corresponding to the selected at least one subject object on the second layer.

[0078] In some embodiments, the subject identification corresponding to each subject object includes at least one of the following: contour identification information for defining the subject object from the target image, or an object indicator presented at a position corresponding to the subject object in the target image.

[0079] In some embodiments, the rendering module 820 includes: a segmentation information acquisition module, configured to acquire subject segmentation information of the target image, the subject segmentation information at least indicating the position of each of at least one subject object in the target image; and a subject object extraction module, configured to extract at least one subject object from the target image based on the subject segmentation information; and a second rendering module, configured to render a third layer corresponding to at least one subject object on the second layer.

[0080] In some embodiments, the segmentation information acquisition module includes: a sending module configured to send the target image to the service device; and a receiving module configured to receive the subject segmentation information from the service device.

[0081] In some embodiments, the device 800 also includes: a second operation receiving module, configured to receive a user's selection operation on a second special effects object of a predetermined type; a third rendering module, configured to render a fourth layer corresponding to the second special effects object on the first layer corresponding to the target image in response to the selection operation on the second special effects object, and the fourth layer is below the third layer; and a second object determination module, configured to determine the edited or confirmed second special effects object in the fourth layer based on the user's editing operation or confirmation operation on the second special effects object, wherein the edited target image also includes the fourth layer.

[0082] In some embodiments, the editing operation of the second special effect object is independent of the editing operation of the first special effect object.

[0083] In some embodiments, the editing operation on the first special effect object or the second special effect object includes at least one of the following: a position movement operation, a direction movement operation, and a scaling operation.

[0084] The modules and / or units included in the device 800 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules and / or units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the modules and / or units in the device 800 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0085] Figure 9 shows a block diagram of an electronic device 900 that can implement one or more embodiments of the present disclosure. It should be understood that the electronic device 900 shown in Figure 9 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 900 shown in Figure 9 can be used to implement the terminal device 110 of Figure 1 or the apparatus 800 of Figure 8.

[0086] As shown in FIG9 , electronic device 900 is a general-purpose electronic device. Components of electronic device 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. Processing unit 910 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 920. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 900.

[0087] The electronic device 900 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 900, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 920 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 930 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 900.

[0088] The electronic device 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 9 , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 920 may include a computer program product 925 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0089] The communication unit 940 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 900 can be implemented as a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 900 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0090] The input device 950 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 960 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 900 may also communicate with one or more external devices (not shown) through the communication unit 940 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 900, or with any device that allows the electronic device 900 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0091] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above.

[0092] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0094] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0095] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0096] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A method for image editing, comprising: In the editing interface of the target image, receiving a user's selection operation of a first special effect object of a predetermined type; In response to the selection operation on the first special effect object, Rendering a second layer corresponding to the first special effect object on the first layer corresponding to the target image, and Rendering a third layer corresponding to at least one main object in the target image on the second layer; as well as Based on the user's editing operation or confirmation operation on the first special effect object, the edited or confirmed first special effect object is determined in the second layer to obtain an edited target image, and the edited target image includes the first layer, the second layer and the third layer.

2. The method according to claim 1, wherein rendering a third layer corresponding to at least one main object in the target image on the second layer comprises: If multiple candidate subject objects are identified from the target image, subject identifiers corresponding to the multiple candidate subject objects are presented on the editing interface; receiving a selection by the user of a subject identifier of at least one subject object among the plurality of candidate subject objects; as well as A third layer corresponding to the selected at least one subject object is rendered on the second layer.

3. The method according to claim 2, wherein the subject identifier corresponding to each subject object includes at least one of the following: Contour identification information, used to define the subject object from the target image, or An object indicator is presented at a position corresponding to the subject object in the target image.

4. The method according to claim 1, wherein rendering a third layer corresponding to at least one main object in the target image on the second layer comprises: Acquire subject segmentation information of the target image, wherein the subject segmentation information at least indicates a position of each of the at least one subject object in the target image; as well as extracting the at least one subject object from the target image based on the subject segmentation information; as well as A third layer corresponding to the at least one main object is rendered on the second layer.

5. The method according to claim 4, wherein obtaining the subject segmentation information of the target image comprises: sending the target image to a service device; as well as The body split information is received from the service device.

6. The method according to claim 1, further comprising: receiving a user's selection operation of a second special effect object of the predetermined type, In response to the selection operation on the second special effect object, Rendering a fourth layer corresponding to the second special effect object on the first layer corresponding to the target image, wherein the fourth layer is below the third layer; as well as Based on the user's editing operation or confirmation operation on the second special effect object, determining the edited or confirmed second special effect object in the fourth layer, The edited target image also includes the fourth layer. 7 . The method according to claim 6 , wherein an editing operation of the second special effect object is independent of an editing operation of the first special effect object.

8. The method according to any one of claims 1 to 7, wherein the editing operation on the first special effect object or the second special effect object comprises at least one of the following: a position moving operation, a direction moving operation, and a scaling operation.

9. A device for image editing, comprising: A selection receiving module is configured to receive a user's selection operation of a first special effect object of a predetermined type in an editing interface of a target image; a rendering module, configured to, in response to the selection operation on the first special effect object, render a second layer corresponding to the first special effect object on the first layer corresponding to the target image, and render a third layer corresponding to at least one main object in the target image on the second layer; as well as The image determination module is configured to determine the edited or confirmed first special effect object in the second layer based on the user's editing operation or confirmation operation on the first special effect object, and obtain an edited target image, wherein the edited target image includes the first layer, the second layer and the third layer.

10. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 8 when executed by the at least one processing unit.

11. A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images