Methods, apparatus, devices, and storage media for image editing

Abstract

Embodiments of this disclosure provide a method, apparatus, device, and storage medium for image editing. The method includes, in an editing interface for a target image, accepting a user selection operation for a first effect object of a predetermined type; rendering a second layer corresponding to the first effect object on a first layer corresponding to the target image in response to the selection operation for the first effect object; rendering a third layer corresponding to at least one subject object in the target image on the second layer; and determining the edited or confirmed first effect object on the second layer based on the user's editing or confirmation operation for the first effect object, thereby obtaining an edited target image, wherein the edited target image includes the first layer, the second layer, and the third layer. In this way, a sandwich presentation of effect objects can be realized, allowing the user to freely edit the effect objects in the sandwich layers and satisfying the user's diverse effect editing needs.

Description

Technical Field

[0005]

[0001] This application claims the priority of a Chinese patent application filed on September 8, 2023, with the title "Method, Apparatus, Device, and Storage Medium for Image Editing" and the application number 202311161804.5, the entire content of which is incorporated herein by reference.

[0002] Exemplary embodiments of the present disclosure generally relate to information processing, and more specifically, to a method, apparatus, device, and computer-readable storage medium for image editing.

Background Art

[0003] Currently, applications designed to provide various services to users are increasing rapidly. For example, users can create, view, comment on, and retweet various contents in applications, including various media contents such as videos, images, image sets, and audios. Further, these applications support interaction with users so that users can shoot or create contents. When creating contents, an effect function can also be provided to users.

[0004] For example, users can add effect elements, such as animals, landscapes, and items, to the created contents.

Summary of the Invention

[0005] A first aspect of this disclosure provides a method for image editing. This method includes, in an editing interface for a target image, accepting a user selection operation for a first effect object of a predetermined type; rendering a second layer corresponding to the first effect object on a first layer corresponding to the target image in response to the selection operation for the first effect object; rendering a third layer corresponding to at least one subject object in the target image on the second layer; and determining the edited or confirmed first effect object on the second layer based on the user's editing or confirmation operation for the first effect object, thereby obtaining an edited target image, the edited target image comprising the first layer, the second layer, and the third layer.

[0006] A second aspect of this disclosure provides an apparatus for image editing. The apparatus comprises: a selection receiving module configured to accept a user selection operation to a first effect object of a predetermined type in an editing interface for a target image; a rendering module configured to render a second layer corresponding to the first effect object on a first layer corresponding to the target image, and render a third layer corresponding to at least one subject object in the target image on the second layer, in response to the selection operation to the first effect object; and an image determination module configured to determine the edited or confirmed first effect object on the second layer based on the user's editing or confirmation operation to the first effect object, and to obtain an edited target image, wherein the edited target image includes a first layer, a second layer, and a third layer.

[0007] A third aspect of the present disclosure provides an electronic device comprising at least one processing unit and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit. When the instructions are executed by the at least one processing unit, the electronic device causes to perform the method described in the first aspect of the present disclosure.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium on which a computer program is stored, and which is executable by a processor to perform the method described in the first aspect of this disclosure.

[0009] It should be understood that the contents described in this section are not intended to limit the main or important features of the embodiments of this disclosure, nor to limit the scope of this disclosure. Other features of this disclosure will be readily apparent from the following description. [Brief explanation of the drawing]

[0010] The above and other features, advantages, and aspects of each implementation form disclosed herein will become clearer when viewed in conjunction with the attached drawings and the following detailed description. In the attached drawings, identical or similar reference numerals indicate identical or similar elements.

[0011] [Figure 1] A schematic diagram of an exemplary environment in which embodiments of this disclosure may be implemented is shown.

[0012] [Figure 2] A flowchart of the image editing process according to some embodiments of this disclosure is shown.

[0013] [Figure 3A] Schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure are shown. [Figure 3B]Schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure are shown. [Figure 3C] Schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure are shown. [Figure 3D] Schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure are shown.

[0014] [Figure 4A] The following are schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure, in which the target image includes multiple subject objects. [Figure 4B] The following are schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure, in which the target image includes multiple subject objects.

[0015] [Figure 5A] The following are schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure, in which the target image includes multiple effect objects. [Figure 5B] The following are schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure, in which the target image includes multiple effect objects. [Figure 5C] The following are schematic diagrams of exemplary editing interfaces relating to some embodiments of this disclosure, in which the target image includes multiple effect objects.

[0016] [Figure 6] A schematic diagram of the layer rendering process according to some embodiments of this disclosure is shown.

[0017] [Figure 7A] Schematic diagrams of exemplary images for subject segmentation according to some embodiments of the present disclosure are shown. [Figure 7B] Schematic diagrams of exemplary images for subject segmentation according to some embodiments of the present disclosure are shown. [Figure 7C]A schematic diagram of an exemplary image for subject segmentation according to some embodiments of the present disclosure is shown.

[0018] [Figure 8] A schematic configuration block diagram of an apparatus for image editing according to some embodiments of the present disclosure is shown.

[0019] [Figure 9] A block diagram of an electronic device that can be used to implement some embodiments of the present disclosure is shown.

Embodiments for Carrying Out the Invention

[0020] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein. Rather, it should be understood that these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the protection scope of the present disclosure.

[0021] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as non-limiting inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter.

[0022] The term "in response to" indicates that the corresponding event occurs or the condition is met. It should be understood that the timing of the execution of a subsequent action in response to that event or condition is not necessarily strongly correlated with the time the event occurred or the condition was met. In some cases, the subsequent action may be executed immediately upon the occurrence of the event or the fulfillment of the condition, while in others, the subsequent action may be executed some time after the event occurred or the condition was met.

[0023] Please understand that the data related to this proposed technology (including, but not limited to, the data itself, its acquisition, or its use) must comply with the requirements of applicable laws and regulations.

[0024] Before using the technical proposals disclosed in each example of this publication, please understand that you should notify the user of the types of personal information related to this publication, the scope of use, and the usage scenarios through appropriate means in accordance with relevant laws and regulations, and obtain the user's consent.

[0025] For example, in response to receiving an unsolicited request from a user, the system may send a prompt message to the user explicitly informing them that the requested action requires access to and use of the user's personal information. Based on this prompt message, the user can autonomously choose whether or not to provide personal information to software or hardware such as electronic devices, applications, servers, or storage media that perform the actions of the proposed technical demonstration of this disclosure.

[0026] As an optional and non-exclusive implementation, a method for sending a prompt message to the user in response to receiving an unsolicited request from the user may, for example, be via a pop-up window, in which case the prompt message may be presented in text. Furthermore, the pop-up window may further include selection controls for the user to choose whether to "agree" or "disagree" to providing personal information to the electronic device.

[0027] The above notice and user authorization process are merely illustrative and do not limit the implementation of this disclosure. It should be understood that other methods that comply with applicable laws and regulations may also be applied to implementations of this disclosure.

[0028] Figure 1 shows a schematic diagram of an exemplary environment 100 in which embodiments of the present disclosure may be implemented. In this exemplary environment 100, an application 120 is installed on a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or an accessory device of the terminal device 110.

[0029] In some embodiments, application 120 may be a content sharing application that can provide users 140 with a variety of services related to media content items, including viewing, commenting on, retweeting, creating (e.g., taking photos and / or editing), and posting content. In some embodiments, application 120 may be a content editing application, such as an image and / or video editing application.

[0030] In the environment 100 of Figure 1, when application 120 is active, terminal device 110 can present the interface 150 of application 120. Interface 150 may include various pages that application 120 can provide, such as a content presentation page, a content creation page, a content posting page, a message page, and a personal homepage. Application 120 may provide content creation functions, including taking, uploading, editing, and / or posting media content items. In some implementations, application 120 may provide content viewing functions for viewing various content posted on application 120. In some implementations, application 120 may further provide interaction functions, including interaction with other users (e.g., adding friends) and interaction with various content (including liking, favorites, and comments on content).

[0031] In some embodiments, the terminal device 110 communicates with the server 130 to provide services to the application 120. The terminal device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, and includes mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book readers, game devices, or any combination thereof, and further includes accessories and peripherals for these devices, or any combination thereof. In some embodiments, the terminal device 110 may also support any type of user-specific interface (e.g., “wearable” circuitry). The server 130 may be any type of computing system / server capable of providing computing power, including but not limited to mainframes, edge computing nodes, and computing devices in a cloud environment.

[0032] The structure and function of each element in Environment 100 are described for illustrative purposes only and should not be understood as limiting the scope of this disclosure in any way.

[0033] As mentioned earlier, users may expect to add several elements (e.g., text and various graphics) to the created content. For example, users can perform image editing on an image, adding text information to the image using text tools, or adding image elements using other sticker-type effects. Typically, such text and image elements are presented directly overlaid on the top layer of the user image. Such image editing effects are singular and may obscure the main area of ​​the image (e.g., the subject object in the image). In some editing scenarios, it would be desirable to have more ways to present sticker-type effects.

[0034] In contrast, embodiments of the present disclosure propose an improved image editing solution. According to this solution, the image editing interface accepts a user selection operation for a predetermined type of effect object. In response to this selection operation, a layer corresponding to the effect object is rendered on the layer corresponding to the image, and a layer corresponding to at least one subject object in the image is rendered on the layer corresponding to the effect object. Based on the user's editing or confirmation operation on the effect object, the effect object is determined and an edited image is obtained. The edited image includes a layer corresponding to the image, a layer corresponding to the effect object, and a layer corresponding to at least one subject object.

[0035] According to the image editing solution disclosed herein, when effect objects are added to the image being edited, the image editing process achieves a sandwich presentation of effect objects, resulting in a variety of rich visual effects with a sense of depth. Furthermore, users can freely edit the effect objects in the sandwich layer, satisfying their diverse effect editing needs.

[0036] Hereinafter, several exemplary embodiments of this disclosure will be described with reference to the attached drawings.

[0037] Figure 2 shows a flowchart of an image editing process 200 according to some embodiments of the present disclosure. Process 200 may be implemented in a terminal device 110. For the sake of ease of discussion, process 200 will be described with reference to the environment 100 in Figure 1. The operations performed by the terminal device 110 as described above, and the operations performed by the terminal device 110 described thereafter, may specifically be performed by relevant applications installed on the terminal device 110.

[0038] In box 210, terminal device 110 accepts a user selection operation for a predetermined type of first effect object in the target image editing interface.

[0039] "Effect object" refers to a visual object that can be selected using the effect editing function provided in image editing. In this specification, the effect editing function is also called the template function or tool function, and refers to a function that can add at least certain visual effects to an image, such as adding static or dynamic objects to the screen, providing interaction effects, changing the color contrast of the screen, or changing a person's makeup. In some examples, the effect object may include dynamic effect objects, such as dynamic objects with animation effects. The effect object may also include static effect objects, such as adding static objects. In some embodiments, the effect object may include augmented reality (AR) effects to provide AR effects.

[0040] In some embodiments, the target image to be edited may be a single image or a video image within a video. For example, in the video editing process, the user may be supported to edit one or more video images within a video. In the image editing process, the user may be allowed to request the use of specific effect objects or tool objects.

[0041] In embodiments of this disclosure, the user is provided with a selection of a predetermined type of effect object during the image editing process. As will be described in more detail below, this predetermined type of effect object is added sandwiched between the subject object and the background of the target image. This type of effect object is sometimes referred to as a sandwich effect.

[0042] In some embodiments, the terminal device 110 may present operational controls related to the selection of effect objects in the image editing interface. Figures 3A to 3D show schematic diagrams of exemplary editing interfaces according to some embodiments of the present disclosure. As shown in Figure 3A, the editing interface 300A of the target image 310 presents at least an effect selection control 301. In response to detection of a selection operation on the effect selection control 301, the terminal device 110 may present at least one type of effect object. Exemplarily, at least one type of effect object may include, but is not limited to, an effect object that may be presented on the top layer of the target image, an effect object that may be presented on the bottom layer of the target image, or an effect object that may be presented between the bottom layer of the target image and the top layer of the subject object.

[0043] In discussing the embodiments of this disclosure, we assume that the user chooses to add a predetermined type of effect object to the target image, i.e., an effect object that can be presented between the lowest layer of the target image and the top layer of the subject object. The first effect object may be any suitable object, such as text, an image, an icon, or a GIF image. This disclosure is not limited thereto. It should be understood that, as needed for editing, other types of effect objects, such as effect objects rendered on the top layer, may be configured in the image editing interface.

[0044] The terminal device 110 may further present the first effect object in association with the target image in response to a selection operation for a predetermined type of first effect object. Specifically, referring to Figure 2, in box 220, the terminal device 110, in response to a selection operation for a predetermined type of first effect object, renders a second layer corresponding to the first effect object on a first layer corresponding to the target image, and renders a third layer corresponding to at least one subject object in the target image on the second layer. In other words, the first effect object selected by the user is rendered on an intermediate layer, and a layer corresponding to the subject object of the target image is further rendered on the intermediate layer. In this way, when sandwich presentation is performed, it is possible to automatically avoid the added effect object obscuring a specific image subject, present a sandwich and three-dimensional effect effect different from a "sticker" which is generally presented on the top layer, and make the effect object and the image appear more integrated.

[0045] As shown in Figure 3B, in the editing interface 300B, in response to the user selecting the effect object 320, the terminal device 110 renders a layer corresponding to the effect object 320 (i.e., the first effect object) (i.e., the second layer) on the layer corresponding to the target image 310 (i.e., the first layer). The visual effect of the editing interface 300B is to superimpose the effect object 320 onto the target image 310.

[0046] Furthermore, the terminal device 110 may determine at least one subject object in the target image. Here, the at least one subject object may be at least one object in the foreground area of ​​the target image. The terminal device 110 may determine a layer corresponding to the at least one subject object as a third layer. The terminal device 110 can achieve sandwich presentation of effect objects by rendering the third layer on top of the second layer corresponding to the first effect object. As shown in Figure 3C, in the editing interface 300C, the terminal device 110 may, after rendering a layer corresponding to the effect object 320 on the layer corresponding to the target image 310, render a layer corresponding to the subject object 315 (e.g., the illustrated cup) in the target image 310 (i.e., the third layer) on top of the layer corresponding to the effect object 320.

[0047] Furthermore, the position, size, and contour of at least one subject object in the third layer are rendered to be exactly the same as the position, size, and contour of at least one subject object in the first layer. In other words, after the third layer is rendered on the second layer, at least one subject object in the third layer will perfectly overlap with at least one subject object in the first layer.

[0048] In some embodiments, to ensure the user's image editing experience, the process of rendering a second layer on a first layer and a third layer on a second layer is not visible to the user. That is, the editing interface 300C is presented to the user after rendering is complete, without presenting the editing interface 300B to the user. From the user's perspective, after starting to select a first effect object of a predetermined type, they directly see the editing interface 300C as shown in Figure 3C, that is, the sandwich effect of the effect objects is directly presented. In some embodiments, the application may present the layer-by-layer rendering process to the user.

[0049] In some embodiments, the at least one subject object corresponding to the third layer may be, for example, all subject objects in the target image. Specifically, the at least one subject object may be one or more subject objects automatically recognized from the target image.

[0050] In some embodiments, at least one subject object corresponding to the third layer may be at least some of the subject objects among a plurality of subject objects in the target image. Specifically, what is rendered in the top layer may be one or more subject objects selected by the user from the subject recognition results. In some embodiments, if a plurality of subject object candidates are recognized from the target image, the terminal device 110 may present a subject identifier corresponding to each of the plurality of subject object candidates in the editing interface. The subject identifier corresponding to each subject object includes at least one of contour identification information for defining this subject object from the target image, or an object indicator presented at the position corresponding to this subject object in the target image.

[0051] The terminal device 110 accepts the user's selection of a subject identifier for at least one subject object from among a plurality of subject object candidates, and may render a third layer on the second layer corresponding to the selected at least one subject object. The terminal device 110 may, for example, determine that a selection for a corresponding subject object has been detected in response to detecting a selection operation for a subject identifier and / or a selection operation for a subject object. The terminal device 110 then determines the layer corresponding to the selected at least one subject object as the third layer and renders that third layer on the second layer.

[0052] Figures 4A and 4B show schematic diagrams of exemplary editing interfaces according to some embodiments of the present disclosure, in which the target image includes multiple subject objects. As shown in Figure 4A, if the target image 410 includes multiple subject object candidates (subject object 411 and subject object 412), the terminal device 110 may present subject identifiers corresponding to each of the multiple subject object candidates in the editing interface 400A. Specifically, the terminal device 110 may present contour identification information 413 and object indicator 414 of subject object 411 in association with subject object 411 in the editing interface 400A. The terminal device 110 may also present contour identification information 415 and object indicator 416 of subject object 412 in association with subject object 412 in the editing interface 400A. The contour identification information (including contour identification information 413 and contour identification information 414) may be used to finely represent the contours of the corresponding subject objects, or, for example, a bounding box may be used to define individual subject objects from the target image. Object indicators 414 and 416 may include, for example, corresponding anchors and labels (e.g., the illustrated labels "Subject 1" and "Subject 2"). In some embodiments, the terminal device 110 displays only contour identification information corresponding to the subject object, only the object indicator corresponding to the subject object, or both the contour identification information and the object indicator.

[0053] Furthermore, in response to receiving selection operations for subject object 412, contour identification information 415, and / or object indicator 416, the terminal device 110 determines that it has received a selection operation for subject object 412. The terminal device 110 may also determine the layer corresponding to subject object 412 as the third layer.

[0054] As shown in Figure 4B, in the editing interface 400B, the layer corresponding to the target image 410 is the first layer, the layer corresponding to the first effect object (for example, the multiple clouds shown in Figure 4B) is the second layer, and the layer corresponding to the subject object 412 is the third layer. The terminal device 110 renders the second layer on the first layer and the third layer on the second layer. As a result, the subject object 412 is rendered on the top layer and is not obscured by the first effect object, while the subject object 411 may be obscured by the first effect object (depending on the placement of the first effect object). In this way, the user can choose which subject objects to present on the top layer of the target image (i.e., which subject objects should not be obscured by the first effect object, and which subject objects should be obscured by the first effect). While enabling the sandwich presentation of effect objects, this feature allows users to select the subject object presented on the top layer of the image, satisfying diverse user effect editing needs and further improving the user experience with effects. It should be noted that when selecting subject objects, users may choose to select only a single subject object placed on the top layer, or they may choose to select multiple subject objects.

[0055] The above describes an example in which the user selects one effect object of a predetermined type, but in some embodiments, the user may select multiple effect objects of a predetermined type. Specifically, the terminal device 110 may accept a user selection operation for a second effect object of a predetermined type. In response to the selection operation for the second effect object, the terminal device 110 may render a fourth layer corresponding to the second effect object on the first layer corresponding to the target image. The fourth layer is below the third layer. The fourth layer corresponding to the second effect object may be above or below the second layer corresponding to the first effect object, but it is always between the first layer corresponding to the target image and the third layer corresponding to the subject object. In some embodiments, the terminal device 110 may sort the different effect objects in descending or ascending order of user selection, depending on the order in which the user selected the effect objects. The first selected effect object may be at the bottom or top of the effect object layer. It should be noted that regardless of the order of the layers corresponding to the effect objects, all effect objects of a predetermined type are always sandwiched between the first layer and the third layer.

[0056] In box 230, terminal device 110 determines the edited or confirmed first effect object in the second layer based on the user's editing or confirmation operation on the first effect object, and acquires the edited target image, which includes the first layer, the second layer and the third layer. The editing operation here may include, for example, at least one of position shift operation, direction shift operation or scaling operation.

[0057] In some embodiments, the terminal device 110 can render a second layer on a first layer, then render a third layer on the second layer, and then generate a target image including a sandwich effect. The terminal device 110 may also adjust the first effect object in the second layer based on user editing operations on the first effect object. Specifically, the terminal device 110 may adjust the display position of the first effect object in the second layer in response to receiving a user position shift operation on the first effect object. The terminal device 110 may also adjust the display direction of the first effect object in the second layer in response to receiving a user direction shift operation on the first effect object. The terminal device 110 may also adjust the display size of the first effect object in the second layer in response to receiving a user zoom operation on the first effect object. As shown in Figure 3D, in response to receiving an editing operation on the effect object 320, the terminal device 110 adjusts the display position, display direction, and display size of the effect object 320 in the corresponding layer.

[0058] Similarly, if the user selects multiple effect objects of a predetermined type, the terminal device 110 may determine which effect objects have been edited or confirmed in the layer corresponding to each of the multiple effect objects, based on the user's editing or confirmation operations on each of the multiple effect objects. The edited target image includes multiple layers corresponding to the multiple effect objects. Exemplarily, if the user also selects a second effect object, the terminal device 110 may determine which effect object has been edited or confirmed in the fourth layer, based on the user's editing or confirmation operations on the second effect object. The edited target image further includes the fourth layer. In some embodiments, editing operations on the second effect object are independent of editing operations on the first effect object; that is, editing operations on different effect objects are independent of each other and do not affect each other.

[0059] Figures 5A to 5C show schematic diagrams of exemplary editing interfaces in which a target image includes multiple effect objects, according to some embodiments of the present disclosure. As shown in Figure 5A, the target image 510 in the editing interface 500A includes a first layer on which the target image 510 is located, a second layer corresponding to an effect object 520, and a third layer corresponding to a subject object 515. As shown in Figure 5B, in response to receiving a user selection operation for an effect object 530 (i.e., the second effect object), the terminal device 110 may further render a fourth layer corresponding to the effect object 530 between the first and third layers to present the target image 510 shown in the editing interface 500B. As shown in Figure 5C, in response to receiving further editing operations for the effect object 520 and the effect object 530, respectively, the terminal device 110 may adjust the effect object 520 in the second layer and the effect object 530 in the fourth layer, respectively, to present the target image 530 shown in the editing interface 500C.

[0060] While the first and second effect objects have been described above, it should be understood that in the actual editing process, the user can add more effect objects of a predetermined type to the target image as needed, and furthermore, flexibly edit these effect objects as needed. In some embodiments, in response to a confirmation operation on at least one effect object, the terminal device 110 may generate a target image including a first layer, at least one layer corresponding to each of the at least one effect object, and a third layer.

[0061] In this way, while achieving a sandwich presentation of effect objects, it becomes possible for users to freely edit the effect objects in the sandwich layer, making the display of effect objects more flexible.

[0062] The user interaction in the terminal device 110 has been described above in conjunction with Figures 2 to 5C. In some embodiments, in order to render a layer corresponding to a subject object, the terminal device 110 acquires subject segmentation information of the target image, which indicates the position of at least one subject object in the target image. Based on the subject segmentation information, the terminal device 110 may extract at least one subject object from the target image and render a third layer corresponding to at least one subject object on the second layer (and possibly a fourth layer). In some embodiments, the subject segmentation information may be obtained after the terminal device 110 has processed the target image by a subject segmentation algorithm. In some embodiments, the terminal device 110 may upload the target image to another device, such as a remote server 130, determine the subject segmentation objects, and receive subject segmentation information from the server 130.

[0063] The interaction process between the terminal device 110 and the server 130 will be explained below in conjunction with Figure 6.

[0064] Figure 6 shows a schematic diagram of a layer rendering process 600 according to some embodiments of the present disclosure.

[0065] In box 610, terminal device 110, in response to receiving a user selection for a predetermined type of effect object, generates an algorithm request 615 to request an algorithm corresponding to this predetermined type of effect object. A predetermined type of effect object in this specification may also be called, for example, a sandwich template. The algorithm corresponding to algorithm request 615 may be, for example, a subject segmentation algorithm.

[0066] In some embodiments, the terminal device 110 may send the generated algorithm request 615 to the server 130 (which may also be called a service device) and obtain an algorithm from the server 130 corresponding to this predetermined type of effect object. Based on this algorithm, the terminal device 110 may perform image processing on the target image itself to obtain a final target image including the sandwich effect.

[0067] The terminal device 110 may send the target image to the server 130 along with the algorithm request 615 so that the server 130 can process the target image. In some embodiments, the terminal device 110 may send the target image to the server 130 for processing after obtaining user authorization.

[0068] In box 620, terminal device 110 renders a second layer corresponding to the selected effect object on a first layer corresponding to the target image, and the second layer may be called an intermediate layer.

[0069] In box 630, server 130 receives algorithm request 615 and target image. Server 130 may perform corresponding image processing on the target image based on the algorithm corresponding to algorithm request 615. For example, server 130 may recognize subjects in the target image based on a subject segmentation algorithm.

[0070] In box 640, server 130 may call a subject segmentation task to determine subject segmentation information for the target image, thereby dividing the subjects within the target image. Subject segmentation information may include, for example, positions that can identify each subject object within the target image.

[0071] Figures 7A to 7C show schematic diagrams of exemplary images for subject segmentation according to some embodiments of the present disclosure. As shown in Figure 7A, the target image 700A includes a subject object 701 (i.e., a cup in the figure). The terminal device 110 provides the target image 700A to the server 130, which may invoke a subject segmentation task to identify subject segmentation information corresponding to the target image 700A using a subject segmentation algorithm. The subject segmentation information may be shown, for example, in Figure 7B. The subject segmentation information may include, for example, a subject segmentation map 700B corresponding to the size of the target image. Each subject object has one subject segmentation map, and the pixel values ​​in this subject segmentation map are 0 or 1, where 1 indicates that the corresponding pixel position in the target object belongs to the corresponding subject object, and 0 indicates that it does not belong. The server 130 may transmit the subject segmentation map 700B shown in Figure 7B to the terminal device 110.

[0072] In box 650, terminal device 110 receives subject segmentation information transmitted from server 130 and renders a third layer corresponding to the subject object of the target image on the second layer (this may also be called subject information recovery processing). Exemplaryly, terminal device 110 obtains the subject segmentation map 700B shown in Figure 7B from server 130 and determines the width, height, starting point information, etc. of the subject object based on the contour information 702 of the subject object shown in the subject segmentation map 700B. Terminal device 110 may generate a third layer 700C corresponding to the subject object shown in Figure 7C based on the subject segmentation map 700B and the target image 700A. The third layer 700C contains only the image corresponding to the subject object 701. Terminal device 110 may further render the third layer 700C on the second layer corresponding to the effect object.

[0073] It should be understood that the subject segmentation process described for the server in Figure 6 may also be implemented locally in the terminal device 110.

[0074] In summary, the image editing solution disclosed herein enables the presentation of effect objects in a sandwich format, allowing users to freely edit the effect objects in the sandwich layer and satisfying their diverse effect editing needs. Furthermore, when adding effect objects to an image, the added effect objects can automatically avoid obscuring the image subject information.

[0075] Figure 8 shows a schematic block diagram of an image editing apparatus 800 according to some embodiments of the present disclosure. The apparatus 800 may be implemented as a terminal device 110 or may be included in the terminal device 110. Individual modules / components within the apparatus 800 may be implemented by hardware, software, firmware, or any combination thereof.

[0076] As shown in Figure 8, the device 800 includes an operation reception module 810 configured to accept a user selection operation to a first effect object of a predetermined type in the target image editing interface. The device 800 further includes a rendering module 820 configured to render a second layer corresponding to the first effect object on a first layer corresponding to the target image, and render a third layer corresponding to at least one subject object in the target image on the second layer, in response to a selection operation to the first effect object. The device 800 further includes an image determination module 830 configured to determine the edited or confirmed first effect object in the second layer based on a user editing or confirmation operation to the first effect object, and to obtain an edited target image, the edited target image including the first layer, the second layer and the third layer.

[0077] In some embodiments, the rendering module 820 includes an identifier presentation module configured to present a subject identifier corresponding to each of the multiple subject object candidates to the editing interface when multiple subject object candidates are recognized from the target image; a selection acceptance module configured to accept a user selection for a subject identifier of at least one subject object among the multiple subject object candidates; and a first rendering module configured to render a third layer corresponding to the selected at least one subject object on a second layer.

[0078] In some embodiments, the subject identifier corresponding to each subject object includes at least one of contour identification information for defining the subject object from the target image, or an object indicator presented at the location in the target image corresponding to the subject object.

[0079] In some embodiments, the rendering module 820 comprises a segmentation information acquisition module configured to acquire subject segmentation information of a target image, wherein the subject segmentation information indicates at least the position of each of at least one subject object in the target image; a subject object extraction module configured to extract at least one subject object from the target image based on the subject segmentation information; and a second rendering module configured to render a third layer corresponding to at least one subject object on a second layer.

[0080] In some embodiments, the segmentation information acquisition module comprises a transmitting module configured to send a target image to a service device, and a receiving module configured to receive subject segmentation information from the service device.

[0081] In some embodiments, the apparatus 800 includes a second operation receiving module configured to receive a user selection operation to a second effect object of a predetermined type, and a third rendering module configured to render a fourth layer corresponding to the second effect object on a first layer corresponding to a target image in response to the selection operation to the second effect object, wherein the fourth layer further includes a third rendering module located below the third layer, and a second object determination module configured to determine the edited or confirmed second effect object in the fourth layer based on a user editing or confirmation operation to the second effect object, and the edited target image further includes the fourth layer.

[0082] In some embodiments, editing operations on the second effect object are independent of editing operations on the first effect object.

[0083] In some embodiments, the editing operation on the first or second effect object includes at least one of a position shift operation, a direction shift operation, or a scaling operation.

[0084] Modules and / or units included in the device 800 may be implemented using a variety of means, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules and / or units may be implemented using software and / or firmware (e.g., machine-executable instructions stored on a storage medium). In addition to, or instead of, machine-executable instructions, some or all of the modules and / or units in the device 800 may be implemented at least partially by one or more hardware logic components. Non-limiting examples of usable and exemplary hardware logic components include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on a chip (SOCs), and composite programmable logic devices (CPLDs).

[0085] Figure 9 shows a block diagram of an electronic device 900 that may carry out one or more embodiments of the present disclosure. It should be understood that the electronic device 900 shown in Figure 9 is merely illustrative and should not constitute any limitation on the function and scope of the embodiments described herein. The electronic device 900 shown in Figure 9 may be used to implement the terminal device 110 of Figure 1 or the device 800 of Figure 8.

[0086] As shown in Figure 9, the electronic device 900 is in the form of a general-purpose electronic device. The components of the electronic device 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage devices 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. The processing unit 910 may be an actual processor or a virtual processor and is capable of performing various processes based on a program stored in memory 920. In a multiprocessor system, multiple processing units execute computer executable instructions in parallel to enhance the parallel processing capability of the electronic device 900.

[0087] The electronic device 900 typically includes multiple computer storage media. Such media may include, but are not limited to, volatile and non-volatile media, removable and non-removable media, and may be any available media accessible to the electronic device 900. Memory 920 may be volatile memory (e.g., registers, caches, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or a combination thereof. Storage device 930 may be removable or non-removable media, and may include machine-readable media such as flash drives, magnetic disks, or any other media that can be used to store information and / or data and may be accessible within the electronic device 900.

[0088] The electronic device 900 may further include other removable / non-removable, volatile / non-volatile storage media. Not shown in Figure 9, a magnetic disk drive for reading and writing to removable non-volatile magnetic disks (e.g., “floppy disks”) and an optical disk drive for reading and writing to removable non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. The memory 920 may include a computer program product 925 having one or more program modules configured to perform various methods or operations of various embodiments of the present disclosure.

[0089] The communication unit 940 enables communication with other electronic devices via a communication medium. Furthermore, the functions of the components of the electronic device 900 may be implemented as a single computing cluster or as multiple computing machines, and these computing machines can communicate via a communication connection. Therefore, the electronic device 900 can use logical connections to one or more other servers, networked personal computers (PCs), or other network nodes to operate in a networked environment.

[0090] The input device 950 may be one or more input devices, such as a mouse, keyboard, or tracking ball. The output device 960 may be one or more output devices, such as a monitor, speaker, or printer. The electronic device 900 may, if necessary, communicate with one or more external devices (not shown), such as a storage device or display device, via the communication unit 940, with one or more devices that enable a user to interact with the electronic device 900, or with any device (e.g., a network card or modem) that enables the electronic device 900 to communicate with one or more other electronic devices. Such communication may be performed via an input / output (I / O) interface (not shown).

[0091] According to exemplary implementations of this disclosure, a computer-readable storage medium is provided which stores one or more computer instructions to be executed by a processor in order to accomplish the method described above.

[0092] Each aspect of this disclosure is described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products implemented in accordance with this disclosure. It should be understood that each box in the flowcharts and / or block diagrams, and each combination of boxes in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions, when provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, generate a machine that, when executed by the computer or other programmable data processing device's processing unit, produces a device that performs the functions / operations specified in one or more boxes of a flowchart and / or block diagram. Furthermore, by storing these computer-readable program instructions, which cause computers, programmable data processing devices, and / or other devices to operate in specific manners, in a computer-readable storage medium, the computer-readable medium on which the instructions are stored has a product containing instructions that perform each of the functions / operations specified in one or more boxes of a flowchart and / or block diagram.

[0094] When computer-readable program instructions are loaded onto a computer, other programmable data processing device, or other device, a series of operational steps are executed on the computer, other programmable data processing device, or other device to generate a computer implementation process, thereby enabling the instructions executed on the computer, other programmable data processing device, or other device to realize a function / operation specified in one or more boxes of a flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate architectures, functions, and operations that may be implemented in several implemented systems, methods, and computer program products relating to this disclosure. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and a module, program segment, or part of an instruction may contain one or more executable instructions for realizing a specified logical function. In some alternative implementations, the functions associated with a box may occur in a different order than those associated with the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or in reverse order depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and combinations of boxes in the block diagram and / or flowchart, may be implemented in a dedicated hardware-based system that performs a given function or operation, or in a combination of dedicated hardware and computer instructions.

[0096] The above descriptions of the various implementations of this disclosure are illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and changes will be apparent to an ordinary art engineer without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best describe the principles, practical applications, or improvements in the technology in the market of each implementation, or to enable other ordinary art engineers in the art to understand each implementation disclosed herein.

Claims

1. In the editing interface for the target image, the user's selection operation for a predetermined type of first effect object is accepted. In response to the selection operation on the first effect object, a second layer corresponding to the first effect object is rendered on the first layer corresponding to the target image, and a third layer corresponding to at least one subject object in the target image is rendered on the second layer. Based on the user's editing or confirmation operation on the first effect object, the edited or confirmed first effect object is determined in the second layer, and an edited target image is obtained, wherein the edited target image includes the first layer, the second layer, and the third layer. Methods for image editing, including...

2. Rendering a third layer on the second layer, corresponding to at least one subject object in the target image, is: When multiple candidate subject objects are recognized from the target image, the subject identifier corresponding to each of the multiple candidate subject objects is presented to the editing interface. Accepting the user's selection of the subject identifier for at least one subject object among the plurality of subject object candidates, Rendering a third layer corresponding to the selected at least one subject object onto the second layer, The method according to claim 1, including the method described in claim 1.

3. The subject identifier corresponding to each subject object is: Contour recognition information for defining the subject object from the target image, or An object indicator is presented at the position in the target image corresponding to the subject object. The method according to claim 2, comprising at least one of the above.

4. Rendering a third layer on the second layer, corresponding to at least one subject object in the target image, is: The method involves obtaining subject segmentation information of the target image, wherein the subject segmentation information includes, at a minimum, the position of each of the at least one subject object in the target image. Based on the subject segmentation information, extract at least one subject object from the target image, Rendering a third layer corresponding to at least one subject object onto the second layer, The method according to claim 1, including the method described in claim 1.

5. Obtaining subject segmentation information of the aforementioned target image is: The target image is transmitted to the service device, Receiving the subject segmentation information from the service device, The method according to claim 4, including the method described in claim 4.

6. Accepting the user's selection operation for the predetermined type of second effect object, In response to the selection operation on the second effect object, a fourth layer corresponding to the second effect object is rendered on the first layer corresponding to the target image, wherein the fourth layer is below the third layer. Based on the user's editing or confirmation operation on the second effect object, the edited or confirmed second effect object is determined in the fourth layer. It further includes, The edited target image further includes the fourth layer, The method according to claim 1.

7. The method according to claim 6, wherein the editing operation of the second effect object is independent of the editing operation of the first effect object.

8. The method according to any one of claims 1 to 7, wherein the editing operation on the first effect object or the second effect object includes at least one of a position shift operation, a direction shift operation, or a scaling operation.

9. A selection receiving module configured to accept a user's selection operation to a first effect object of a predetermined type in the editing interface of a target image, A rendering module configured to render a second layer corresponding to the first effect object on a first layer corresponding to the target image, in response to the selection operation on the first effect object, and to render a third layer corresponding to at least one subject object in the target image on the second layer, An image determination module configured to determine the edited or confirmed first effect object in the second layer based on the user's editing or confirmation operation on the first effect object, and to obtain an edited target image, wherein the edited target image includes the first layer, the second layer and the third layer, A device for image editing, equipped with the following features.

10. At least one processing unit, An electronic device comprising at least one processing unit and at least one memory that stores instructions to be executed by the at least one processing unit, When the instruction is executed by the at least one processing unit, it causes the device to perform the method according to any one of claims 1 to 8. electronic equipment.

11. When executed by a processor, a computer program that implements the method described in any one of claims 1 to 8 is stored. Computer-readable storage medium.

Images