Method and apparatus for generating media content, and device and storage medium

Abstract

A method and apparatus for generating media content, and a device and a storage medium. The method comprises: determining a group of reference media content for media content generation (310), wherein the group of reference media content is determined from a media library associated with a user; and presenting target media content generated on the basis of the group of reference media content (320), the target media content comprising at least one foreground element superposed and displayed on a background image, wherein the foreground element comprises an image portion determined by means of segmenting corresponding reference media content in the group of reference media content. The method can improve the quality and efficiency of generating media content.

Description

A method, apparatus, device, and storage medium for generating media content. Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to a method, apparatus, device, and computer-readable storage medium for generating media content. Background Technology

[0002] In recent years, with the development of the internet, more and more users are publishing media content on online platforms. Some content creation tools allow users to edit existing media content, while others enable the creation of media content based on multiple media assets.

[0003] Summary of the Invention

[0004] In a first aspect of this disclosure, a method for generating media content is provided, comprising: determining a set of reference media content for media content generation, the set of reference media content being determined from a media library associated with a user; and presenting target media content generated based on the set of reference media content, the target media content including at least one foreground element overlaid on a background image, wherein the foreground element includes an image portion determined by segmenting corresponding reference media content in the set of reference media content.

[0005] In a second aspect of this disclosure, an apparatus for generating media content is provided. The apparatus includes: a determining module configured to determine a set of reference media content for media content generation, the set of reference media content being determined from a media library associated with a user; and a rendering module configured to render target media content generated based on the set of reference media content, the target media content including at least one foreground element overlaid on a background image, wherein the foreground element includes an image portion determined by segmenting corresponding reference media content within the set of reference media content.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 shows a schematic diagram of an example environment capable of real-time implementation of some embodiments of this disclosure;

[0011] Figures 2A to 2H illustrate example interfaces according to some embodiments of the present disclosure;

[0012] Figure 3 illustrates a flowchart of an example process for generating media content according to some embodiments of the present disclosure;

[0013] Figure 4 shows a schematic structural block diagram of an example apparatus for generating media content according to some embodiments of the present disclosure; and

[0014] Figure 5 shows a block diagram of an electronic device capable of implementing several embodiments of the present disclosure. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0019] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0020] As briefly mentioned above, with the development of the internet, more and more users are publishing media content on online platforms. In the process of publishing media content, users expect to create content based on multiple media materials. However, existing online platforms can only perform simple combinations of images or generate complex videos based on images when generating media content.

[0021] Embodiments of this disclosure propose a scheme for generating media content. According to this scheme, a set of reference media content for media content generation can be determined, the set of reference media content being determined from a media library associated with a user; and target media content generated based on the set of reference media content can be presented, the target media content including at least one foreground element overlaid on a background image, wherein the foreground element includes an image portion determined by segmenting corresponding reference media content within the set of reference media content.

[0022] In this manner, embodiments of the present disclosure can determine reference media content from a user's media library, and based on the reference media content, determine a background image and segment at least one foreground element, thereby generating target media content.

[0023] Thus, the embodiments of this disclosure can help users extract media materials from reference media content, improving the efficiency of media content processing and saving users time. Furthermore, the embodiments of this disclosure use media materials extracted from reference media content as at least one foreground element, and overlay this foreground element onto a background image to generate target media content, thereby reducing the user's editing costs.

[0024] Furthermore, the target media content generated by the embodiments of this disclosure presents at least one foreground element within the same background image. Compared to media content generated based on traditional jigsaw puzzle or image-based video modes, it retains only the core elements of the reference media content and removes irrelevant features. Therefore, the target media content generated by the embodiments of this disclosure is of higher quality and has a better presentation effect.

[0025] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0026] Example Environment

[0027] Figure 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in Figure 1, the example environment 100 may include an electronic device 110.

[0028] In this example environment 100, electronic device 110 may run an application 120 that supports user interface interaction. Application 120 may be any suitable type of application for user interface interaction, and examples may include, but are not limited to, video applications, social applications, or other suitable applications. User 140 may interact with application 120 via electronic device 110 and / or its attached devices.

[0029] In environment 100 of Figure 1, if application 120 is active, electronic device 110 can use application 120 to present interface 150 for supporting interface interaction.

[0030] In some embodiments, electronic device 110 communicates with server 130 to provide services to application 120. Electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0031] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for applications 120 that support user interface interaction in electronic devices 110.

[0032] A communication connection can be established between server 130 and electronic device 110. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, server 130 and electronic device 110 can achieve signaling interaction through the communication connection between them.

[0033] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0034] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0035] Example of generating media content

[0036] Figures 2A to 2H illustrate example interfaces 200A to 200H according to some embodiments of the present disclosure. Interfaces 200A to 200H may be provided, for example, by the electronic device 110 shown in Figure 1.

[0037] In some embodiments, as shown in FIG2A, the electronic device 110 may present target media content 205 on a target interface 200A. As an example, the electronic device 110 may present the target interface 200A in response to a trigger on the media library (e.g., opening the photo album).

[0038] In some embodiments, continuing to refer to FIG2A, the target media content 205 may be generated based on a set of reference media content. As an example, the set of reference media content is determined from a media library associated with the user. As an example, the media library associated with the user may include the user's local photo album, cloud photo album, and / or other hardware or software applications that can store the user's media content.

[0039] In some embodiments, the electronic device 110 may receive a request to generate media content. Further, in response to the generation request, the electronic device 110 may determine a set of reference media content associated with a preset theme from a media library. As an example, the preset theme may include one or more themes.

[0040] As an example, preset themes can include time themes, event themes, and / or object themes. For example, a time theme can correspond to a preset time period, such as a day or a week. For example, an event theme can include a holiday theme (e.g., Chinese New Year, Christmas, etc.), a travel theme (e.g., XXX travel diary), or a food theme (e.g., XXX local cuisine). For example, an object theme can include people (e.g., user selfies), animals (e.g., kittens), or other objects (e.g., sunrise, ocean).

[0041] In some embodiments, continuing to refer to FIG2A, electronic device 110 may determine a set of reference media content associated with an event theme (e.g., “Christmas theme”) from a media library. Further, electronic device 110 and / or server 130 may generate target media content 205 based on the set of reference content associated with the “Christmas theme”.

[0042] In some embodiments, the electronic device 110 may present a media selection interface. The media selection interface displays multiple media contents from a media library. As an example, the electronic device 110 may present the media selection interface in response to receiving a preset operation from a user (e.g., a click operation on a control associated with the media selection interface). Further, the electronic device 110 may receive a selection of a set of reference media contents from multiple media contents via the media selection interface to determine a set of reference media contents.

[0043] In some embodiments, as shown in FIG2B, electronic device 110 may present target media content 210 in editing interface 200B. As an example, target media content 210 may be generated based on a set of reference media content selected from a media selection interface. It should be understood that target media content 210 may be generated by electronic device 110, server 130 and / or other suitable electronic devices or combinations thereof.

[0044] As an example, electronic device 110 may display editing interface 200B in response to a set of reference media content being selected via a media selection interface. As an example, electronic device 110 may also display editing interface 200B in response to a set of reference media content being selected from a media library based on a preset theme.

[0045] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may provide a media control 216 in an editing interface 200B. Further, the electronic device 110 may, in response to a triggering of the media control 216, present a media selection interface to add reference media content via the media selection interface.

[0046] In some embodiments, continuing to refer to FIG2B, the target media content 210 may include a background image 211 and at least one foreground element overlaid on the background image 211. As an example, the at least one foreground element may include foreground element 212-1, foreground element 212-2, foreground element 212-3, foreground element 212-4, and foreground element 212-5, etc.

[0047] In some embodiments, at least one foreground element may include: static image content and / or dynamic image content. As an example, dynamic image content may include animated pictures and video content.

[0048] In some embodiments, the foreground element may include an image portion determined by segmenting the reference media content. As an example, electronic device 110 or server 130 may utilize an image processing model to segment an image portion from the reference media content corresponding to a predetermined object as a foreground element. Such an object may include any suitable type of object, such as a person, animal, building, food, etc. As an example, the image processing model may be implemented as a machine learning model capable of image segmentation. This disclosure is not intended to limit the specific content and training process of the image processing model.

[0049] In some embodiments, the image portion may include a first image portion determined from reference media content based on a first editing operation by a user. As an example, the first editing operation may indicate a first region to be segmented. As an example, the electronic device 110 may determine the first region to be segmented in the reference media content based on a user's first editing operation on the reference media content (e.g., drawing lines, selecting boxes, etc.) to determine the first image portion based on the first region.

[0050] In some embodiments, the image portion may include a second image portion. The second image portion is determined by segmenting a second region in the reference media content that corresponds to a preset object. As an example, the preset object may include a person (e.g., a person in a portrait photograph), an animal (e.g., an animal in a photograph), or a thematic object associated with a preset theme (e.g., a Christmas tree, Christmas gifts, Santa hats, etc.) in a set of reference media content. As an example, electronic device 110 or server 130 associated with electronic device 110 may use an image recognition model to segment (or identify) the second region corresponding to the preset object from a set of reference media content to determine the second image portion. As an example, the image recognition model may be implemented as a machine learning model capable of recognizing images. This disclosure is not intended to limit the specific content and training process of the image recognition model.

[0051] In some embodiments, electronic device 110 and / or server 130 may determine style information corresponding to at least one foreground element. The style information may indicate at least one display attribute of the at least one foreground element. Further, electronic device 110 and / or server 130 may overlay the at least one foreground element onto a background image based on the style information to generate target media content.

[0052] In some embodiments, at least one display attribute may include the display position of the foreground element, the display size of the foreground element, the display orientation of the foreground element, and / or the display level of the foreground element.

[0053] In some embodiments, electronic device 110 and / or server 130 may utilize a template to determine style information of at least one foreground element. Specifically, electronic device 110 and / or server 130 may determine a first template to be applied. The first template may include: a preset template, a template selected by a user, or a template determined based on at least one foreground element. As an example, electronic device 110 and / or server 130 may determine the first template based on a user's selection. Alternatively, electronic device 110 and / or server 130 may also use a preset template. In some embodiments, electronic device 110 and / or server 130 may also use a template determined based on at least one foreground element as the first template. Further, electronic device 110 and / or server 130 may determine style information corresponding to at least one foreground element based on the first template to be applied.

[0054] As an example, the preset template may include templates corresponding to default style information. As an example, the electronic device 110 may provide the user with multiple candidate templates. Further, the electronic device 110 may use the template selected by the user from the multiple candidate templates as the first template. As an example, the electronic device 110 and / or the server 130 may match a corresponding template from the multiple candidate templates based on the content information of at least one foreground element, to use as the first template. As an example, the electronic device 110 and / or the server 130 may match a Christmas-related template from the multiple candidate templates based on the inclusion of Christmas-related content (e.g., Christmas tree, Santa hat, etc.) in at least one foreground element, to use as the first template.

[0055] In some embodiments, at least one display attribute of at least one foreground element satisfies a preset display constraint, which at least indicates the occlusion relationship between elements. For example, the occlusion relationship can ensure the integrity of the content presented by the foreground element (e.g., a face is not occluded). For example, the display constraint can also indicate the correlation between at least one foreground element (e.g., a cloud image and a sun image can be close together, while a cloud image and a grass image can be far apart). Thus, at least one foreground element in the target media content generated by the embodiments of this disclosure can be correlated with each other, thereby ensuring the interest and harmony of the target media content.

[0056] In some embodiments, the background image of the target media content may also be determined based on the set of reference media content. In some embodiments, the electronic device 110 and / or the server 130 may determine a first media content that meets preset conditions from a set of reference media content. Further, a background image is determined based on the first reference media content. As an example, the preset conditions met by the first media content include: the first media content has few elements and a lot of white space. As an example, a blue sky image has relatively simple elements and a lot of white space in the picture (e.g., a large area of ​​blue sky), which meets the preset conditions. This is only an illustrative example; in addition to blue skies, walls, monochrome images, lakes, or images with few elements (e.g., to facilitate the removal of interfering elements) also meet the preset conditions.

[0057] In some embodiments, the background image may include at least a portion of the first media content. As an example, the background image may include a portion of a photograph.

[0058] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may present a set of reference media content in the editing interface 200B. In some embodiments, the electronic device 110 may present a set of thumbnails corresponding to a set of reference media content in the editing interface 200B. As an example, a set of thumbnails may include thumbnail 215-1, thumbnail 215-2, and thumbnail 215-3.

[0059] In some embodiments, continuing to refer to FIG2B, the foreground element includes an image portion defined by segmenting a corresponding reference media content from a set of reference media content. As an example, foreground element 212-1 includes an image portion segmented from the reference media content corresponding to thumbnail 215-1.

[0060] In some embodiments, continuing to refer to FIG2B, at least one foreground element includes a third foreground element and a fourth foreground element, which may correspond to different image portions of the same reference media content. That is, a reference media content can be segmented into one or more foreground elements. As an example, foreground element 212-1 and foreground element 212-3 correspond to different image portions of the reference media content corresponding to thumbnail 215-1.

[0061] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may, in response to the selection of a target thumbnail from a set of thumbnails, distinguishably display at least one foreground element corresponding to the target thumbnail in the target media content 210. As an example, the electronic device 110 may, in response to the selection of thumbnail 215-2, distinguishably display foreground elements 212-5 corresponding to thumbnail 215-2 in the target media content 210. As an example, distinguishing foreground elements 212 may include highlighting foreground elements 212-5, reducing the transparency of other content in the target media content 210 besides foreground elements 212-5, or presenting a separate viewing interface for foreground elements 212-5, etc.

[0062] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may distinguishably display at least one foreground element corresponding to the target thumbnail in the target media content, which may include presenting a contour element in the target media content to indicate the boundary of at least one foreground element. As an example, the electronic device 110 may display a contour element (e.g., a contour line that wraps around the foreground elements 212-5) in the target media content 210 to indicate the boundary of the foreground elements 212-5.

[0063] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may, in response to a first operation (e.g., a click or long press) on a first foreground element among at least one foreground element, present a viewing interface for reference media content corresponding to the first foreground element.

[0064] In some embodiments, as shown in FIG2C, the electronic device 110 may, in response to a first operation on a first foreground element (e.g., foreground element 212-1), present a viewing interface 200C of reference media content corresponding to the first foreground element. The electronic device 110 may distinguishably display the image portion 220 corresponding to the first foreground element (e.g., foreground element 212-1) in the viewing interface 200C. As an example, the electronic device 110 may reduce the transparency of the portion other than the image portion 220 in the viewing interface 200C of the reference media content to highlight that image portion.

[0065] In some embodiments, continuing to refer to FIG2C, the electronic device 110 may, in response to receiving a second editing operation in the viewing interface 200C, adjust the image portion 220 based on the second editing operation to update the first foreground element. As an example, as shown in FIG2C, the electronic device 110 may provide a first adjustment control 225.

[0066] In some embodiments, as shown in FIG2D, electronic device 110 and / or server 130 may adjust the region boundary corresponding to image portion 220 based on a received second editing operation. Further, electronic device 110 and / or server 130 may determine a new image portion based on the adjusted region boundary. As an example, electronic device 110 may receive a user's adjustment operation based on a first adjustment control 225 to adjust the region boundary corresponding to image portion 220. For example, before receiving the adjustment operation, the region corresponding to image portion 220 may only include a first portion 220-1 (e.g., a human figure); after receiving the adjustment operation, the new region corresponding to image portion 220 may also include a second portion 220-2 (e.g., a kitten).

[0067] In some embodiments, as shown in FIG2E, electronic device 110 and / or server 130 may determine existing foreground element 212-1 and new foreground element 212-6 based on new image portion 220. Further, electronic device 110 may add foreground element 212-6 to the target media content.

[0068] In some embodiments, continuing to refer to FIG2C, the electronic device 110 may provide a second adjustment control 235 (or cropping control) in the viewing interface 200C. Further, the electronic device 110 may receive a user's cropping operation based on the second adjustment control 235 to adjust the image portion 220.

[0069] In some embodiments, continuing to refer to FIG2C, the second editing operation may also trigger the electronic device 110 to add at least one additional visual element to the image portion 220. As an example, the electronic device 110 may provide a paintbrush control 230 in the viewing interface 200C. Further, the electronic device 110 and / or the server 130 may obtain additional visual elements (e.g., additional text content or additional image content) input by the user based on the paintbrush control 230 to present the additional visual elements in the image portion 220.

[0070] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may, in response to a second operation on a second foreground element among at least one foreground element, distinguishably display the second foreground element in the target media content 210.

[0071] In some embodiments, as shown in FIG2F, the electronic device 110 may, in response to a preset operation (e.g., a click or long press) on a second foreground element (e.g., foreground element 212-1), distinguishably display the second foreground element in the target media content 210. For example, reducing the transparency of other content besides foreground element 212-1. Furthermore, the electronic device 110 may, in response to a second user operation (e.g., a click, a long press, or triggering of a hierarchical control, etc.), adjust the display hierarchy of the second foreground element.

[0072] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may provide a position adjustment control 240-1 associated with the target media content in the editing interface 200B. Further, the electronic device 110 may adjust the display position of at least one foreground element in response to the triggering of the position adjustment control 240-1 (e.g., based on a style randomly determined from a variety of preset styles).

[0073] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may present a template control 240-2 in the editing interface 200B. Further, the electronic device 110 may present a set of preset templates in response to a triggering of the template control 240-2.

[0074] In some embodiments, as shown in FIG2G, the electronic device 110 can present a set of preset templates 245 in the editing interface 200G. The set of preset templates 245 corresponds to a set of presentation styles. The presentation styles may correspond to the style information of foreground elements. Further, the electronic device 110 can, in response to the selection of a second template from the set of preset templates 245, present updated media content generated based on the second template and a set of reference media content. As an example, the updated media content may be generated based on the style information corresponding to the second template and a set of reference media content.

[0075] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may provide an add entry 240-3 in the editing interface 200B of the target media content 210. Further, the electronic device 110 may present a set of candidate materials in response to triggering the add entry 240-3. Further, the electronic device 110 may add an additional foreground element corresponding to the target material to the target media content in response to selecting a target material from the set of candidate materials. As an example, the set of candidate materials may include historical materials (i.e., previously generated foreground elements). The set of candidate materials may also include other image content (e.g., stickers, etc.).

[0076] In some embodiments, a set of candidate materials is associated with a material library. The electronic device 110 may, in response to a preset operation on target media content, add at least one foreground element from at least one foreground element in the target media content to the material library.

[0077] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may provide a text control 240-4. Further, the electronic device 110 may obtain text content input by the user based on the text control 240-4 to present the input text content in the target media content 210.

[0078] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may provide drawing controls 240-5. Further, the electronic device 110 may acquire lines or image content drawn by the user based on the drawing controls 240-5, so as to present the drawn lines or image content in the target media content 210.

[0079] In some embodiments, continuing to refer to FIG2B, the electronic device 110 may provide a publishing control 245 in the editing interface 200B. Further, the electronic device 110 may, in response to the triggering of the publishing control 245, present the target media content 210 in the work interface of the information flow.

[0080] In some embodiments, as shown in FIG2H, the electronic device 110 can present the target media content 210 in the content interface 200H of the information flow. The electronic device 110 can present the publisher information of the target media content (e.g., avatar or nickname, etc.) in the content interface 200H. The electronic device 110 can present the interactive information of the target media content 210 (e.g., likes, comments, favorites, etc.) in the content interface 200H.

[0081] In some embodiments, continuing to refer to FIG2H, the electronic device 110 may distinguishably display at least one foreground element. As an example, the electronic device 110 may highlight at least one foreground element included in the target media content 210 based on the presentation of outline elements indicating the boundaries of at least one foreground element.

[0082] Based on the process described above, embodiments of this disclosure can determine reference media content from a user's media library, determine a background image based on the reference media content, and segment at least one foreground element to generate target media content. Furthermore, embodiments of this disclosure can also adjust the presentation style of the reference media content, at least one foreground element, and / or at least one foreground element based on user actions.

[0083] In this way, the embodiments of this disclosure can generate target media content including multiple hierarchical relationships for users, and provide users with a simple and quick editing method, thereby meeting users' needs for generating media content and improving the quality and efficiency of generating media content.

[0084] Example process

[0085] Figure 3 shows a flowchart of an example process 300 for generating media content according to some embodiments of the present disclosure. Process 300 can be implemented at electronic device 110. Process 300 will now be described with reference to Figure 1.

[0086] As shown in the figure, in box 310, electronic device 110 determines a set of reference media content for media content generation, the set of reference media content being determined from a media library associated with the user.

[0087] In box 320, electronic device 110 presents target media content generated based on a set of reference media content, the target media content including at least one foreground element overlaid on a background image, wherein the foreground element includes an image portion determined by segmenting corresponding reference media content in the set of reference media content.

[0088] In some embodiments, determining a set of reference media content for media content generation includes: presenting a media selection interface that displays multiple media contents in a media library; and receiving a selection of a set of reference media content from the multiple media contents via the media selection interface to determine a set of reference media content.

[0089] In some embodiments, determining a set of reference media content for media content generation includes: receiving a generation request for generating media content; and in response to the generation request, determining the set of reference media content associated with a preset theme from a media library.

[0090] In some embodiments, the image portion includes: a first image portion determined from reference media content based on a first editing operation by a user, the first editing operation indicating a first region to be segmented; or a second image portion determined by segmenting a second region in the reference media content corresponding to a preset object.

[0091] In some embodiments, the background image is determined based on the following process: determining first reference media content that meets preset conditions from a set of reference media content; and determining a background image based on the first reference media content, the background image including at least a portion of the first reference media content.

[0092] In some embodiments, the background image is determined based on the following process: generating a background image based on a set of reference media content.

[0093] In some embodiments, target media content is generated based on the following process: determining style information corresponding to at least one foreground element, the style information indicating at least one display attribute of at least one foreground element; and based on the style information, overlaying at least one foreground element onto a background image to generate target media content.

[0094] In some embodiments, at least one display attribute includes at least one of the following: the display position of the foreground element, the display size of the foreground element, the display direction of the foreground element, and the display level of the foreground element.

[0095] In some embodiments, determining style information corresponding to at least one foreground element includes: determining a first template to be applied, wherein the first template includes: a preset template, a template selected by the user, or a template determined based on the at least one foreground element; and determining style information corresponding to at least one foreground element based on the first template.

[0096] In some embodiments, at least one display attribute of at least one foreground element satisfies a preset display constraint, the display constraint indicating at least the occlusion relationship between elements.

[0097] In some embodiments, process 300 further includes: in response to a first operation on a first foreground element among at least one foreground element, presenting a viewing interface of first reference media content corresponding to the first foreground element; and in the viewing interface, distinguishably displaying an image portion corresponding to the first foreground element.

[0098] In some embodiments, process 300 further includes: in response to receiving a second editing operation in the viewing interface, adjusting an image portion based on the second editing operation to update a first foreground element.

[0099] In some embodiments, adjusting an image portion based on a second editing operation includes: adjusting the boundaries of the image portion; and / or adding at least one additional visual element to the image portion.

[0100] In some embodiments, process 300 further includes: adjusting the display hierarchy of the second foreground element in the target media content based on a second operation performed on the second foreground element among at least one foreground element.

[0101] In some embodiments, process 300 further includes: presenting an editing interface for the target media content; presenting a set of candidate materials in response to triggering an add entry in the editing interface; and adding an additional foreground element corresponding to the target material to the target media content in response to selecting a target material from the set of candidate materials.

[0102] In some embodiments, a set of candidate materials is associated with a material library, and process 300 further includes: in response to a preset operation on the target media content, adding at least one foreground element from at least one foreground element in the target media content to the material library.

[0103] In some embodiments, process 300 further includes: presenting a position adjustment control associated with the target media content; and adjusting the display position of at least one foreground element in the target media content in response to triggering the position control.

[0104] In some embodiments, process 300 further includes: presenting a template control associated with the target media content; presenting a set of candidate templates in response to triggering the template control; and presenting updated media content generated based on the second template and a set of reference media content in response to selecting a second template from the set of candidate templates.

[0105] In some embodiments, at least one foreground element includes a third foreground element and a fourth foreground element, the third foreground element and the fourth foreground element corresponding to different image portions of the same reference media content.

[0106] In some embodiments, process 300 further includes: presenting a set of thumbnails corresponding to a set of reference media content; and in response to the selection of a target thumbnail in the set of thumbnails, distinguishably displaying at least one foreground element corresponding to the target thumbnail in the target media content.

[0107] In some embodiments, distinguishing at least one foreground element corresponding to the target thumbnail in the target media content includes: presenting an outline element in the target media content to indicate the boundary of the at least one foreground element.

[0108] Example devices and equipment

[0109] Embodiments of this disclosure also provide corresponding apparatus for implementing the methods or processes described above. Figure 4 shows a schematic structural block diagram of an example apparatus 400 for generating media content according to certain embodiments of this disclosure. Apparatus 400 may be implemented as or included in an electronic device. The various modules / components in apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0110] As shown in Figure 4, the device 400 includes a determination module 410 configured to determine a set of reference media content for media content generation, the set of reference media content being determined from a media library associated with a user; and a presentation module 420 configured to present target media content generated based on the set of reference media content, the target media content including at least one foreground element overlaid on a background image, wherein the foreground element includes an image portion determined by segmenting a corresponding reference media content in the set of reference media content.

[0111] In some embodiments, the determining module 410 is further configured to: present a media selection interface that displays multiple media contents in a media library; and receive a selection of a set of reference media contents from the multiple media contents via the media selection interface to determine a set of reference media contents.

[0112] In some embodiments, the determining module 410 is further configured to: receive a generation request for generating media content; and, in response to the generation request, determine the set of reference media content associated with a preset theme from a media library.

[0113] In some embodiments, the image portion includes: a first image portion determined from reference media content based on a first editing operation by a user, the first editing operation indicating a first region to be segmented; or a second image portion determined by segmenting a second region in the reference media content corresponding to a preset object.

[0114] In some embodiments, the background image is determined based on the following process: determining first reference media content that meets preset conditions from a set of reference media content; and determining a background image based on the first reference media content, the background image including at least a portion of the first reference media content.

[0115] In some embodiments, the background image is determined based on the following process: generating a background image based on a set of reference media content.

[0116] In some embodiments, target media content is generated based on the following process: determining style information corresponding to at least one foreground element, the style information indicating at least one display attribute of at least one foreground element; and based on the style information, overlaying at least one foreground element onto a background image to generate target media content.

[0117] In some embodiments, at least one display attribute includes at least one of the following: the display position of the foreground element, the display size of the foreground element, the display direction of the foreground element, and the display level of the foreground element.

[0118] In some embodiments, determining style information corresponding to at least one foreground element includes: determining a first template to be applied, wherein the first template includes: a preset template, a template selected by the user, or a template determined based on the at least one foreground element; and determining style information corresponding to at least one foreground element based on the first template.

[0119] In some embodiments, at least one display attribute of at least one foreground element satisfies a preset display constraint, the display constraint indicating at least the occlusion relationship between elements.

[0120] In some embodiments, the apparatus 400 further includes a viewing module configured to: in response to a first operation on a first foreground element among at least one foreground element, present a viewing interface of a first reference media content corresponding to the first foreground element; and in the viewing interface, distinguishably display an image portion corresponding to the first foreground element.

[0121] In some embodiments, the device 400 further includes an editing module configured to: in response to receiving a second editing operation in the viewing interface, adjust an image portion based on the second editing operation to update a first foreground element.

[0122] In some embodiments, the editing module is further configured to: adjust the boundaries of the image portion; and / or add at least one additional visual element to the image portion.

[0123] In some embodiments, the apparatus 400 further includes a hierarchy module configured to adjust the display hierarchy of a second foreground element in target media content based on a second operation performed on a second foreground element among at least one foreground element.

[0124] In some embodiments, the device 400 further includes a material module, which is configured to: present an editing interface for target media content; present a set of candidate materials in response to triggering an add entry in the editing interface; and add additional foreground elements corresponding to the target material to the target media content in response to selecting a target material from the set of candidate materials.

[0125] In some embodiments, a set of candidate materials is associated with a material library, and the material module is further configured to add at least one foreground element from at least one foreground element in the target media content to the material library in response to a preset operation on the target media content.

[0126] In some embodiments, the device 400 further includes a positioning module configured to: present a positioning adjustment control associated with target media content; and adjust the display position of at least one foreground element in the target media content in response to triggering the positioning control.

[0127] In some embodiments, the apparatus 400 further includes a template module configured to: present a template control associated with target media content; present a set of candidate templates in response to triggering the template control; and present updated media content generated based on the second template and a set of reference media content in response to selecting a second template from the set of candidate templates.

[0128] In some embodiments, at least one foreground element includes a third foreground element and a fourth foreground element, the third foreground element and the fourth foreground element corresponding to different image portions of the same reference media content.

[0129] In some embodiments, the apparatus 400 further includes a selection module configured to: present a set of thumbnails corresponding to a set of reference media content; and, in response to the selection of a target thumbnail in the set of thumbnails, distinguishably display at least one foreground element corresponding to the target thumbnail in the target media content.

[0130] In some embodiments, the selection module is further configured to: present, in the target media content, an outline element for indicating the boundary of at least one foreground element.

[0131] Figure 5 illustrates a block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 500 shown in Figure 5 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 500 shown in Figure 5 can be used in electronic devices.

[0132] As shown in Figure 5, the electronic device 500 is in the form of a general-purpose electronic device. Components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage devices 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in the memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0133] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0134] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0135] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0136] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0137] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0138] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0139] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0140] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0142] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating media content, comprising: A set of reference media content is determined for media content generation, the set of reference media content being determined from a media library associated with the user; as well as The target media content is generated based on the set of reference media content, the target media content including at least one foreground element superimposed on a background image, wherein the foreground element includes an image portion determined by segmenting the corresponding reference media content in the set of reference media content.

2. The method of claim 1, wherein determining a set of reference media content for media content generation includes: A media selection interface is presented, which displays multiple media contents in the media library; as well as The selection of a set of reference media content from the multiple media content is received via the media selection interface to determine the set of reference media content.

3. The method of claim 1, wherein determining a set of reference media content for media content generation includes: Receive generation requests for generating media content; as well as In response to the generation request, the set of reference media content associated with the preset theme is determined from the media library.

4. The method according to claim 1, wherein the image portion comprises: Based on the user's first editing operation, a first image portion is determined from the reference media content, the first editing operation indicating a first region to be segmented; or The second image portion is determined by segmenting a second region in the reference media content that corresponds to a preset object.

5. The method of claim 1, wherein the background image is determined based on the following process: Determine a first set of reference media content that meets preset conditions from the set of reference media content; and The background image is determined based on the first reference media content, and the background image includes at least a portion of the first reference media content.

6. The method of claim 1, wherein the background image is determined based on the following process: The background image is generated based on the set of reference media content.

7. The method of claim 1, wherein the target media content is generated based on the following process: Determine style information corresponding to the at least one foreground element, the style information indicating at least one display attribute of the at least one foreground element; and Based on the style information, at least one foreground element is overlaid onto the background image to generate the target media content.

8. The method of claim 7, wherein the at least one display attribute comprises at least one of the following: The display position of the foreground element. The display size of the foreground element. Display direction of foreground elements The display hierarchy of foreground elements.

9. The method of claim 7, wherein determining the style information corresponding to the at least one foreground element comprises: A first template to be applied is determined, wherein the first template includes: a preset template, a template selected by the user, or a template determined based on the at least one foreground element; and Based on the first template, the style information corresponding to the at least one foreground element is determined.

10. The method of claim 7, wherein at least one display attribute of the at least one foreground element satisfies a preset display constraint, the display constraint indicating at least the occlusion relationship between the elements.

11. The method according to claim 1, further comprising: In response to a first operation on a first foreground element among the at least one foreground element, a viewing interface for the first reference media content corresponding to the first foreground element is presented; as well as In the viewing interface, the image portion corresponding to the first foreground element is displayed separately.

12. The method of claim 11, further comprising: In response to receiving a second editing operation in the viewing interface, the image portion is adjusted based on the second editing operation to update the first foreground element.

13. The method of claim 12, wherein adjusting the image portion based on the second editing operation comprises: Adjust the boundaries of the image portion; and / or Add at least one additional visual element to the image portion.

14. The method according to claim 1, further comprising: Based on a second operation performed on the second foreground element among the at least one foreground element, the display level of the second foreground element in the target media content is adjusted.

15. The method according to claim 1, further comprising: An editing interface is presented to display the target media content; In response to the triggering of the add entry in the editing interface, a set of candidate materials is presented; as well as In response to the selection of a target material from the set of candidate materials, additional foreground elements corresponding to the target material are added to the target media content.

16. The method of claim 15, wherein the set of candidate materials is associated with a material library, and the method further comprises: In response to a preset operation on the target media content, at least one foreground element from at least one foreground element in the target media content is added to the media library.

17. The method according to claim 1, further comprising: Presents a position adjustment control associated with the target media content; as well as In response to the triggering of the position control, the display position of the at least one foreground element in the target media content is adjusted.

18. The method according to claim 1, further comprising: Present template controls associated with the target media content; In response to the triggering of the template control, a set of candidate templates is presented; as well as In response to the selection of the second template from the set of candidate templates, updated media content generated based on the second template and the set of reference media content is presented.

19. The method of claim 1, wherein the at least one foreground element comprises a third foreground element and a fourth foreground element, the third foreground element and the fourth foreground element corresponding to different image portions of the same reference media content.

20. The method according to claim 1, further comprising: Present a set of thumbnails corresponding to the aforementioned set of reference media content; as well as In response to the selection of a target thumbnail from the set of thumbnails, at least one foreground element corresponding to the target thumbnail is displayed distinctly in the target media content.

21. The method of claim 20, wherein distinguishing at least one foreground element corresponding to the target thumbnail in the target media content comprises: In the target media content, outline elements are presented to indicate the boundaries of the at least one foreground element.

22. An apparatus for generating media content, comprising: The determination module is configured to determine a set of reference media content for media content generation, said set of reference media content being determined from a media library associated with the user; as well as The presentation module is configured to present target media content generated based on the set of reference media content, the target media content including at least one foreground element overlaid on a background image, wherein the foreground element includes an image portion determined by segmenting a corresponding reference media content in the set of reference media content.

23. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 21.

24. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 21.

Images