Method and apparatus for generating media content, and device and storage medium

Abstract

According to the embodiments of the present disclosure, provided are a method and apparatus for generating media content, and a device and a storage medium. The method comprises: receiving a selection for a target template, wherein the target template is associated with reference text; on the basis of the reference text, generating target text regarding a target object, wherein the target text has style information and / or a content structure matching the reference text; and generating a media editing result on the basis of the target text, wherein the media editing result comprises a group of media clips determined on the basis of the target text and from at least one media material associated with the target object. The embodiments of the present disclosure can automatically process a material with reference to text content of a template, and generate a media editing result. Thus, the embodiments of the present disclosure can improve the quality of generated media content.

Description

Method, device, equipment and storage medium for generating media content

[0001] This application claims priority to the Chinese invention patent application entitled “Method, apparatus, device and storage medium for generating media content” filed on March 28, 2024, with application number 202410370680.X. The entire contents of that application are incorporated by reference into this application. Technical Field

[0002] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to providing methods, devices, apparatuses, and computer-readable storage media for generating media content. Background Art

[0003] With the rapid development of intelligent technology, various electronic devices are significantly enriching people's daily lives. These devices can generate media content tailored to user needs based on a variety of media assets. For example, users can upload videos to an electronic device and automatically synthesize a new video containing clips from those videos. Summary of the Invention

[0004] In a first aspect of the present disclosure, a method for generating media content is provided. The method includes: receiving a selection of a target template, the target template being associated with a reference text; generating a target text about a target object based on the reference text, the target text having style information and / or content structure matching the reference text; and generating a media editing result based on the target text, the media editing result including a set of media segments determined from at least one media material associated with the target object based on the target text.

[0005] In a second aspect of the present disclosure, a device for generating media content is provided. The device includes: a first receiving module configured to receive a selection of a target template, the target template being associated with a reference text; a first generating module configured to generate a target text about a target object based on the reference text, the target text having style information and / or content structure matching the reference text; and a second generating module configured to generate a media editing result based on the target text, the media editing result including a set of media segments determined based on the target text from at least one media material associated with the target object.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0008] In a fifth aspect of the present disclosure, a computer program product is provided, which includes computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method of the first aspect is implemented.

[0009] It should be understood that the content described in this summary section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG2 shows a schematic diagram of an example interface according to some embodiments of the present disclosure;

[0013] 3A to 3D illustrate schematic diagrams of example interfaces according to some embodiments of the present disclosure;

[0014] 4A to 4K are schematic diagrams illustrating example interfaces according to some embodiments of the present disclosure;

[0015] FIG5 shows a schematic diagram of an example interface according to some embodiments of the present disclosure;

[0016] FIG6 shows a flowchart of a process for generating media content according to some embodiments of the present disclosure;

[0017] FIG7 shows a schematic structural block diagram of an apparatus for generating media content according to certain embodiments of the present disclosure;

[0018] FIG8 shows a block diagram of an electronic device capable of implementing various embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0020] It should be noted that the titles of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and any type of embodiment may be included under any section / subsection. Furthermore, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or in different sections / subsections.

[0021] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0022] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects shall comply with the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user is aware of and confirms them. Accordingly, when implementing the various embodiments of the present disclosure, the types, scope of use, and usage scenarios of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method may vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0023] If this specification and the solutions in the examples involve the processing of personal information, such processing will be done only with a legitimate basis (such as with the consent of the subject of personal information or as necessary for the performance of a contract) and only within the prescribed or agreed scope. A user's refusal to process personal information other than that required for basic functions will not affect the user's use of basic functions.

[0024] Embodiments of the present disclosure provide a method for generating media content. According to the method, a selection of a target template can be received, where the target template is associated with a reference text. Furthermore, a target text can be generated for a target object based on the reference text, where the target text has style information and / or content structure that matches the reference text. Accordingly, a media editing result can be generated based on the target text, where the media editing result includes a set of media segments determined based on the target text from at least one media material associated with the target object.

[0025] Based on this approach, the embodiments of the present disclosure can automatically process the material by referring to the text content of the template and generate a media editing result. Thus, the embodiments of the present disclosure can improve the quality of the generated media content.

[0026] Sample Environment

[0027] FIG1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in FIG1 , the example environment 100 may include an electronic device 110 .

[0028] In this example environment 100, electronic device 110 may run an application 120 that supports interface interaction. Application 120 may be any appropriate type of application for interface interaction, examples of which may include, but are not limited to, media editing applications and video applications. User 140 may select a target template based on application 120. The electronic device may generate media content associated with the target template based on the target template. The target template may be any appropriate type of media content, such as images, videos, or voice, and the media content generated in association with the target template may also be any appropriate type of media content, such as images, videos, or voice.

[0029] A user 140 may interact with the application 120 via the electronic device 110 and / or its attached devices.

[0030] In the environment 100 of FIG. 1 , if the application 120 is in an active state, the electronic device 110 may present an interface 150 for supporting interface interaction through the application 120 .

[0031] In some embodiments, the electronic device 110 communicates with the server 130 to enable the provision of services for the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a handheld computer, a portable game terminal, a VR / AR device, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface for the target user (such as a "wearable" circuit, etc.).

[0032] The server 130 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. For example, the server 130 may include a computing system / server such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. The server 130 may provide background services for the application 120 that supports interface interaction in the electronic device 110.

[0033] A communication connection may be established between the server 130 and the electronic device 110. The communication connection may be established in a wired or wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the server 130 and the electronic device 110 may implement signaling interaction through the communication connection between the two.

[0034] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0035] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0036] Example Interaction

[0037] The following describes an example interaction process according to an embodiment of the present disclosure with reference to Figures 2 to 5. Figures 2 to 5 show example interfaces 200 to 500 according to some embodiments of the present disclosure, which may be provided by the electronic device 110 shown in Figure 1, for example.

[0038] The following specific embodiments will use the target template and the media content generated based on the target template as an example of a video to describe the example process of the present disclosure. It should be noted that the target template and the media content generated based on the target template can also be any appropriate type of media content such as images or voice, which will not be described in detail here.

[0039] 2 , electronic device 110 may present interface 200, which may display at least one template, such as template 210 and template 220. In some embodiments, template 210 and template 220 may include media content published by the platform, or media content created and published by other users.

[0040] In some embodiments, the template may be associated with reference text (also referred to as reference copy). The reference text may include, for example, text corresponding to subtitles used in the media content, or text corresponding to spoken content in the video content.

[0041] In some embodiments, the electronic device 110 may also provide other types of templates, such templates may be related to corresponding reference texts, and such templates may not be associated with a specific work. For example, the electronic device 110 may provide the user with a selection entry corresponding to a group of selectable templates to receive the user's selection of a specific template.

[0042] As shown in Figure 2 , interface 200 may also display selection controls corresponding to multiple template types. For example, Figure 2 shows selection control 230 for clothing matching templates, selection control 240 for food and beverage templates, and so on. When a user clicks the selection control corresponding to each template type, interface 200 may display all templates corresponding to that template type.

[0043] Of course, a selection control 250 corresponding to the template type may also be displayed on the interface 200. After the user clicks on the selection control 250, the interface 200 may display all templates of all template types that can be selected.

[0044] In some embodiments, after receiving the selection of template 210 or template 220, electronic device 110 may present a playback interface for the media file corresponding to template 210 or template 220, and may provide a usage portal corresponding to template 210 or template 220. In other embodiments, after receiving the selection of template 210 or template 220, electronic device 110 may also directly present an interface for creating media content based on template 210 or template 220.

[0045] In some embodiments, the electronic device 110 may, for example, present a configuration interface 300A as shown in FIG3A based on a request to use a target template. Configuration controls for multiple types of information may be displayed in the interface 300A to support the user in configuring parameter information for the media content to be generated. In some embodiments, the parameter information may include, but is not limited to, the target duration of the media content, the size of the video, descriptive information about the target object, and the like.

[0046] Taking FIG3A as an example, interface 300A may include a configuration control 301 corresponding to the size of the media content, a configuration control 302 corresponding to the duration of the media content, and so on. The user may select a target size for the media content to be generated based on the configuration control 301 corresponding to the size of the media content. The user may also select a target duration for the media content to be generated based on the configuration control 302 corresponding to the duration of the media content, so that the generated media edited content has the specified target duration.

[0047] As shown in FIG3A , interface 300A may also include input controls 303 and 304 for object description information, where the description information may include the object's name, object description content, and the like. In some embodiments, such objects may include, for example, objects associated with the media editing result to be generated, such as a product object. In some embodiments, when the media editing result is promotional content for a product object, such description content may include, for example, information such as the product object's selling points.

[0048] In some embodiments, the electronic device 110 may also obtain at least one media material. In some embodiments, the electronic device 110 may obtain at least one media material selected by the user. Alternatively, such media material may be automatically generated using generation technology.

[0049] Figure 3B illustrates an interface 300B according to some embodiments of the present disclosure. As shown in Figure 3B , interface 300B may, for example, display multiple candidate media materials. These multiple candidate media materials may include local media materials stored on electronic device 110 and / or media materials stored in the cloud. As shown in Figure B , interface 300B may include a selection control 310 corresponding to the media material. A user may click on media material selection control 310 to select at least one media material.

[0050] Taking Figure 3B as an example, interface 300B may display a selection control 330 corresponding to media material 1, a selection control 340 corresponding to media material 2, a selection control 350 corresponding to media material 3, and so on. The user can click the selection control corresponding to the media material to be selected to select the media material to be used. In some embodiments, such media materials may include but are not limited to picture materials, video materials, audio materials, etc.

[0051] In some examples, the description information of the target object can be obtained by the electronic device via the configuration interface 300B, or can be obtained by the electronic device via the interface 300B. Taking Figure 3B as an example, the interface 300B includes an input control 320 for the description information of the object. The user can also click the input control 320 corresponding to the description information of the object to enter the description information of the target object.

[0052] After the user clicks the input control 320 corresponding to the object's description information in interface 300B, the electronic device may display interface 300C. Interface 300C may display a soft keyboard 360 to support input of the target object's description information into an input box 370 based on the soft keyboard 360. After the user enters the target object's description information into the input box 370, they may click the corresponding control 380 in interface 300C to allow the electronic device to receive the target object's description information entered by the user.

[0053] As an example, if the video to be generated is content related to the target product, the user can set the duration of the video to be generated to duration A based on interface 300A, and the user can click the import control 330 corresponding to media material 1, the import control 340 corresponding to media material 2, and the import control 350 corresponding to media material 3 in interface 300B to import media material 1, media material 2, and media material 3 associated with the target product.

[0054] After receiving this input information, the electronic device may display an interface 300D. Interface 300D may display at least one selected media material and description information of the target object (e.g., "Sunglasses, cool and fashionable"). The at least one media material includes media material 1, media material 2, and media material 3. Interface 300D may also display a control 390 for triggering the generation process. The user may click on the control 390 corresponding to the media content generation to cause the electronic device 110 to generate the target text based on the reference text associated with the template 210 selected by the user, media material 1, media material 2, media material 3, description information of the target object, and duration A.

[0055] In some embodiments, the generated target text may have style information and / or content structure that matches the reference text. Such style information may include the language style of the reference text. Such content structure may indicate the distribution of various parts of the reference text.

[0056] In some embodiments, the electronic device 110 may generate such a target text using a model. Specifically, the electronic device 110 may generate guidance information (also referred to as prompt) to the target model based on the reference text.

[0057] In some embodiments, when the electronic device 110 obtains configuration parameters via the configuration interface 300A, such guidance information may also indicate one or more of the obtained configuration parameters. For example, such guidance information may indicate the name or description of the received target object. Alternatively or additionally, the guidance information may be generated based on at least one media material associated with the target object selected by the user.

[0058] In some embodiments, such guidance information can also instruct the model to generate a target text about the target object based on the language style and / or content structure of the reference text. In some embodiments, the complete content of the reference text can be included in the guidance information to instruct the model to generate the target text based on the language style and / or content structure of the reference text.

[0059] Alternatively, the reference text may be preprocessed to determine the style information and / or structure information of the reference text. Furthermore, the style information and / or structure information of the reference text may be indicated in the guide information instead of including the complete content of the reference text in the guide information.

[0060] Furthermore, the electronic device 110 may obtain a target text generated by the model based on the reference text. In some scenarios, such a target text may also be referred to as a script.

[0061] In some embodiments, the electronic device 110 may also generate a media editing result based on the target text and provide the media editing result to the user. Such a media editing result may include a set of media segments determined from the at least one selected material based on the target text. In some scenarios, such a set of media segments may also be referred to as highlight segments.

[0062] In some embodiments, the cumulative duration of such a set of media segments can meet the duration range configured via interface 300A. In some embodiments, the cumulative duration of such a set of media segments can be determined based on the duration of the spoken content corresponding to the target text, so that the cumulative duration can match the duration of the spoken content.

[0063] The process of determining the group of media segments will be further described below.

[0064] In some embodiments, the electronic device 110 and / or the server 130 may perform storyboarding on the selected at least one media material to obtain a plurality of candidate media segments having associated semantics. Taking video content as an example of a media material, such candidate media segments may include, for example, video segments having associated semantic information.

[0065] Furthermore, the electronic device 110 and / or the server 130 may determine a group of media segments matching the target text from a plurality of candidate media segments based on the generated target text.

[0066] Specifically, the electronic device 110 and / or the server 130 may divide the target text into multiple text parts, and sequentially determine a media segment matching each text part.

[0067] Specifically, electronic device 110 and / or server 130 may determine the media segments that match each text portion based on the semantic similarity between each text portion and each candidate text segment. In some embodiments, such a video segment may include all or part of the content in the candidate media segment.

[0068] In some embodiments, when determining the set of media segments, the electronic device 110 and / or the server 130 may also consider the image quality of each candidate media segment. Specifically, the electronic device 110 and / or the server 130 may comprehensively screen the set of media segments based on semantic similarity and image quality, so that the image quality of the resulting set of media segments exceeds a threshold.

[0069] Furthermore, the electronic device 110 may combine the set of media segments based on the order of the multiple text segments in the target text. For example, the electronic device 110 may combine the corresponding multiple media segments based on the order of the multiple text segments to obtain a media editing result.

[0070] In some embodiments, such media editing results may include directly generated media content, such as video content. Alternatively or additionally, such media editing results may also include, for example, media draft files.

[0071] Figure 4A shows an example editing interface 400A according to some embodiments of the present disclosure. As shown in Figure 4A, the editing interface 400A can present the generated media editing results, such as a media draft file.

[0072] As shown in FIG. 4A , the media editing result may include a plurality of media segments determined from the selected media material based on the target text, for example, a media segment 410 , a media segment 420 , and a media segment 430 .

[0073] In some embodiments, the electronic device 110 may also add other materials based on the multiple media segments to generate a media editing result.

[0074] In some embodiments, the electronic device 110 may, for example, add subtitle content or short title content corresponding to corresponding text portions of the target text on the screens of the plurality of media segments 410 to 430 .

[0075] In some embodiments, the electronic device 110 may further overlay audio content corresponding to the target text on the basis of the plurality of media segments 410 to 430. As an example, such audio content may include spoken content corresponding to the target text.

[0076] In some embodiments, such audio content can be generated based on the audio attributes configured by the user. For example, the user can select a specific sound effect (e.g., timbre), and further, the electronic device 110 can generate the audio content based on the selected sound effect.

[0077] In some embodiments, such audio content (also referred to as first audio content) can also be generated based on the audio properties (e.g., timbre) of audio content associated with the selected target (also referred to as second audio content). As an example, the electronic device 110 can generate the first audio content corresponding to the target text based on timbre cloning of the second audio content. Alternatively, the electronic device 110 can generate the first audio content using the same sound effects as the second audio content.

[0078] In some embodiments, the electronic device 110 may further enable the user to edit the generated first audio content, for example, to modify a specified audio attribute of the first audio content. For example, the electronic device 110 may edit the first audio content based on another sound effect selected by the user, thereby modifying the timbre of the first audio content.

[0079] In some embodiments, the electronic device 110 may also overlay background music on the basis of the plurality of media segments 410 to 430. Such background music may include background music specified by the user, background music recommended based on the target text, or background music used by the target template.

[0080] In some embodiments, the electronic device 110 may also receive a user's editing operation on the target text, for example, to modify the first text into the second text. Further, the electronic device 110 may, based on the process discussed above and based on the modified second text, re-determine a group of media content that matches the second text from at least one media material. Further, the electronic device 110 may update the media editing result based on the group of media content. The process of re-determining the group of media content based on the second text can be referred to the description above and will not be repeated here.

[0081] In some embodiments, the electronic device 110 may further support the user in performing various appropriate types of editing on the media draft in the interface 400A, and generate corresponding media content based on the edited media draft.

[0082] Using Figures 4A and 4B as examples, interfaces 400A and 400B may also display a control 435 for adding media materials, allowing the user to add other media materials besides Media Materials 1, 2, and 3, and thereby generate new media content based on these media materials. The user can click on control 435 for adding media materials, and the electronic device will then display interface 400C. This interface 400C will display all available media materials. For previously added media materials, the electronic device will add corresponding identification information to display interface 400C to indicate that they have been added as media materials. Specifically, Media Materials 1, 2, and 3 will be identified as added media materials. For previously unadded media materials, the electronic device will display corresponding add controls in display interface 400C. Unadded media materials are media materials besides Media Materials 1, 2, and 3. Taking FIG. 4C as an example, the interface 400C may display a selection control 440 corresponding to media material 4, a selection control 450 corresponding to media material 5, a selection control 351 corresponding to media material 8, and the like.

[0083] Taking the example of a user clicking on the import control 351 corresponding to media material 8 in interface 400C, the electronic device may display interface 400D. Interface 400D may highlight the selection of media material 8. For example, in FIG4D , a check mark may appear in the circle corresponding to media material 8 to indicate its selection. The electronic device may also display the selected media material 8 at the display position 481 of the added media material in interface 400D. Other prominent display methods may also be used to indicate the selection of a media material, such as using different display colors to distinguish selected and unselected media materials. After the user selects to add media material 8, the electronic device may display interface 400E. Interface 400E may display a target prompt, prompting the user to confirm whether to generate new media content based on the newly added media material. Interface 400E may also display a confirmation control 482. The user may click control 482 to confirm the generation of new media content based on the newly added media material 8. After the user clicks the control 482 , the electronic device may display an interface 400F, which may display prompt information about the new media content being generated.

[0084] In some embodiments, at least one editing operation in the editing page may also be a first operation for a first segment in a group of media segments, the first operation indicating a selection of a first media material to replace the first segment with a second segment in the first media material, or / and a second operation for a third segment in a group of media segments, the second operation indicating a selection of a fourth segment in a target media material corresponding to the third segment to replace the third segment with the fourth segment.

[0085] As shown in interface 400G of FIG4G , interface 400G may present a replace control 491 and a crop control 492. Replace control 491 is used to allow the user to replace media material 2 with another media material. Crop control 492 is used to allow the user to convert sub-media material 420 in media material 2 into another sub-media material in media material 2.

[0086] After the user clicks the replace control 491, the electronic device can display an interface 400H. This interface 400H can display all media materials, and for previously selected media materials, the electronic device can add a corresponding identifier in the display interface 400H to indicate that this media material has been added. For example, the preset position of media material 1, media material 2, and media material 3 can display an "Added" field. For media materials that were not previously selected, the electronic device can display their corresponding selection controls in the display interface. As shown in Figure 4H, there is an import control 440 corresponding to media material 4, an import control 450 corresponding to media material 5, an import control 351 corresponding to media material 8, and an import control 352 corresponding to media material 10.

[0087] For example, when a user clicks on the import control 352 corresponding to the media material 10 in interface 400H, the electronic device may display interface 400I. Interface 400I may highlight the selection of the media material 10. For example, in FIG4I , a check mark may be displayed in the circle corresponding to the media material 10 in interface 400I to indicate that it has been selected.

[0088] After the user selects media material 10, the electronic device may display interface 400J. Interface 400J may display a prompt indicating that new media content is being synthesized based on media material 1, media material 10, and media material 3. After the new media content is synthesized, the electronic device may display interface 400K. This interface 400K may display new media content 490 generated based on media material 1, media material 10, and media material 3. Interface 400K may also display sub-media materials corresponding to media material 1, media material 10, and media material 3, and the sub-media material corresponding to the media material being played may be highlighted on the interface. As an example, the electronic device may highlight the sub-media content being played on the playback interface in a preset color. As another example, the electronic device may highlight the sub-media content being played at a preset position on the playback interface, where the preset position may be the center of the playback interface.

[0089] Returning to Figure 4G, if the user clicks on the cropping control 492 in interface 400G when playing the sub-media material corresponding to a certain media material, the electronic device can display interface 500. Interface 500 can display all the sub-candidate materials 510 contained in this media material, such as all the images contained in a video. Interface 500 can also display a selection box 520 of the selected sub-media material in this media material. The user can slide the selection box 520 to crop the sub-media material, wherein the sub-media material located in the selection box 520 is the sub-media material that needs to be displayed in the synthesized media content. For example, the second to fifteenth frames of images in media material 2 displayed in the original media content can be replaced by the fourth to seventeenth frames of images in media material 2 by sliding the selection box 520. The length corresponding to the second to fifteenth frames of images can be the same as the length corresponding to the fourth to seventeenth frames of images.

[0090] In some embodiments, the user can also edit the first audio content in the media editing results. The electronic device can modify the target audio properties of the first audio content based on the editing operation on the first audio content. Taking Figure 4G as an example, interface 400G can present editing controls for the first audio content, such as volume control 493 and adjustment control 494. Sound control 493 is used to support users to modify the timbre of the first audio content. Adjustment control 493 is used to support users to modify the background music in the first audio content.

[0091] In some embodiments, the electronic device can also publish the media work corresponding to the media editing results based on a preset operation on the media editing results. As shown in Figure 5, the interface 500 may include a corresponding export control to support the publication of the media work. The user can publish the media work by clicking the corresponding export control.

[0092] Based on the process described above, the embodiment of the present disclosure can automatically process the material with reference to the text content of the template and generate a media editing result. Thus, the embodiment of the present disclosure can improve the quality of the generated media content.

[0093] Example Process

[0094] FIG6 shows a flow chart of a process 600 for generating media content according to some embodiments of the present disclosure. The process 600 may be implemented at the electronic device 110. The process 600 is described below with reference to FIG1.

[0095] At block 610 , the electronic device 110 receives a selection of a target template, the target template being associated with a reference text.

[0096] In block 620 , the electronic device 110 generates a target text about the target object based on the reference text, where the target text has style information and / or content structure matching the reference text.

[0097] In block 630 , the electronic device 110 generates a media editing result based on the target text, where the media editing result includes a set of media segments determined from at least one media material associated with the target object based on the target text.

[0098] In some embodiments, a group of media segments is determined based on the following process: determining a plurality of candidate media segments based on storyboarding of at least one media material; and determining a group of media segments from the plurality of candidate media segments based on a degree of matching between the plurality of candidate media segments and a reference text.

[0099] In some embodiments, determining a group of media segments from the plurality of candidate media segments based on the matching levels of the plurality of candidate media segments with the reference text includes:

[0100] Based on the matching degree and the picture quality of the plurality of candidate media segments, a group of media segments is determined from the plurality of candidate media segments, wherein the picture quality of the group of media segments is higher than a threshold.

[0101] In some embodiments, a set of media segments corresponds to a set of text portions of a reference text, and generating a media editing result based on the target text includes: combining a set of media segments based on an order of the set of text portions in the reference text to generate the media editing result.

[0102] In some embodiments, generating target text about the target object based on the reference text includes: acquiring configuration information via a configuration interface, the configuration information including descriptive information about the target object; and generating the target text based on the reference text and the configuration information.

[0103] In some embodiments, the configuration information includes parameter information about the media content to be generated.

[0104] In some embodiments, the parameter information includes a target duration for the media content, and the generated target text matches the target duration.

[0105] In some embodiments, generating target text about the target object based on the reference text includes: receiving a selection of at least one media material associated with the target object; and generating the target text based on the reference text and the at least one media material.

[0106] In some embodiments, generating target text about the target object based on the reference text includes: generating guidance information to the target model based on the reference text and at least one media material; and providing the guidance information to the target model to obtain the target text generated by the target model based on the reference text.

[0107] In some embodiments, the target template includes reference media content, and receiving a selection of the target template includes: displaying a viewing interface for the reference media content; and

[0108] A selection of a create entry in the viewing interface is received.

[0109] In some embodiments, the group of media segments is a first group of media segments, the target text is a first text, and process 600 further includes: receiving an editing operation on the first text; determining a second group of media segments from at least one media material based on the edited second text; and updating the media editing results based on the second group of media segments.

[0110] In some embodiments, the media editing result includes subtitle content corresponding to the target text and / or first audio content corresponding to the target text.

[0111] In some embodiments, the first audio content is generated based on a first audio attribute of the second audio content of the target template and / or a second audio attribute configured by the user.

[0112] In some embodiments, process 600 further includes modifying a target audio property of the first audio content based on the editing operation on the first audio content.

[0113] In some embodiments, the media editing result includes a media draft, and process 600 further includes: presenting the media draft in an editing interface; and updating the media draft based on at least one editing operation in the editing interface.

[0114] In some embodiments, at least one editing operation includes: a first operation on a first segment in a group of media segments, the first operation indicating a selection of a first media material to replace the first segment with a second segment in the first media material; or a second operation on a third segment in a group of media segments, the second operation indicating a selection of a fourth segment in a target media material corresponding to the third segment to replace the third segment with the fourth segment.

[0115] In some embodiments, process 600 further includes: based on a preset operation on the media editing result, publishing the media work corresponding to the media editing result.

[0116] Example devices and equipment

[0117] Embodiments of the present disclosure also provide corresponding apparatuses for implementing the above-described methods or processes. FIG7 shows a schematic structural block diagram of an apparatus 700 for providing media content according to certain embodiments of the present disclosure. Apparatus 700 may be implemented as or included in electronic device 110 as discussed above. The various modules / components in apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0118] As shown in Figure 7, the device 700 includes a first receiving module 710, configured to receive a selection of a target template, the target template being associated with a reference text; a first generating module 720, configured to generate a target text about a target object based on the reference text, the target text having style information and / or content structure matching the reference text; and a second generating module 730, configured to generate a media editing result based on the target text, the media editing result including a set of media segments determined from at least one media material associated with the target object based on the target text.

[0119] In some embodiments, a group of media segments is determined based on the following process: determining a plurality of candidate media segments based on storyboarding of at least one media material; and determining a group of media segments from the plurality of candidate media segments based on a degree of matching between the plurality of candidate media segments and a reference text.

[0120] In some embodiments, determining a group of media segments from the plurality of candidate media segments based on the matching levels of the plurality of candidate media segments with the reference text includes:

[0121] Based on the matching degree and the picture quality of the plurality of candidate media segments, a group of media segments is determined from the plurality of candidate media segments, wherein the picture quality of the group of media segments is higher than a threshold.

[0122] In some embodiments, a set of media segments corresponds to a set of text portions of a reference text, and generating a media editing result based on the target text includes: combining a set of media segments based on an order of the set of text portions in the reference text to generate the media editing result.

[0123] In some embodiments, the first generating module 720 is specifically configured to obtain configuration information via a configuration interface, where the configuration information includes description information about the target object; and based on the reference text and the configuration information.

[0124] In some embodiments, the configuration information includes parameter information about the media content to be generated.

[0125] In some embodiments, the parameter information includes a target duration for the media content, and the generated target text matches the target duration.

[0126] In some embodiments, the first generating module 720 is specifically configured to: receive a selection of at least one media material associated with a target object; and generate a target text based on the reference text and the at least one media material.

[0127] In some embodiments, the first generating module is specifically configured to generate guidance information to the target model based on the reference text and at least one media material; and provide the guidance information to the target model to obtain the target text generated by the target model based on the reference text.

[0128] In some embodiments, the target template includes reference media content, and the receiving module 710 is specifically configured to display a viewing interface of the reference media content; and receive a selection of a creation entry in the viewing interface.

[0129] In some embodiments, a group of media segments is a first group of media segments, the target text is a first text, and the device 700 further includes a second receiving module configured to: receive an editing operation on the first text; a determination module configured to: determine a second group of media segments from at least one media material based on the edited second text; and a first updating module configured to: update the media editing result based on the second group of media segments.

[0130] In some embodiments, the media editing result includes subtitle content corresponding to the target text and / or first audio content corresponding to the target text.

[0131] In some embodiments, the first audio content is generated based on a first audio attribute of the second audio content of the target template and / or a second audio attribute configured by the user.

[0132] In some embodiments, the apparatus 700 further includes a modification module configured to modify the target audio attribute of the first audio content based on the editing operation on the first audio content.

[0133] In some embodiments, the media editing result includes a media draft, and the device 700 also includes a presentation module configured to: present the media draft in the editing interface; and a second update module configured to: update the media draft based on at least one editing operation in the editing interface.

[0134] In some embodiments, at least one editing operation includes: a first operation on a first segment in a group of media segments, the first operation indicating a selection of a first media material to replace the first segment with a second segment in the first media material; or a second operation on a third segment in a group of media segments, the second operation indicating a selection of a fourth segment in a target media material corresponding to the third segment to replace the third segment with the fourth segment.

[0135] In some embodiments, the apparatus 700 further includes a publishing module configured to publish a media work corresponding to the media editing result based on a preset operation on the media editing result.

[0136] The units included in the device 700 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 700 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0137] FIG8 shows a block diagram of an electronic device 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 800 shown in FIG8 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 800 shown in FIG8 can be used to implement the electronic device 110 shown in FIG1 .

[0138] As shown in FIG8 , electronic device 800 is a general-purpose electronic device. Components of electronic device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 800.

[0139] The electronic device 800 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and can be accessed within the electronic device 800.

[0140] The electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG8 , a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0141] The communication unit 840 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 800 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 800 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0142] The input device 850 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 860 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 800 may also communicate with one or more external devices (not shown) via the communication unit 840 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 800, or with any device that allows the electronic device 800 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0143] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0144] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0145] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0146] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0147] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0148] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating media content, comprising: receiving a selection of a target template, the target template being associated with a reference text; generating a target text about a target object based on the reference text, wherein the target text has style information and / or content structure matching that of the reference text; as well as A media editing result is generated based on the target text, where the media editing result includes a group of media segments determined from at least one media material associated with the target object based on the target text.

2. The method of claim 1 , wherein the set of media segments is determined based on the following process: Determining a plurality of candidate media segments based on storyboarding of the at least one media material; and The group of media segments is determined from the plurality of candidate media segments based on the matching degrees between the plurality of candidate media segments and the reference text.

3. The method according to claim 2, wherein determining the set of media segments from the plurality of candidate media segments based on the matching degree between the plurality of candidate media segments and the reference text comprises: Based on the matching degree and the picture quality of the plurality of candidate media segments, the group of media segments is determined from the plurality of candidate media segments, wherein the picture quality of the group of media segments is higher than a threshold.

4. The method of claim 1 , wherein the set of media segments corresponds to a set of text portions of the reference text, and generating a media editing result based on the target text comprises: Based on the order of the set of text portions in the reference text, the set of media segments are combined to generate the media editing result.

5. The method according to claim 1, wherein generating a target text about a target object based on the reference text comprises: Acquiring configuration information via a configuration interface, the configuration information including descriptive information about the target object; as well as The target text is generated based on the reference text and the configuration information. The method according to claim 5 , wherein the configuration information further includes parameter information about the media content to be generated. 7 . The method according to claim 6 , wherein the parameter information includes a target duration of the media content, and the generated target text matches the target duration.

8. The method according to claim 5, wherein generating a target text about a target object based on the reference text comprises: receiving a selection of the at least one media material associated with the target object; as well as The target text is generated based on the reference text and the at least one media material.

9. The method according to claim 1, wherein generating a target text about a target object based on the reference text comprises: generating guidance information to a target model based on the reference text; as well as The guide information is provided to the target model to obtain the target text generated by the target model based on the reference text.

10. The method of claim 1 , wherein the target template comprises reference media content, and receiving a selection of the target template comprises: Displaying a viewing interface of the reference media content; as well as The selection to create a portal in the viewing interface is received.

11. The method according to claim 1 , wherein the group of media segments is a first group of media segments, the target text is a first text, and the method further comprises: receiving an editing operation on the first text; determining a second group of media segments from at least one media material based on the edited second text; as well as Based on the second group of media segments, the media editing result is updated. 12 . The method according to claim 1 , wherein the media editing result includes subtitle content corresponding to the target text and / or first audio content corresponding to the target text. 13 . The method according to claim 12 , wherein the first audio content is generated based on a first audio attribute of the second audio content of the target template and / or a second audio attribute configured by a user.

14. The method according to claim 12, further comprising: Based on the editing operation on the first audio content, a target audio attribute of the first audio content is modified.

15. The method according to claim 1, wherein the media editing result comprises a media draft, further comprising: Presenting the media draft in an editing interface; as well as The media draft is updated based on at least one editing operation in the editing interface.

16. The method of claim 15, wherein the at least one editing operation comprises: a first operation on a first segment of the set of media segments, the first operation indicating selection of a first media asset to replace the first segment with a second segment of the first media asset; or A second operation is performed on a third segment in the group of media segments, the second operation indicating selection of a fourth segment in the target media material corresponding to the third segment, so as to replace the third segment with the fourth segment.

17. An apparatus for generating media content, comprising: a first receiving module configured to receive a selection of a target template, wherein the target template is associated with a reference text; A first generating module is configured to generate a target text based on the reference text and at least one media material, wherein the target text has style information and / or content structure matching that of the reference text; as well as The second generating module is configured to generate a media editing result based on the target text, where the media editing result includes a group of media segments determined from the at least one media material based on the target text.

18. An electronic device comprising: at least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 16 when executed by the at least one processing unit.

19. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 16 is implemented.

20. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 16.

Images