Methods, apparatus, devices, media, and program products for generating media content

Abstract

The present invention provides methods, apparatus, devices, media, and programs for the generation of improved media content. [Solution] The method includes: determining at least one keyword corresponding to at least one of several elements related to media content generation based on a user's media content generation request; determining a set of materials from a media content material library that matches at least one keyword; generating descriptive text for the media content to be generated based on the set of materials and at least one keyword corresponding to at least one element; selecting multiple media content materials from the set of materials that match the story description; and generating media content based on the multiple media content materials.

Description

Technical Field

[0001] Exemplary embodiments of the present invention generally relate to the field of computers, and more particularly, to methods, apparatuses, devices, computer-readable storage media, and computer program products for media content generation.

Background Art

[0002] With the development of machine learning technology, it has become possible to use a machine learning model to learn and train a large amount of data, and then use the obtained model to automatically generate various forms of content such as text, images, audio, and media content. Currently, in various types of practical applications of automatically creating images, such as generating media content based on a user's media content material, it is expected to obtain high-quality media content that meets the user's needs.

Summary of the Invention

Means for Solving the Problems

[0003] In a first aspect of the present invention, a method for media content generation is provided. The method includes, in response to receiving a user's media content generation request, determining at least one keyword corresponding to at least one element among a plurality of elements related to media content generation based on the media content generation request; determining a set of materials matching the at least one keyword from a media content material library; generating an explanatory text for the media content to be generated based on the set of materials and the at least one keyword; selecting a plurality of media content materials matching the explanatory text from the set of materials; and generating media content based on the plurality of media content materials.

[0004] In a second aspect of the present invention, an apparatus for generating media content is provided. The apparatus includes: a keyword determination module configured to determine at least one keyword corresponding to at least one of a plurality of elements related to media content generation based on a media content generation request in response to receiving a media content generation request from a user; a material set determination module configured to determine a set of materials from a material library that matches at least one keyword for the media content; a descriptive text generation module for generating descriptive text for the media content to be generated based on the material set and at least one keyword; a media content material selection module configured to select a plurality of media content materials from the material set that match the descriptive text; and a media content generation module for generating media content based on the plurality of media content materials.

[0005] A third aspect of the present invention provides an electronic device comprising at least one processing unit and at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device causes the device to perform the method of the first aspect.

[0006] A fourth embodiment of the present invention provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method of the first embodiment can be realized.

[0007] A fifth embodiment of the present invention provides a computer program product, which is physically stored in a computer storage medium and includes computer executable instructions, which, when executed by a device, cause the device to execute the method of the first embodiment.

[0008] It should be understood that the information described in the summary of the present invention is not intended to limit the main or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will be readily apparent from the following description. [Brief explanation of the drawing]

[0009] Referring to the following detailed description in conjunction with the drawings will further clarify the above-mentioned features and other features, advantages, and aspects of each embodiment of the present invention. In the drawings, the same or similar reference numerals indicate the same or similar elements, where, [Figure 1] A schematic diagram of an exemplary environment in which embodiments of the present invention can be realized is shown. [Figure 2] A flowchart of the process for generating media content according to some embodiments of the present invention is shown. [Figure 3] This shows a flowchart of the semantic analysis process for media content generation requests according to some embodiments of the present invention. [Figure 4] A flowchart of the process for initial screening of media content materials according to some embodiments of the present invention is shown. [Figure 5] This diagram shows an exemplary configuration for performing a secondary material screening on a candidate material set according to some embodiments of the present invention. [Figure 6] A schematic diagram of an exemplary configuration for generating a story description according to some embodiments of the present invention is shown. [Figure 7] A flowchart shows the process for selecting a media content template according to some embodiments of the present invention. [Figure 8] A flowchart of the process for generating media content according to some embodiments of the present invention is shown. [Figure 9] A block diagram of an apparatus for generating media content according to some embodiments of the present invention is shown. [Figure 10]The following are block diagrams of one or more embodiments of the present invention that can be implemented. [Modes for carrying out the invention]

[0010] The embodiments of the present invention will be described in more detail below with reference to the drawings. Although specific embodiments of the present invention are shown in the drawings, the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Rather, these embodiments are provided for a more thorough and complete understanding of the present invention. The drawings and embodiments of the present invention are for illustrative purposes only and should not be used to limit the scope of protection of the present invention.

[0011] In the description of embodiments of the present invention, the term “including” and similar terms are open-ended inclusions meaning “including, but not limited to, ~”. The term “based on” should be understood as “based on at least part of”. The term “one embodiment” or “the embodiment” should be understood as “at least one embodiment”. The term “some embodiments” should be understood as “at least some embodiments”. The following specification may contain other explicit and implicit definitions.

[0012] In this specification, unless expressly otherwise described, performing a step "in response to A" does not mean performing that step immediately after "A," but may include one or more intermediate steps.

[0013] It is understood that data related to this technical solution (including, but not limited to, the data itself, its acquisition, or its use) must comply with applicable laws and related designated requirements.

[0014] Before using the technical solutions disclosed in each embodiment of the present invention, it is necessary to notify relevant users in an appropriate manner in accordance with applicable laws and regulations of the type of information relating to the present invention, its scope of use, and usage scenarios, and to obtain their permission. Relevant users may include any type of rights holder, such as individuals, companies, or groups.

[0015] For example, when responding to an unauthorized request from a user, prompt information is sent to the user in question to explicitly prompt them that the requested operation requires the acquisition and use of their personal information. This allows the user to independently choose, based on the prompt information, whether or not to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present invention.

[0016] As a selective but non-restrictive implementation, a method for sending prompt information to a user in response to receiving an unsolicited request from the user may, for example, utilize a pop-up window, where the prompt information can be displayed in text form. The pop-up window may also include selection controls for the user to choose whether to "agree" or "disagree" to providing personal information to the electronic device.

[0017] The notification and user permission acquisition processes described above are merely general outlines and do not limit the embodiments of the present invention. It is understood that other methods that comply with relevant laws and regulations may also be applied to embodiments of the present invention.

[0018] As used herein, the term "model" can learn the relationship between corresponding inputs and outputs from training data and, after training is completed, generate the corresponding output for a given input. The generation of the model can be based on machine learning techniques. Deep learning is a type of machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. A neural network model is an example of a model based on deep learning. In this specification, "model" may also be referred to as "machine learning model", "learning model", "machine learning network", or "learning network", and these terms are used interchangeably herein.

[0019] A "neural network" is a type of machine learning network based on deep learning. A neural network can process an input to provide a corresponding output and typically includes an input layer, an output layer, and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. Each layer of the neural network is sequentially connected such that the output of the previous layer is provided as the input to the next layer. The input layer receives the input of the neural network, and the output of the output layer functions as the final output of the neural network. Each layer of the neural network includes one or more nodes (also called processing nodes or neurons), and each node processes the input from the previous layer.

[0020] Generally, machine learning is broadly classified into three stages: a training stage, a testing stage, and an application stage (also called an inference stage). In the training stage, a specific model can be trained using a large amount of training data, and the parameter values are continuously and iteratively updated until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, it is considered that the model can learn the relationship between the input and the output (also called the mapping from the input to the output) from the training data. The parameter values of the trained model are determined. In the testing stage, test inputs are applied to the trained model to test whether the model can provide correct outputs, thereby judging the performance of the model. The testing stage may sometimes be integrated into the training stage. In the application or inference stage, the trained model can be used to process actual model inputs based on the parameter values obtained during training and determine the corresponding model outputs.

[0021] FIG. 1 shows a schematic diagram of an exemplary environment 100 in which embodiments of the present invention can be implemented. The exemplary environment 100 may include a terminal device 110. An application 115 for managing a media content material library is installed on the terminal device 110. Media content materials such as images and videos are stored in the media content material library. The media content material library may sometimes be referred to as a visual material library or a gallery. In some embodiments, the application 115 may be an album application for storing and managing photos, videos, etc. taken by the user 140 via the terminal device 110. It should be understood that the application 115 may be any other suitable application capable of storing and managing media content materials.

[0022] In some embodiments, user 140 can interact with application 115 via terminal device 110 and / or a device connected to terminal device 110. In embodiments of the present invention, application 115 may have intelligent dialogue and task processing capabilities. Typically, application 115 can support user 140 inputting requests in natural language, perform tasks based on natural language input comprehension and logical reasoning capabilities, and generate corresponding media content stories such as media content 150. For example, application 115 may support text dialogue services, voice dialogue services, and other modes of content dialogue with user 140.

[0023] In some embodiments, the terminal device 110 communicates with the service-side device 120 to provide services to the application 115. In some embodiments, the service-side device 120 can provide services to the application 115 using a machine learning model 130 (which may include one or more machine learning models, such as machine learning model 130-1, machine learning model 130-2, ..., machine learning model 130-N, where N is a positive integer; for convenience of explanation, one or more machine learning models are collectively referred to as machine learning model 130). Different machine learning models 130 may be configured to implement different functions in different ways to satisfy the provision of services from the service-side device 120 to the application 115. The machine learning model 130 may run locally on the terminal device 110 or the service-side device 130, or it may be deployed on a remote device, in a cloud environment, etc., and is not limited to these locations.

[0024] In some embodiments, the terminal device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, locator devices, television receivers, radio receivers, e-book devices, game devices, or any combination thereof, and may include accessories, peripherals, or any combination thereof for these devices. In some embodiments, the terminal device 110 may also support any type of interface to the user (such as a “wearable” circuit).

[0025] The service-side device 120 may be any type of computing system / server capable of providing computing functions, including but not limited to mainframes, edge computing nodes, and computing devices in a cloud environment. The service-side device 130 may be implemented, for example, based on a cloud environment.

[0026] The structure and function of each element of Environment 100 are described for illustrative purposes only and should not be considered as limiting the scope of the present invention.

[0027] As briefly mentioned above, the automatic video creation function of a media content library (e.g., albums) refers to the automatic analysis, organization, and editing of materials such as photos and videos on a user's terminal device (e.g., a mobile phone) to generate media content (e.g., videos). However, current automatic video creation technology has several shortcomings.

[0028] In the material analysis step, the currently extracted material tags contain only a few words (e.g., blue sky, grass, tent), making it impossible to fully analyze the content submitted by the materials and the relationships between them. This results in a large amount of information being lost, and inaccurate and erroneous image and video extraction. In the story theme recommendation step, the current solution relies on predefined templates for the themes of created videos, which are limited in number and relatively coarse in granularity. Furthermore, there is a lack of flexibility, preventing users from arranging stories to suit their own ideas or personalizing their created videos.

[0029] In view of this, embodiments of the present invention provide an improved solution for media content generation. In this solution, in response to receiving a media content generation request from a user, at least one keyword corresponding to at least one of a plurality of elements related to media content generation is determined based on the media content generation request. A set of materials matching at least one keyword is determined from a media content material library. Descriptive text for the media content to be generated is generated based on the set of materials and at least one keyword. Multiple media content materials matching the descriptive text are selected from the set of materials. Media content is generated accordingly based on the multiple media content materials.

[0030] In this way, by querying the meaning of the user's input request with multiple elements, querying both with the explanatory text, and querying the media content material with the explanatory text, the generated media content can be adapted to the meaning of the user's request and the actual submitted content of the media content material, thus avoiding the loss of information expressed by the media content material. This expands the relationships between media content materials, resulting in high-quality storytelling media content that meets the user's needs and helps improve the user experience.

[0031] It should be understood that the structure and function of each element within environment 100 are described for illustrative purposes only and do not imply any limitation to the scope of the present invention.

[0032] The following description will continue with reference to the attached drawings, illustrating some exemplary embodiments of the present invention.

[0033] Figure 2 shows a flowchart of process 200 for media content generation according to some embodiments of the present invention. For the sake of discussion, these embodiments will be described with reference to environment 100 in Figure 1. In some examples, these embodiments may be implemented on the service-side device 120 in Figure 1. In other examples, these embodiments may be implemented locally on the client, i.e., on the terminal device 110, or may be completed through cooperation between the terminal device 110 and the service-side device 120. The following specific embodiments are implemented on the service-side device 120 as an example.

[0034] As shown in Figure 2, the service-side device 120 can receive media content generation requests from user 140 (210). In some embodiments, such media content generation requests can be received by the service-side device 120 from the terminal device 110. For example, the terminal device 110 can receive a request (query) statement entered by user 140 via application 115. Application 115 can support user 140 entering the request or query statement in natural language. For example, application 115 can support user 140's text input, voice input, and other modes of input. Here, user 140's media content generation request may include information related to the story, such as the story theme and story summary.

[0035] Furthermore, when the service-side device 120 receives a media content generation request from user 140, it determines, based on the media content generation request, at least one keyword corresponding to at least one of several elements related to media content generation (220). In some embodiments, the several elements may include at least two of the following: time elements, place elements, person elements, and event elements. Time elements, place elements, person elements, and / or event elements constitute several elements of a story. By considering these elements, media content can be generated from available materials that meet the user's needs and align with the development of the story. It should be understood that the several elements may further include, but are not limited to, any other appropriate elements related to the story. In some embodiments, each element may correspond to one or more keywords. For example, a media content generation request may provide several keywords related to place, several keywords related to event, etc., which are not repeated here.

[0036] In some embodiments, the service-side device 120 can use one or more machine learning models 130 to analyze the meaning of a media content generation request and determine at least one keyword corresponding to at least one of several elements. Such machine learning models 130 may include, for example, language models (LMs) or other models that can understand natural language and perform semantic-related tasks. In the field of machine learning, a language model may refer to a language model with a large number of parameters and a complex structure.

[0037] In some embodiments, when determining at least one keyword corresponding to at least one of multiple elements, the service-side device 120 can extract at least one keyword corresponding to at least one of multiple elements from a media content generation request. For example, assuming that user 140's media content generation request is "I will travel to a specific place on X year X month X day", the service-side device 120 can use a language model to extract the keyword "X year X month X day" corresponding to the time element and the keyword "specific place" corresponding to the location element from the request sentence.

[0038] Alternatively or additionally, in some embodiments, when determining at least one keyword corresponding to at least one element among a plurality of elements, the service-side device 120 can determine a keyword corresponding to at least one other element among the plurality of elements based on semantic analysis of the media content generation request. For example, the service-side device 120 can use a language model to perform semantic analysis on the media content generation request and determine whether the media content generation request contains keywords corresponding to the plurality of elements. If it is determined that the media content generation request is missing a keyword corresponding to at least one other element among the plurality of elements, the keyword corresponding to the other element can be supplemented by adding content to the media content generation request.

[0039] For example, with respect to the media content generation request mentioned above, "Travel to a specific place on X year X month X day," the service-side device 120 can use a language model to determine that the request sentence is missing keywords corresponding to person elements and event elements, and then add the keywords corresponding to person elements and / or event elements to the request sentence.

[0040] Figure 3 shows a flowchart of a semantic analysis process 300 for a media content generation request according to some embodiments of the present invention. Process 300 is shown as some embodiment of the steps in box 220 of Figure 2 and performs rewriting or information supplementation of an input media content generation request. In process 300, the service-side device 120 can use a language model to extract multiple elements 301 from the media content generation request that correspond to time, place, person, and event (including event type and event description). The information of these elements may be general information, such as time range and location range. In some embodiments, if the media content generation request does not clearly contain information corresponding to one or more elements, the service-side device 120 can also use the model to supplement the information of the missing elements by combining it with contextual information or by using other information.

[0041] In some embodiments, the service-side device 120 can also supplement the input media content generation request by screening relevant media content materials from the media content material library based on multiple elements 301 extracted from the input media content generation request, performing semantic merging based on the screened media content materials, and rewriting the input media content generation request using the merged information. For example, if the input media content generation request is missing event and person elements, the service-side device 120 can use a language model to aggregate media content materials in the media content material library whose location and time elements are similar to those of the input media content generation request (310), thereby obtaining the person and event corresponding to the location and time. The service-side device 120 can also perform semantic merging based on the aggregation result (320), and obtain an updated media content generation request based on the merged information. Here, the aggregation result may include one or more data entries, and the format of each data entry may be a keyword corresponding to the time element + a keyword corresponding to the location element + a keyword corresponding to the person element + a keyword corresponding to the event element. When performing semantic mergers on the aggregated results, similar entry mergers can be performed on keywords corresponding to time elements and keywords corresponding to location elements, and refined summaries can be performed on keywords corresponding to people elements and keywords corresponding to event elements. Based on the merger results, the input media content generation request is rewritten to fill in any missing event and people elements.

[0042] In this way, by complementing the semantic understanding of user 140's media content generation request with keywords corresponding to some missing elements of the media content generation request, more information can be obtained and used for subsequent material queries, thereby improving the scene coverage range of the generated media content.

[0043] Referring back to Figure 2, the service-side device 120 further determines a set of materials from the media content material library that matches at least one keyword (230). The media content material library may also be a resource library for storing various media content elements such as images, videos, and animated images for the user 140. Media content materials in the media content material library can be updated, for example, by adding new media content materials or deleting old or poor-quality media content materials. As an example, the media content material library may be a local album for storing photos, videos, etc., taken by the user 140 via the terminal device 110. Accordingly, updating media content materials can be performed by the user 140 through interaction with the terminal device 110. Alternatively, the media content material library may be a set of media content materials uploaded by a specific application 115. In another example, the media content materials in the media content material library may be automatically updated according to a predetermined request. The media content material library may also be called a visual material library or gallery.

[0044] In some embodiments, media content materials in a media content material library may be annotated with tags corresponding to at least some of several elements (i.e., time elements, place elements, people elements, and / or event elements). In this way, when determining a material set, a material set can be selected from the media content material library by extracting tags corresponding to at least one element annotated to each media content material in the media content material library and querying the tags corresponding to at least one element of each visual material with at least one keyword.

[0045] In such embodiments, media content materials may be annotated with tags corresponding to at least some of the elements of time, place, people, and events. For example, a particular image material may be annotated with tags corresponding to time, place, and event elements, respectively. Another image material may be annotated with tags corresponding to event and people elements, respectively.

[0046] In some embodiments, when annotating media content materials in a media content material library with tags corresponding to at least some of multiple elements, the service-side device 120 can analyze the descriptive information corresponding to the media content materials in the media content material library to determine the tags corresponding to at least some of the multiple elements corresponding to the media content materials. Next, based on the multiple elements, the service-side device 120 can construct a knowledge graph corresponding to the media content material library (224), and the knowledge graph shows the tags corresponding to at least some of the elements annotated to each media content material in the media content material library.

[0047] In such embodiments, descriptive information corresponding to media content material can be transmitted from terminal device 110 to service-side device 120. For example, terminal device 110 may be configured with at least a machine learning model for extracting descriptive information from media content material. For example, the machine learning model can extract captions, metadata (which may include creation time, latitude and longitude information, and background information) from images and videos, as well as information such as material aesthetics, similarity, or other information, which can be used as descriptive information for the media content. Note that when extracting captions from video, corresponding captions can be generated using video sequence frames. Terminal device 110 can maintain data consistency by periodically scanning materials and synchronizing the descriptive information of newly added materials in the media content material library with service-side device 120 (222). It should be understood that the extraction of descriptive information from media content material is done with the user's permission and complies with the requirements of relevant laws and regulations.

[0048] In some embodiments, a machine learning model 130 can be used to analyze synchronized media content material description information and complete a knowledge graph. In some embodiments, the knowledge graph is asynchronously constructed by the service-side device 120 and can be used as a search library to find at least one tag corresponding to each media content material. By using the knowledge graph to assist in the selection of materials to subsequently create images, the relationships between media content materials are extended, and the generated story media content becomes closely related to the actual representation of the user material.

[0049] In some embodiments, when selecting a material set from the media content material library, the service-side device 120 can select candidate material sets from the media content material library whose tags match at least one keyword (232), then filter the candidate material sets based on at least a predetermined material selection request (234), and finally determine the material set (230). In this way, an initial screening is performed on the media content materials, followed by a secondary screening to determine a high-quality material set. The following describes in detail specific embodiments of performing initial and secondary screening on media content materials in the media content material library.

[0050] In some embodiments, when selecting a candidate material set from a media content material library whose tags match at least one keyword, the service-side device 120 can select a predetermined number of media content materials from the media content material library whose tags match the keywords of each element, based on each of at least one element. For example, in the case of the keyword "X year X month X day" corresponding to the time element, the service-side device 120 can select media content materials from the media content material library that have "X year X month X day" in their tags. Similarly, a predetermined number of media content materials can be selected. Subsequently, the service-side device 120 can select a candidate material set related to the media content generation request from the predetermined number of media content materials. Here, the predetermined number of media content materials can be semantically compared with the media content generation request, and multiple media content materials that match the input meaning of the user 140 can be retained to form a candidate material set.

[0051] Figure 4 shows a flowchart of process 400 for initial screening of media content material according to some embodiments of the present invention. Process 400 is shown as a specific embodiment of the step in box 232 of Figure 2. In process 400, a rewrite request of a media content generation request is input (410), written to the rewrite request according to the input parameter format (420), and then an initial screening of media content material (the screening process shown in Figure 3) can be performed in batch according to at least one keyword in the rewrite request (430). In some embodiments, in the batch execution process, a predetermined number (e.g., 1000 or other) of media content materials whose tags match the keywords of these two elements can be selected from the media content material library as a pre-selected set (432), based on a time element, a location element, and keywords corresponding to these two elements (431).

[0052] Next, if the predetermined number is greater than a predetermined threshold, an initial RAG (Retrieval-Augmented Generation) screening can be performed. Specifically, the pre-selected set is analyzed to assemble a domain-specific language DSL (433), and an initial RAG screening based on vector features is performed based on the pre-selected set (434) to select a certain number (e.g., 200 or another number) of media content materials. If the predetermined number is less than or equal to a predetermined threshold, or if the initial RAG screening has already been performed (434), a semantic initial screening can be performed on the pre-selected set or the certain number of media content materials (435). Specifically, the pre-selected set or the certain number of media content materials is semantically compared with the media content generation request, and a list of materials that match the user input meaning can be retained as a candidate material set. The results of the initial media content material screening can then be output (440). It should be understood that process 400 is merely an example and not an exhaustive one.

[0053] Figure 5 shows a schematic diagram of an exemplary structure 500 for performing a secondary material screening on a candidate material set according to some embodiments of the present invention. The exemplary structure 500 is shown as the structure used in the step of box 234 in Figure 2. In the exemplary structure 500, a secondary media content material screening is performed to obtain a material set by employing an engineering selection method 510 or a model-based selection method 520 based on a media content generation request 501 and a candidate material set 504.

[0054] In the engineering selection method 510, in some embodiments, a deduplication process can be performed on the media content materials in the candidate material set 504 to exclude duplicate or similar media content materials. Next, it can be determined whether or not to filter the candidate material set based on the total number of materials in the candidate material set 504 and the number of materials corresponding to the keyword 502 within it. For example, if the total number of materials in the candidate material set exceeds 200, and / or if the number of materials corresponding to each keyword within it exceeds 40, it can be determined to perform filtering. The filtering method may include, but is not limited to, an aesthetic score filtering method and a time and location filtering method. In the case of the aesthetic score filtering method, the number of materials in the candidate material set 504 can be compressed based on the selection requirement of a minimum aesthetic score threshold and a method for filtering the first K (where K is a positive integer whose value range is greater than 1 and less than the number of materials in the candidate material set) media content materials. In this way, blurry, overexposed, dark, or unclear media content materials can be excluded. In the case of time and location filtering methods, materials with a wide time range and different locations within the candidate material set 504 can be excluded. It should be understood that the filtering method is not limited to this, and appropriate secondary screening filtering methods can be adopted depending on the actual needs.

[0055] In the model-based selection method 520, in some embodiments, when filtering the candidate material set, the service-side device 120 can use a machine learning model to filter the candidate material set based on the degree of match between the descriptive information corresponding to each media content material in the candidate material set and the material selection request. For example, the material selection request may include material quality requirements such as subject prominence requirements and detail clarity requirements. In some embodiments, the material selection request may further include the removal of duplicate materials and the selection of materials with an aesthetic score higher than a threshold. Specific material selection requirements can be configured according to the needs of the actual application and are not limited to these. Exemplarily, a material set can be obtained by using a language model to query the descriptive information corresponding to each media content material in the candidate material set against quality requirements and excluding a certain number of media content materials with low match based on numerical requirements.

[0056] In this way, by performing initial and secondary screening on media content materials, it is possible to screen for high-quality media content materials that match the meaning of user input requests, which helps to improve the quality of the created video works.

[0057] Referring back to Figure 2, the service-side device 120 further generates descriptive text for the media content to be generated (240) based on the material set and at least one keyword corresponding to at least one element. In some embodiments, if the media content to be generated is a video, the descriptive text may be called a story description. The descriptive text may include a series of segment descriptions for the media content to be generated. In some embodiments, if the media content to be generated is a video, the segments may be called storyboards. Accordingly, the segment descriptions may be called storyboard descriptions for storyboards in the video.

[0058] Figure 6 shows a schematic diagram of an exemplary structure 600 for generating a story description according to some embodiments of the present invention. The exemplary structure 600 is shown as the structure adopted in step box 240 of Figure 2. In structure 600, a story description template 610 can be used when generating a story description. The story description template 610 may include, for example, descriptive content regarding content outline methods, narrative style, content focus, etc., and a storyboard description can be directly entered therein.

[0059] In some embodiments, when generating descriptive text, the service-side device 120 can generate multiple segment descriptions for the media content to be generated based on the material set, the media content generation request, and keywords corresponding to the target element, for at least one of the target elements. The generated segment descriptions are multiple storyboard descriptions, which may include, for example, generated storyboard description 620-1, storyboard description 620-2, and storyboard description 620-3. The service-side device 120 can then merge the multiple storyboard descriptions into a story description (630). As an example, the service-side device 120 can generate multiple storyboard descriptions and a story description using a language model.

[0060] For example, if at least one element is a time element, a language model can be used to generate a storyboard description corresponding to the time element based on the material set, the media content generation request, and keywords corresponding to the time element (e.g., "X year X month X day"). In short, this storyboard description may be a time-based storyboard description. Accordingly, multiple storyboard descriptions can be generated for at least one element.

[0061] In some embodiments, the descriptive text may include a content summary that proceeds according to a segment description corresponding to the target element. The content summary can explain the order, context, etc., of the story. For example, if the target element is a time element, the descriptive text can be generated based on time changes, such as the time changes in a day (morning, noon, afternoon, etc.) or the time changes from a certain day to the third day, to show the content summary that changes over time.

[0062] In this way, based on the semantic understanding capabilities of the language model, it is possible to support the explanation of any storyline in natural language and provide more personalized automatic video creation effects.

[0063] Referring back to Figure 2, the service-side device 120 further selects multiple media content materials from the material set that match the descriptive text (250). In some embodiments, when selecting multiple media content materials from the material set that match the descriptive text, the service-side device 120 can select multiple media content materials from the material set based on whether the descriptive information corresponding to each media content material in the material set matches the descriptive text.

[0064] In such an embodiment, the service-side device 120 can determine multi-item material description information that matches the description text for the description information corresponding to the media content material in the material set, and can select multiple media content materials from the material set corresponding to each of the multiple description information items. The number of such multiple media content materials can be determined in advance. In some embodiments, multiple media content materials within a media content are sorted based on a content summary. For example, the service-side device 120 can sort multiple media content materials according to the content summary after selecting multiple media content materials from a material set. In another example, when selecting multiple media content materials from a material set, the service-side device 120 can directly select multiple media content materials in order according to the content summary.

[0065] In this way, based on the explanatory text and multiple media content materials that match the explanatory text, the generated media content can be made to better match the actual representation of the media content materials and the meaning required by the user, thereby improving the user experience.

[0066] Furthermore, the service-side device 120 generates media content 150 based on multiple media content materials. The media content may include, for example, video. In some embodiments, when generating media content 150, the service-side device 120 can determine a media content template based on descriptive text and multiple media content materials. The service-side device 120 can then generate media content 150 based on the multiple media content materials and the media content template. For example, multiple media content materials can be input into the media content template to generate media content 150. It should be understood that media content 150 can be generated based on multiple media content materials and a media content template in other ways.

[0067] In some embodiments, when determining a media content template based on descriptive text and multiple media content materials, the service-side device 120 can select at least one media content template from multiple candidate media content templates, based on the number of media content materials, that has a corresponding number of material placement positions. In such embodiments, each media content template has its own multiple material placement positions. For example, one material can be placed in each material placement position. Furthermore, the service-side device 120 can select a media content template from at least one media content template based on the descriptive text (260).

[0068] In some embodiments, when selecting a media content template from at least one media content template, the service-side device 120 selects the media content template by using a language model to determine whether the template description corresponding to each of the at least one media content template matches the descriptive text. The template description corresponding to the media content template matches the descriptive text, and the template description may include at least a scene description.

[0069] For example, scenes can be classified into multiple categories, such as social scenes, travel scenes, festival scenes, pet scenes, dining scenes, sports scenes, and reading scenes, but are not limited to these. Each scene may have a corresponding scene description. Furthermore, a general-purpose scene description can be provided to adapt to general-purpose scenes. Exemplaryly, a template description may further include descriptions of time (e.g., descriptions related to seasons), descriptions of music (e.g., descriptions of different musical styles such as relaxing, cheerful, and rock), or descriptions of other dimensions.

[0070] Figure 7 shows a flowchart of process 700 for selecting a media content template according to some embodiments of the present invention. Process 700 is shown as some embodiments of the steps in box 260 of Figure 2. In process 700, a language model can be used to extract multiple matching tags from the descriptive text (710), after which the service-side device 120 can perform an initial template screening (720). In the initial template screening (720) process, first, media content templates with more slot positions (i.e., material placement positions) than the number of materials are excluded in order to avoid repeated playback of media content material. Then, among media content templates with the same slots, those with a high degree of tag matching are sorted to the top. Subsequently, media content templates with a high quality score are sorted to the top. In this way, an initial set of candidate templates containing multiple media content templates is selected. In addition, the recommended results within a certain time period are removed from the candidate template set to ensure that duplicate media content templates are not used in a short period of time.

[0071] Furthermore, the service-side device 120 executes the template array (730). Specifically, it can use a language model to select the optimal media content template based on the degree of match between the descriptive text and the template description. It also outputs the ID (identification information) of the media content template. Subsequently, it queries the template ID against multiple media content materials (740). Next, the service-side device 120 can send the media content template and multiple media content materials to the terminal device 110's video creation software toolkit (SDK). The SDK may be a set of related tools, documentation, and sample code for developing a specific software application program, software framework, hardware platform, operating system, etc. The SDK on the terminal device 110 may retrieve the media content template and multiple media content materials and perform real-time rendering of the media content 150.

[0072] In some embodiments, the service-side device 120 can use a separate machine learning model to generate matching music text content based on the descriptive text of the media content to be generated. Then, a melody matching the music text content is added to generate the target music. Furthermore, the target music is added to the media content. This allows for obtaining media content with background music.

[0073] In such embodiments, another machine learning model may be built upon a language model. This machine learning model may be configured to generate lyrics (i.e., musical text content) that match the descriptive text based on an input prompt word. The prompt word may be, for example, "You are a professional lyricist. You can create short, upbeat lyrics based on a given theme. The theme is XXX." After the lyrics are generated, a machine learning model with a music tool or other music generation capabilities can be invoked to generate a matching melody based on the lyrics. By combining the melody and lyrics, the target music can be generated. In this way, the target music may match the screen of the media content. Thus, richer media content that fits the context of the story can be obtained. Embodiments of the present invention can match the generated story media content to the user's requested meaning and to the actual submitted content of the media content material by querying the user's input request meaning with multiple elements, querying both with the story description, and querying the media content material with the story description, thereby avoiding loss of information expressed by the media content material. This expands the relevance between media content materials, resulting in high-quality storytelling media content that meets user needs and helps improve the user experience.

[0074] Figure 8 shows a flowchart of process 800 for generating media content according to some embodiments of the present invention. Process 800 can be implemented in terminal device 110 and / or service-side device 120. Below, an example in which the embodiment is implemented in service-side device 120 will be given. Process 800 will be described below with reference to Figure 1.

[0075] As shown in the drawing, in box 810, in response to receiving a media content generation request from a user, the service-side device 120 determines, based on the media content generation request, at least one keyword corresponding to at least one of a plurality of elements related to media content generation.

[0076] In box 820, the service-side device 120 determines a set of materials from the media content material library that matches at least one keyword.

[0077] In box 830, the service-side device 120 generates descriptive text for the media content to be generated, based on the material set and at least one keyword.

[0078] In box 840, the service-side device 120 selects multiple media content materials from the material set that match the descriptive text.

[0079] In box 850, the service-side device 120 generates media content based on multiple media content materials.

[0080] In some embodiments, determining at least one keyword includes extracting at least one keyword corresponding to at least one element of a plurality of elements from a media content generation request, and / or determining a keyword corresponding to at least one other element of a plurality of elements based on semantic analysis of the media content generation request.

[0081] In some embodiments, multiple elements include at least two of the following: time elements, place elements, person elements, and event elements.

[0082] In some embodiments, media content materials in a media content material library are annotated with tags corresponding to at least some of several elements, and determining a material set includes extracting tags corresponding to at least one element annotated to each visual material in the media content material library, and selecting a material set from the media content material library by querying the tags corresponding to at least one element of each visual material with at least one keyword.

[0083] In some embodiments, selecting a material set from a media content material library includes selecting candidate material sets from the media content material library whose tags match at least one keyword, and filtering the candidate material sets based on at least a predetermined material selection request to determine the material set.

[0084] In some embodiments, selecting a set of candidate materials from a media content material library whose tags match at least one keyword includes selecting a predetermined number of media content materials from the media content material library whose tags match the keyword of each element, based on each of at least one element, and selecting a set of candidate materials from the predetermined number of media content materials that are relevant to the media content generation request.

[0085] In some embodiments, filtering the candidate material set involves using a machine learning model to filter the candidate material set based on the degree of match between the descriptive information corresponding to each media content material in the candidate material set and the material selection request.

[0086] In some embodiments, process 800 further includes annotating media content materials in a media content material library with tags corresponding to at least some of the elements of a plurality of elements by analyzing descriptive information corresponding to media content materials in a media content material library to determine tags corresponding to at least some of the elements of a plurality of elements corresponding to the media content materials; and constructing a knowledge graph corresponding to the media content material library, wherein the knowledge graph shows tags corresponding to at least some of the elements annotated to each media content material in the media content material library.

[0087] In some embodiments, generating descriptive text includes generating multiple segment descriptions for the media content to be generated based on a material set, a media content generation request, and keywords corresponding to the target element, for at least one of the target elements, and merging the multiple segment descriptions into descriptive text, wherein the descriptive text includes multiple segment descriptions arranged in order.

[0088] In some embodiments, the descriptive text includes a content summary that proceeds according to a segment description corresponding to a target element, where multiple media content materials within the media content are sorted based on the content summary.

[0089] In some embodiments, selecting multiple media content materials from a material set that match descriptive text includes selecting multiple media content materials from a material set based on the fact that the descriptive information corresponding to each media content material in the material set matches the descriptive text.

[0090] In some embodiments, generating media content includes determining a media content template based on descriptive text and multiple media content materials, and generating media content based on the multiple media content materials and the media content template.

[0091] In some embodiments, determining a media content template based on descriptive text and multiple media content materials includes selecting at least one media content template having a corresponding number of material placement positions from multiple candidate media content templates based on the number of multiple media content materials, and selecting a media content template from at least one media content template based on the descriptive text, wherein the template description corresponding to the media content template matches the descriptive text, and the template description includes at least a scene description.

[0092] In some embodiments, process 800 further includes using another machine learning model to generate matching music text content based on the descriptive text of the media content to be generated, adding a melody that matches the music text content to generate target music, and adding the target music to the media content.

[0093] Figure 9 shows a schematic structural block diagram of a media content generation apparatus 900 according to a specific embodiment of the present invention. The apparatus 900 may be implemented as a service-side device 120 or included within the service-side device 120. Each module / component within the apparatus 900 may be implemented by hardware, software, firmware, or any combination thereof.

[0094] As shown in the drawing, the apparatus 900 includes: a determination module 910 configured to determine at least one keyword corresponding to at least one element of a plurality of elements related to a media content generation story based on a video media content generation query request, in response to receiving a video media content generation query request from a user; a material set determination module 920 configured to determine a material set from a material library that matches at least one keyword for the media content; a description text generation module 930 configured to generate description text for the media content to be generated based on the material set and at least one keyword; a media content material selection module 940 configured to select a plurality of media content materials from the material set that match the description text; and a media content generation module 950 configured to generate media content based on the plurality of media content materials.

[0095] In some embodiments, the keyword determination module 910 is further configured to extract at least one keyword corresponding to at least one element of a plurality of elements from a media content generation request, and / or to determine a keyword corresponding to at least one other element of a plurality of elements based on semantic analysis of the media content generation request.

[0096] In some embodiments, multiple elements include at least two of the following: time elements, place elements, person elements, and event elements.

[0097] In some embodiments, media content materials in the media content material library are annotated with tags corresponding to at least some of multiple elements, and the material set determination module 920 is configured to further extract tags corresponding to at least one element annotated to each visual material in the material library for the media content, and to select a material set from the media content material library by querying the tags corresponding to at least one element of each visual material with at least one keyword.

[0098] In some embodiments, the apparatus 900 is further configured to select candidate material sets from a visual media content material library whose tags match at least one keyword, and to filter the material quality requirement candidate material sets based on at least a predetermined material selection requirement to determine a material set.

[0099] In some embodiments, the apparatus 900 is further configured to select a predetermined number of media content materials from a media content material library, based on each of at least one of the elements, whose tags match the keywords of each element, and to select a set of candidate materials related to a media content generation request from the predetermined number of media content materials.

[0100] In some embodiments, the device 900 is further configured to use a machine learning model to filter the candidate material set based on the degree of match between the descriptive information corresponding to each media content material in the candidate material set and the material selection request.

[0101] In some embodiments, the apparatus 900 further comprises an annotation module configured to analyze descriptive information corresponding to media content materials in a material library for media content, determine tags corresponding to at least some of the elements among a plurality of elements corresponding to the media content material, and construct a knowledge graph corresponding to the media content material library, the knowledge graph showing tags corresponding to at least some of the elements annotated to each media content material in the media content material library.

[0102] In some embodiments, the descriptive text generation module 930 is further configured to generate multiple segment descriptions for the media content to be generated based on the material set, the media content generation request, and keywords corresponding to the target element, for at least one of the elements, and to merge the multiple segment descriptions into a descriptive text, the descriptive text containing the multiple segment descriptions arranged in order.

[0103] In some embodiments, the descriptive text includes a content summary that proceeds according to a segment description corresponding to a target element, where multiple media content materials within the media content are sorted based on the content summary.

[0104] In some embodiments, the media content material selection module 940 is further configured to include selecting multiple media content materials from a material set that match the descriptive text, based on the fact that the descriptive information corresponding to each media content material in the material set matches the descriptive text.

[0105] In some embodiments, the media content generation module 950 is further configured to determine a media content template based on descriptive text and multiple media content materials, and to generate media content based on the multiple media content materials and the media content template.

[0106] In some embodiments, the apparatus 900 is configured to select at least one media content template having a corresponding number of material placement positions from a plurality of candidate media content templates based on the number of media content materials, and to select a media content template from at least one media content template based on descriptive text, wherein the template description corresponding to the media content template matches the descriptive text, and the template description includes at least a scene description.

[0107] In some embodiments, the device 900 further comprises a music addition module configured to use another machine learning model to generate matching music text content based on the descriptive text of the media content to be generated, add a matching melody to the music text content to generate target music, and add the target music to the media content.

[0108] The units and / or modules included in the device 900 may be implemented using a variety of methods, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to, or instead of, machine-executable instructions, some or all units and / or modules within the device 900 may be implemented at least partially by one or more hardware logic components. Exemplary types of hardware logic components that may be used, but not limited to, include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on a chip (SOCs), and composite programmable logic devices (CPLDs).

[0109] Figure 10 shows a block diagram of an electronic device 1000 that can carry out one or more embodiments of the present invention. It should be understood that the electronic device 1000 shown in Figure 10 is illustrative and should not limit the functionality and scope of the embodiments described herein. The electronic device 1000 shown in Figure 10 may be used to carry out the augmented reality device 120 of Figure 1.

[0110] As shown in Figure 10, the electronic device 1000 is in the form of a general-purpose electronic device. The components of the electronic device 1000 may include, but are not limited to, one or more processors or processing units 1010, memory 1020, storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. The processing unit 1010 may be an actual or virtual processor and can perform various processes based on a program stored in memory 1020. In a multiprocessor system, the parallel processing capability of the electronic device 1000 is improved by having multiple processing units execute computer executable instructions in parallel.

[0111] The electronic device 1000 typically includes multiple computer storage media. Such media may include, but are not limited to, volatile and non-volatile media, removable and non-removable media, and may be any obtainable media accessible by the electronic device 1000. Memory 1020 may be volatile memory (e.g., registers, fast cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or a specific combination thereof. Storage device 1030 may be removable or non-removable media, and may include machine-readable media such as flash memory drives, magnetic disks, or any other media, which may be used to store information and / or data and may be accessible within the electronic device 1000.

[0112] The electronic device 1000 may further include other removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 10, a magnetic disk drive for reading from or writing to removable, non-volatile magnetic disks (e.g., “floppy disks”) and a removable optical disk drive for reading from or writing to non-volatile optical disks may be provided. In these cases, each drive may be connected to a path (not shown) by one or more data medium interfaces. The memory 1020 may also include a computer program product 1025 having one or more program modules, which are configured to perform various methods or operations of various embodiments of the present invention.

[0113] The communication unit 1040 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the electronic device 1000 may be implemented as a single computing cluster or as multiple computing machines, which can communicate via communication connections. Therefore, the electronic device 1000 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or other network nodes.

[0114] The input device 1050 may be one or more input devices such as a mouse, keyboard, or trackball. The output device 1060 may be one or more output devices such as a display, speaker, or printer. The electronic device 1000 may further communicate with one or more external devices (not shown) such as a storage device or display device via the communication unit 1040, as needed, or with one or more devices that enable a user to interact with the electronic device 1000, or with any device (e.g., a netbook card, modem) that enables the electronic device 1000 to communicate with one or more other computing devices. Such communication may be performed via an input / output (I / O) interface (not shown).

[0115] An exemplary embodiment of the present invention provides a computer-readable storage medium in which computer-executable instructions are stored, and the computer-executable instructions are executed by a processor to realize the above method. An exemplary embodiment of the present invention further provides a computer program product, the computer program product including computer-executable instructions, which is tangibly stored on a non-temporary computer-readable medium, and the computer-executable instructions are executed by a processor to realize the above method.

[0116] Herein, each aspect of the present invention has been described with reference to flowcharts and / or block diagrams of methods, apparatus, devices, and computer program products realized by the present invention. It should be understood that each box in the flowcharts and / or block diagrams, and each combination of boxes in the flowcharts and / or block diagrams, may be realized by computer-readable program instructions.

[0117] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device to generate a machine that, when these instructions are executed by the computer or other programmable data processing device's processing unit, generates a device for performing functions / operations specified in one or more boxes in a flowchart and / or block diagram. These computer-readable program instructions may be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other device to operate in a particular manner so that the computer-readable medium on which the instructions are stored constitutes a product containing instructions for performing each aspect of the functions / operations specified in one or more boxes in a flowchart and / or block diagram.

[0118] By loading computer-readable program instructions into a computer, other programmable data processing device, or other device, a series of operational steps are performed on the computer, other programmable data processing device, or other device to generate a computer-implemented process, thereby enabling the instructions executed on the computer, other programmable data processing device, or other device to perform a function / operation specified in one or more boxes in a flowchart and / or block diagram.

[0119] The flowcharts and block diagrams in the drawings illustrate the implementable architectures, functions, and operations of systems, methods, and computer program products of multiple implementations according to the present invention. In this regard, each box in a flowchart or block diagram may represent a module, program segment, or part of an instruction, and a module, program segment, or part of an instruction contains one or more executable instructions for implementing a specified logical function. In some implementations as replacements, the functions represented in the boxes may occur in a different order than those shown in the drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or in reverse order depending on the functions involved. It should also be noted that each box in a block diagram and / or flowchart, and combinations of boxes in a block diagram and / or flowchart, may be implemented by a special-purpose hardware-based system that performs a specified function or operation, or by a combination of special-purpose hardware and computer instructions.

[0120] While the various realizations of the present invention have been described above, the above descriptions are illustrative, not exhaustive, and not limited to the realizations disclosed. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the realizations described. The choice of terms used herein is intended to best interpret the principle, practical application, or improvement to the art in the market of each realization, or to enable those skilled in the art to understand each realization disclosed herein.

Claims

1. A method for generating media content, In response to receiving a media content generation request from a user, the system determines at least one keyword corresponding to at least one of several elements related to media content generation, based on the media content generation request. From the media content material library, determine a set of materials that matches at least one of the aforementioned keywords, Based on the aforementioned set of materials and at least one keyword, the process involves generating descriptive text for the media content to be generated, Selecting multiple media content materials from the aforementioned material set that match the descriptive text, This includes generating media content based on the aforementioned multiple media content materials, method.

2. Determining at least one of the aforementioned keywords is Extracting at least one keyword corresponding to at least one of the multiple elements from the media content generation request, and / or, The process includes determining a keyword corresponding to at least one other element among the plurality of elements, based on semantic analysis of the media content generation request. The method according to claim 1.

3. The aforementioned multiple elements include at least two of the following: time elements, place elements, person elements, and event elements. The method according to claim 1.

4. The media content materials in the aforementioned media content material library are annotated with tags corresponding to at least some of the elements among the multiple elements, Determining the aforementioned set of materials means Extracting tags corresponding to at least one element annotated to each visual material in the media content material library, Selecting the material set from the media content material library by querying the tags corresponding to at least one element of each visual material with the at least one keyword, The method according to claim 1.

5. Selecting the material set from the aforementioned media content material library means Select a set of candidate materials from the media content material library whose tags match at least one of the keywords, This includes filtering the candidate material set based on at least a predetermined material selection requirement to determine the material set, The method according to claim 4.

6. Selecting a set of candidate materials from the aforementioned media content material library whose tags match at least one of the aforementioned keywords is: Based on each of the at least one of the aforementioned elements, a predetermined number of media content materials are selected from the media content material library, the tags of which match the keywords of each element. This includes selecting the candidate material set related to the media content generation request from the predetermined number of media content materials, The method according to claim 5.

7. Filtering the aforementioned set of candidate materials is This includes using a machine learning model to filter the candidate material set based on the degree of match between the descriptive information corresponding to each media content material in the candidate material set and the material selection request, The method according to claim 5.

8. The process involves analyzing the descriptive information corresponding to the media content material in the media content material library to determine tags corresponding to at least some of the elements among the multiple elements corresponding to the media content material, The method involves constructing a knowledge graph corresponding to the media content material library, wherein the knowledge graph indicates tags corresponding to at least some of the elements annotated to each media content material in the media content material library, and further includes annotating the media content materials in the media content material library with tags corresponding to at least some of the elements of the plurality of elements. The method according to claim 4.

9. Generating the aforementioned explanatory text means For a target element among at least one of the aforementioned elements, multiple segment descriptions for the media content to be generated are generated based on the material set, the media content generation request, and keywords corresponding to the target element. The process involves merging the aforementioned multiple segment descriptions into the descriptive text, wherein the descriptive text includes the aforementioned multiple segment descriptions arranged in order, and includes the following: The method according to claim 1.

10. The descriptive text includes a content overview that proceeds according to the segment description corresponding to the target element, and the multiple media content materials within the media content are sorted based on the content overview. The method according to claim 9.

11. Selecting the multiple media content materials from the aforementioned material set that match the descriptive text is, The process includes selecting the multiple media content materials from the material set based on whether the descriptive information corresponding to each media content material in the material set matches the descriptive text. The method according to claim 1.

12. The generation of the aforementioned media content is Based on the aforementioned explanatory text and the aforementioned multiple media content materials, a media content template is determined, This includes generating media content based on the aforementioned plurality of media content materials and the media content template, The method according to claim 1.

13. Determining the media content template based on the descriptive text and the multiple media content materials is, Based on the number of media content materials mentioned above, select at least one media content template from a plurality of candidate media content templates that has a corresponding number of material placement positions. Based on the descriptive text, selecting a media content template from the at least one media content template, wherein the template description corresponding to the media content template matches the descriptive text, and the template description includes at least a scene description, and includes, The method according to claim 12.

14. Using a different machine learning model, generate matching music text content based on the descriptive text of the media content to be generated, The process involves adding a melody that matches the aforementioned musical text content to generate the target music, Adding the target music to the media content, The method according to claim 1.

15. A device for generating media content, A keyword determination module configured to determine at least one keyword corresponding to at least one element among a plurality of elements related to media content generation, in response to receiving a media content generation request from a user, A material set determination module configured to determine a set of materials from a material library that matches at least one of the keywords mentioned above for media content, A descriptive text generation module configured to generate descriptive text for media content to be generated based on the aforementioned material set and the aforementioned at least one keyword, A media content material selection module configured to select multiple media content materials from the aforementioned material set that match the descriptive text, The system includes a media content generation module configured to generate media content based on the aforementioned plurality of media content materials, Device.

16. It is an electronic device, At least one processing unit, The device comprises at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, wherein, when the instructions are executed by the at least one processing unit, the electronic device causes the electronic device to perform the method according to any one of claims 1 to 14. Electronic devices.

17. A computer program is stored, and the computer program is executed by a processor to realize the method according to any one of claims 1 to 14. Computer-readable storage medium.

18. A computer storage medium is tangibly stored and includes a computer executable instruction, wherein when the computer executable instruction is executed by a device, the device performs the method according to any one of claims 1 to 14. Computer program products.

Images