Media content processing method, apparatus, device, and computer program product

Abstract

Embodiments of this disclosure relate to media content processing methods, apparatus, devices, and computer program products. The method includes displaying first media content and a first control configured to trigger the generation of related media content based on the first media content. The method further includes displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content. Furthermore, the method includes displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation. Thus, by generating related media content based on reference media content and user input, this method can improve the similarity between the related media content and the reference media content, making the generated related media content more in line with the user's expectations when creating media content. Moreover, by displaying corresponding operation information to the user instead of displaying prompts, the operational threshold for users can be lowered, improving the interactive efficiency and user experience when creating media content.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically to methods, apparatus, devices and computer program products for processing media content. Background Technology

[0002] With the development of image generation, video generation, and multimedia content editing technologies, users can generate media content such as images and videos through applications, and browse media content published by other users in content communities.

[0003] In content communities or media content generation applications, when users browse media content that interests them, they may want to generate related media content with a similar style. For example, a user might want to reference the visual style, composition, subject proportions, background effects, text layout, or overall visual effect of a published image to generate an image similar to it. Similarly, users may also want to reference published video, audio, or other media content to generate corresponding related media content. Summary of the Invention

[0004] In the example embodiments, a media content processing method, apparatus, device, computer storage medium, and computer program product are provided.

[0005] In a first aspect, a media content processing method is provided. The method includes displaying first media content and a first control configured to trigger the generation of related media content referencing the first media content. The method further includes displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content. Furthermore, the method includes displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0006] Therefore, this method, by generating relevant media content based on reference media content and user input, can improve the similarity between the relevant media content and the reference media content, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to the user instead of showing prompts, it can lower the operational threshold for users, improve the interaction efficiency and user experience when creating media content.

[0007] In a second aspect, a media content processing apparatus is provided. The apparatus includes a first display module configured to display first media content and a first control configured to trigger the generation of related media content based on the first media content. The apparatus also includes a second display module configured to display first information in response to a first operation on the first control, the first information indicating a second operation on the first media content. Furthermore, the apparatus includes a third display module configured to display second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0008] In a third aspect, an electronic device is provided, comprising: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform an action when executed by the at least one processing unit, the action comprising: displaying first media content and a first control, the first control being configured to trigger the generation of related media content with reference to the first media content; displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; and displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0009] In a fourth aspect, a computer-readable storage medium is provided having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform an action including: displaying first media content and a first control, the first control being configured to trigger the generation of related media content with reference to the first media content; displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; and displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0010] In a fifth aspect, a computer program product is provided, including computer-executable instructions, wherein when executed by a processor, the computer-executable instructions cause the processor to perform actions, the actions including: displaying first media content and a first control, the first control being configured to trigger the generation of related media content with reference to the first media content; displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; and displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0011] It should be understood that the content described in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0013] Figure 1 A schematic diagram of an example environment in which some embodiments of this disclosure may be implemented is shown;

[0014] Figure 2 A flowchart illustrating a media content processing method according to some embodiments of this disclosure is shown;

[0015] Figure 3 A schematic diagram illustrating a replacement process according to some embodiments of the present disclosure is shown;

[0016] Figure 4 A schematic diagram illustrating a modification process according to some embodiments of the present disclosure is shown;

[0017] Figure 5 A flowchart illustrating a process for determining an operation type according to some embodiments of this disclosure is shown;

[0018] Figure 6 A flowchart of a media content generation process according to some embodiments of the present disclosure is shown;

[0019] Figure 7 Block diagrams of media content processing apparatuses according to some embodiments of the present disclosure are shown; and

[0020] Figure 8 A schematic block diagram of an example device suitable for implementing various embodiments of the present disclosure is illustrated. Detailed Implementation

[0021] It is understood that all user-related data involved in this technical solution should be obtained and used only after authorization from the user. This means that if it is necessary to use a user's personal information in this technical solution, the user's explicit consent and authorization are required before obtaining this data; otherwise, no related data collection and use will be carried out. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly followed in the process of data collection, use, and storage, and necessary technical measures should be taken to protect user data security and ensure the secure use of data.

[0022] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0023] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0024] As mentioned earlier, in content communities or media content generation applications, when users browse media content that interests them, they may want to generate media content with a similar presentation style. For example, a user may want to generate an image similar to a published image by referencing its visual style, composition, subject proportions, background effects, text layout, or overall visual effect.

[0025] For example, to facilitate user creation based on viewed media content, some applications offer a "create similar content" feature. While browsing media content, users can click on controls on the media content page to enter the corresponding media content processing flow. This flow typically uses the user-selected media content as reference media content, such as the original work, template work, or work to be referenced, and combines this with user input to generate corresponding related media content. Related media content can be, for example, similar images or videos generated based on the reference media content. The generated related media content can be saved, shared, published, or further edited.

[0026] In related technologies, after a user clicks the control used to create similar content, the system typically displays a prompt to generate reference media content, receives the media content uploaded by the user, and then generates the corresponding related media content based on the prompt and the media content uploaded by the user.

[0027] However, these technologies primarily rely on prompts to drive media content generation, resulting in poor stability of the generated results. Since prompts are typically described using natural language, they struggle to fully represent the display features of the reference media content, such as composition, layout, lighting, style, and object relationships. Therefore, even using the same prompts, the generated results may differ significantly depending on the input, leading to low consistency between the generated results and the reference media content.

[0028] Furthermore, in some scenarios, the display effects in the reference media content are not only related to the prompt words but also to the original media content used to generate the reference media content. For example, the original media content may itself include fireworks elements, while the corresponding prompt words are only used to instruct on how to make the elements in the original media content produce dynamic effects. The reference media content generated based on the original media content and the prompt words can then present dynamic fireworks effects.

[0029] In this scenario, when browsing reference media content, the user might expect the generated related media content to also have a dynamic fireworks effect. However, current technologies typically only display corresponding prompts to the user and generate related media content based on the user's input and the prompts. When the user's input media content does not include fireworks elements, even using the same prompts, it is difficult to generate related media content with a dynamic fireworks effect, resulting in a mismatch between the generated result and the user's expectations.

[0030] Furthermore, related technologies typically present prompts directly to users and require them to complete subsequent actions based on those prompts. However, prompts often contain numerous generation parameters or complex descriptive information, making it difficult for ordinary users to accurately understand the specific meaning of the prompts and their corresponding generation logic. This increases the operational threshold for users and reduces the user experience.

[0031] Furthermore, when browsing reference media content, users are more concerned with how to generate related media content with a similar presentation style, rather than with the prompts corresponding to the reference media content. For example, when browsing reference media content, users typically want to create related media content with similar visual effects, style, or display effects, rather than wanting to know what prompts were used to generate the reference media content, nor do they want to write or adjust the corresponding prompts to generate related media content themselves.

[0032] Compared to the prompts themselves, users are typically more concerned with the next step to complete the creation. Therefore, directly presenting users with prompts and requiring them to perform subsequent actions based on those prompts doesn't align with users' mindset when creating content. This makes it difficult for users to quickly complete the corresponding editing operations, reducing media content generation efficiency and negatively impacting the user experience.

[0033] In summary, the relevant technologies primarily rely on prompts to drive the generation of related media content, guiding users to create similar content by displaying these prompts. However, prompts often fail to fully represent the content characteristics of the referenced media, resulting in low consistency between the generated content and the referenced media. Furthermore, directly displaying prompts to users and requiring them to perform subsequent actions based on them does not align with users' preconceived notions of creating similar content based on referenced media, thus impacting the efficiency of media content generation and the user experience.

[0034] Therefore, embodiments of this disclosure provide a media content processing solution. In this solution, when a user processes media content, corresponding operation information can be displayed to instruct the user to perform the corresponding operation, and relevant media content can be generated and displayed based on reference media content and user input.

[0035] Therefore, by generating relevant media content based on reference media content and user input, the similarity between the relevant media content and the reference media content can be improved, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to users instead of showing prompts, the operational threshold for users can be lowered, improving the interaction efficiency and user experience when creating media content.

[0036] Figure 1 A schematic diagram of an example environment 100 in which some embodiments of this disclosure may be implemented is shown. For example... Figure 1 As shown, environment 100 may include computing device 110. Computing device 110 may be various types of electronic devices, such as mobile devices, desktop computers, laptop computers, or servers, or it may be a computing system, a single server, a distributed server, or a cloud-based server.

[0037] In some embodiments, the computing device 110 may run media content processing applications, content community applications, image generation applications, video generation applications, social applications, or other applications that support media content creation. For example... Figure 1 As shown, the application page can display media content 120 (also known as first media content) and controls 130 (also known as first controls). Media content 120 can be, for example, images, videos, audio, or other media content. When browsing media content, users may want to generate corresponding related media content based on the currently viewed media content 120. For example, users may want to create similar content based on the visual style, composition, background effects, main layout, dynamic effects, or other display effects in media content 120.

[0038] In some embodiments, control 130 can be used to trigger a media content processing flow that generates related media content based on media content 120. For example, a user can operate control 130 by clicking, touching, long-pressing, or other means to trigger the corresponding media content processing flow. In some embodiments, control 130 may display "Make the same style," "Create the same style," "Create the same style," or other information indicating the generation of the same style content to prompt the user to create the same style content through media content 120.

[0039] refer to Figure 1 After the user operates the control 130, the computing device 110 can display information 140 (also referred to as first information) on the page. Information 140 can be used to guide the user to perform a corresponding operation. For example, in some embodiments, information 140 can be used to guide the user to upload media content to replace the main content in media content 120. In some embodiments, information 140 can be used to guide the user to input text content to modify the text content in media content 120. In some embodiments, information 140 can be used to guide the user to add media content, special effects content, or other elements.

[0040] In some embodiments, the computing device 110 may display information 140 in an input control. For example, after clicking control 130, an input box may pop up, displaying information 140 in gray font or other ways to prompt the user to perform a corresponding operation. For example, information 140 may include "Select image to replace the subject," "Enter text to modify," "Add element content," or other prompts. The user can perform corresponding operations by manipulating the corresponding input control. For example, the user can upload media content for replacement, enter text content for modification, add element content, or perform other media content processing operations through the input control.

[0041] After the user performs the corresponding operation, the computing device 110 can display media content 150 (also known as second media content) on the application page. Media content 150 can be generated based on media content 120 and user input. For example, after the user uploads media content to replace the main content, media content 150 can be generated after replacing the main content in media content 120 with the corresponding media content.

[0042] For example, media content 120 could be a perfume image. In this perfume image, the perfume bottle is situated on water, and the image includes display effects such as water ripples and reflections. When browsing this perfume image, a user might want to reference its visual effects to create a headphone image. Then, the user can upload a headphone image as media content to replace the main subject. The main subject in the generated media content 150 can be replaced with the headphone image, while retaining the water ripple and reflection effects, thus generating a headphone image with a similar visual style to the perfume image.

[0043] Therefore, according to the example environment 100 of this disclosure, after the user operates the control 130, the computing device 110 will not display prompts corresponding to the media content 120 to the user, but will directly display information 140 to guide the user to perform the corresponding operation. This avoids the cost to the user in understanding prompts, makes the media content processing flow more consistent with the user's mindset when creating similar content, and thus improves the user interaction experience.

[0044] Furthermore, media content 150 is generated based on media content 120 and user input, which can improve the similarity between media content 150 and media content 120, making the generated media content 150 more in line with the user's expectations when creating similar content. In addition, compared to generating media content based solely on prompts, it can reduce the problem of unstable generation results caused by incomplete prompt descriptions.

[0045] It should be understood that the architecture and functionality in example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Embodiments of this disclosure can also be applied to other environments with different structures and / or functionalities.

[0046] The following will combine Figures 2 to 8 The process according to embodiments of this disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0047] Figure 2 A flowchart of a method 200 for media content processing according to some embodiments of the present disclosure is shown. It should be understood that method 200 may also include additional actions not shown, and the scope of the present disclosure is not limited in this respect. The following is in conjunction with... Figure 1 Example environment 100 is used to describe method 200 in detail.

[0048] At box 202, first media content and a first control are displayed. The first control is configured to trigger the generation of related media content based on the first media content. For example, combined with... Figure 1 As described, computing device 110 can display media content 120 and control 130, which is configured to trigger the generation of related media content from reference media content 120.

[0049] At box 204, in response to a first operation on the first control, first information is displayed, indicating a second operation on the first media content. For example, combined with... Figure 1 As described above, computing device 110 can display information 140 in response to a trigger operation on control 130, and information 140 can instruct the user to perform a corresponding operation to process media content.

[0050] At box 206, in response to the second operation, second media content is displayed. This second media content is generated based on the first media content and user input corresponding to the second operation. For example, combining... Figure 1 As described, computing device 110 can display media content 150 in response to an operation performed by a user, the media content 150 being generated based on media content 120 and user input corresponding to the operation performed by the user.

[0051] Therefore, according to the method 200 of the embodiments of this disclosure, by generating relevant media content based on reference media content and user input, the similarity between the relevant media content and the reference media content can be improved, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to the user instead of showing prompts, the operational threshold for the user can be lowered, improving the interaction efficiency and user experience when creating media content.

[0052] Figure 3 A schematic diagram of a replacement process 300 according to some embodiments of the present disclosure is shown. When browsing reference media content, a user may wish to retain the visual style, composition, background effects, lighting effects, or other display effects of the reference media content, while replacing the main content of the reference media content with the main content uploaded by the user to generate corresponding similar content. For example, a user may wish to reference the visual effects of images of objects, people, pets, landscapes, or other media content to create media content with a similar presentation. In some embodiments of the present disclosure, when the system determines that the user may wish to perform a main content replacement operation, it can display corresponding information to instruct the user to perform the main content replacement operation. After the user uploads media content for replacing the main content, the main content of the reference media content can be replaced with the main content corresponding to the user-uploaded media content, thereby generating corresponding related media content.

[0053] like Figure 3As shown, when browsing media content 302, a user may find its visual effects appealing and wish to use it as a template or reference to create similar content. For example, media content 302 could be a product display image of a perfume bottle, including the main object (i.e., the perfume bottle), water ripple effects, reflection effects, or other display effects. The user can activate the "Make Similar" control 304 to trigger the corresponding media content processing flow. After the user activates the "Make Similar" control 304, information 306 can be displayed to prompt the user to perform a replacement operation.

[0054] For example, such as Figure 3 As shown, information 306 may include "Select an image to replace the subject." This clarifies the subsequent actions for the user, reducing the cost of understanding the prompt. In some embodiments, the computing device may display a media content selection interface 308 (e.g., a user's photo album) for the user to select media content (also referred to as third-party media content) to replace the subject.

[0055] After the user uploads media content 310, they can directly perform the subject replacement operation, as indicated by input box 312. It should be understood that the uploaded media content 310 is the user input corresponding to the replacement operation. Although the user input shown here is in the form of an image, the user input could also be video or audio, etc.

[0056] In some embodiments, the system can pre-set default prompts. For example, default prompts may include "Refer to the effect of the reference work to generate a similar image" or other prompts indicating the display effect of the reference media content to generate related media content. After the user uploads media content 310, the system can generate media content 314 based on media content 302, media content 310, and the default prompts. Therefore, users do not need to write their own prompts to generate similar content to complete the creation of the corresponding similar content.

[0057] In some embodiments, users can also input text content through input box 312 to further adjust the subsequent generation process. That is, user input can include both images and text. For example, users can input "enlarge the water ripple effect", "change to blue background" or other prompts to make the subsequently generated related media content display the corresponding effect.

[0058] Therefore, when Message 306 prompts the user to perform a replacement operation, the user can either upload only the media content to be replaced, or enter the corresponding prompt while uploading the media content. This allows for the preservation of at least some of the display effects of the reference media content, generating related media content with a similar presentation. Furthermore, the subsequent generation process can be adjusted based on the user's input prompts, thereby improving the flexibility of media content generation and the user's interactive experience.

[0059] After obtaining user input, corresponding media content 314 can be generated based on media content 302 and user input. Media content 314 can be, for example, the same media content created by the user. In some embodiments, media content 302, user-uploaded media content 310, and / or prompts entered by the user through input box 312 can be input into the large model to generate corresponding media content. For example, in a subject replacement scenario, at least some of the display effects in media content 302 can be retained, and the subject content in media content 302 can be replaced with the subject content corresponding to media content 310 to generate media content 314. For example, the subject content in media content 302 can be a perfume bottle, but after the replacement operation, the subject content in media content 314 is replaced with headphones, while retaining water ripple effects, reflection effects, or other display effects.

[0060] Therefore, by generating media content 314 based on media content 302 and user input, the generated media content 314 can maintain a high degree of similarity to media content 302, thus better meeting the user's expectations when creating similar content. In the subject replacement scenario, while retaining the water ripple effect and reflection effect in media content 302, the subject content can be replaced with the subject content that the user wants to display, thereby generating similar media content with a similar presentation.

[0061] In contrast, related technologies typically process user-uploaded media content 310 based on prompts corresponding to media content 302. However, due to the inherent instability of generative models, when generating media content solely based on prompts, the generated media content may differ from the reference media content, making it difficult to consistently generate media content that meets user expectations. In some scenarios, users may need to adjust prompts multiple times or repeatedly perform the generation operation to obtain satisfactory results, which reduces user interaction experience and media content generation efficiency.

[0062] In some embodiments, multiple related media content can be displayed simultaneously for the user to choose from. For example, such as Figure 3As shown, media content 314, media content 316, or other related media content can be displayed simultaneously. Different related media content can differ in main layout, background effects, color effects, composition, display style, or other display effects, providing users with more choices.

[0063] In some embodiments, the user can also continue to enter prompts through input box 312 to further adjust the media content 314. For example, the user can enter prompts for adjusting background effects, subject effects, color effects, lighting effects, or other display effects, thereby generating the corresponding adjusted media content.

[0064] It should be understood that, despite Figure 3 While images are used as examples of media content, the embodiments of this disclosure can also be applied to video, audio, or other media content. For instance, in a video scenario, when browsing a reference video, a user may wish to retain the visual style, camera movement, background effects, motion effects, or other display effects of the reference video while replacing the main content. In some embodiments, a user can upload video content, image content, or other media content to replace the main content, and generate a corresponding identical video based on the reference video, the user-uploaded media content, and / or input prompts.

[0065] In some embodiments, the embodiments of this disclosure can also be applied to audio scenarios. For example, when playing reference audio, a user may wish to retain the melody style, rhythm effect, singing style, or other audio effects in the reference audio while replacing the sound content in the reference audio. In some embodiments, a user can upload their own sound content and generate a corresponding similar audio based on the reference audio, the uploaded sound content, and / or the input prompts. For example, at least some of the audio effects in the reference audio can be retained while replacing the sound content in the reference audio with the sound content uploaded by the user.

[0066] Figure 4A schematic diagram illustrating a modification process 400 according to some embodiments of this disclosure is shown. When browsing reference media content, a user may wish to retain the visual style, background effects, composition, dynamic effects, or other display effects of the reference media content, while modifying the text content in the reference media content to generate corresponding similar content. For example, a user may wish to reference the visual effects of holiday posters, promotional images, advertising images, or other media content and modify the text content therein to generate corresponding media content. In some embodiments of this disclosure, when the system determines that a user may wish to perform a text modification operation, it can display corresponding information to the user to instruct the user to perform the text modification operation. After the user inputs the text content to be modified, the text content in the reference media content can be replaced with the text content input by the user, thereby generating corresponding related media content.

[0067] like Figure 4 As shown, when browsing media content 402, a user may find its visual effects appealing and wish to use it as a template or reference to create similar content. For example, media content 402 could be a holiday promotional image, including fireworks effects, stage effects, text effects, or other display effects. The user can then manipulate control 404 to trigger the corresponding media content processing flow.

[0068] After a user creates a 404 error with the same control, a 406 error can be displayed to prompt the user to perform text editing. For example, ... Figure 4 As shown, information 406 may include "send directly or describe the modified content" or other prompts. This allows the user to clearly understand the subsequent action, thus avoiding the cost of understanding prompts. In some embodiments, the computing device may display an input box 410 and an input keyboard 408 for the user to input text content for modification (also referred to as fourth media content). It should be understood that text also falls under the category of media content.

[0069] For example, a user can enter "replace text with Happy Mid-Autumn Festival" or other text content through input box 410. In some embodiments, the user can also enter prompts to adjust the display effects. For example, the user can enter "add fireworks effects," "change text color," "adjust background effects," or other prompts to adjust the subsequent generation process.

[0070] After obtaining user input, corresponding media content 412 can be generated based on media content 402 and user input. Media content 412 can be, for example, the same media content created by the user. In some embodiments, media content 402 and prompts entered by the user through input box 410 can be input into a large model to generate corresponding media content. For example, in a text modification scenario, the original text content in media content 402 can be replaced with text content entered by the user to generate media content 412. For example, the text content in media content 402 can be "Happy New Year", while after performing a text modification operation, the text content in media content 412 can be replaced with "Happy Mid-Autumn Festival", while retaining fireworks effects, stage effects, or other display effects.

[0071] In some embodiments, multiple related media content can be displayed simultaneously for the user to choose from. For example, media content 412, media content 414, or other related media content can be displayed simultaneously. Different related media content can differ in text effects, color effects, background effects, composition methods, or other display effects, thereby improving the flexibility of media content generation.

[0072] In some embodiments, the user can continue to enter prompts through the input box 410 to further adjust the media content 412. For example, the user can enter prompts to adjust text effects, background effects, color effects, lighting effects, or other display effects, thereby generating the corresponding adjusted media content.

[0073] In some embodiments, an addition operation can be performed. For example, when browsing reference media content, a user may want to retain at least part of the display effect of the reference media content while adding new content to the reference media content to generate corresponding similar media content. For example, in some scenarios, the reference media content can be a group photo of celebrities, and users can upload their own pictures to add their own main content to the celebrity group photo, thereby generating a similar image that includes the user's main content.

[0074] In some embodiments, the add operation can be applied to text content. For example, the text content in the reference media content can be "Happy New Year," and the user can enter "Add 2026" or other prompts to add the corresponding text content to the reference media content, thereby generating related media content that includes the text content "Happy New Year 2026." In some embodiments, decorative elements, dynamic effects, background elements, audio content, or other content can be added to the reference media content.

[0075] Figure 5A flowchart of a process 500 for determining an operation type according to some embodiments of the present disclosure is shown. As previously described, after a user operates a control for creating similar content, the system can display corresponding information to instruct the user to perform the corresponding operation. For example, when a user browses reference media content including a clearly defined subject, the system predicts that the user is more likely to want to replace the subject content in the reference media content, and therefore displays information to instruct the user to perform a replacement operation. When a user browses reference media content including text content, the system predicts that the user is more likely to want to modify the text content in the reference media content, and therefore displays information to instruct the user to perform a modification operation. The following is in conjunction with... Figure 5 This describes the process of determining the type of operation.

[0076] like Figure 5 As shown, in step 502, reference media content can be obtained. For example, the reference media content can be the image, video, audio, or other media content currently being viewed by the user. For example, the reference media content can include main content, text content, background content, dynamic effects, or other display content.

[0077] In step 504, reference media content can be input into the model. In some embodiments, the model can be a large language model, a multimodal model, an image recognition model, or other machine learning models. The model can analyze the displayed content, main content, text content, composition, style effects, or other content features in the reference media content.

[0078] In step 506, the model can determine the type of operation the user is likely to perform. For example, if the reference media content includes a clearly defined subject object, the model can determine that the user is more likely to perform a subject replacement operation. If the reference media content includes text content, the model can determine that the user is more likely to perform a text modification operation. If the reference media content has expandable or decorative areas, the model can determine that the user is more likely to perform an element addition operation. In some embodiments, corresponding information can be displayed based on the determined operation to instruct the user to perform the corresponding operation.

[0079] In some embodiments, when determining the types of operations a user may perform, the content processing dimensions of the reference media content can be determined first, and then the corresponding operation type can be determined based on the content processing dimensions. The content processing dimension can be understood as the aspects of the reference media content that can be processed when creating a similar work based on the reference media content. For example, the content processing dimension may include the main body processing dimension, text processing dimension, content addition dimension, or other processing dimensions.

[0080] For example, the subject processing dimension can indicate that the main content in the reference media content is suitable for replacement, adjustment, or processing. When the reference media content includes a clearly defined subject object, the corresponding content processing dimension can be identified as the subject processing dimension, and the corresponding operation type can be further identified as a replacement operation. The text processing dimension can indicate that the text content in the reference media content is suitable for modification, replacement, or processing. When the reference media content includes text content, the corresponding content processing dimension can be identified as the text processing dimension, and the corresponding operation type can be further identified as a modification operation. The content addition dimension can indicate that the reference media content is suitable for adding new main content, text content, decorative elements, background elements, dynamic effects, or other content. When the reference media content has expandable areas or is suitable for supplementing content, the corresponding content processing dimension can be identified as the content addition dimension, and the corresponding operation type can be further identified as an addition operation.

[0081] In some embodiments, process 500 can be executed after the user operates the control used to create the same content. For example, after the user triggers the corresponding media content processing flow, the system can input reference media content into the model to determine the type of operation the user may perform and display information corresponding to that operation type. In some embodiments, process 500 can be executed in advance before the user operates the control used to create the same content. For example, the system can predict the operation type corresponding to the reference media content in advance and store the prediction results. In this case, after the user operates the control used to create the same content, information corresponding to the predicted operation type can be directly displayed, thereby improving user interaction efficiency.

[0082] In some embodiments, after determining that the user's likely action is a replacement operation, an input interface corresponding to the replacement operation can be displayed. For example, the system can directly invoke and display a user album control (also known as a second control) for the user to select media content to replace the subject. For example, combined with Figure 3 As shown, after determining that the user may perform a replacement operation, after the user performs the same control 304, the media content selection interface 308 (i.e., the user's album) can be displayed directly, thereby improving the efficiency of the user performing the main body replacement operation.

[0083] In some embodiments, after determining that the user's likely operation is a modification operation, an input interface corresponding to the modification operation can be displayed. For example, the system can directly display an input keyboard (also known as a third control) for the user to input text content for modification. For example, combined with... Figure 4 As shown, after determining that the user may perform a modification operation, and after the user's operation results in a 404 error for the same control, the keyboard 408 can be displayed directly, thereby improving the efficiency of the user's text modification operation.

[0084] Figure 6 A flowchart of a media content generation process 600 according to some embodiments of the present disclosure is shown. Figure 6 As shown, in step 602, reference media content can be obtained. For example, the reference media content may include images, videos, audio, or other media content currently being viewed by the user.

[0085] In step 604, user input may be obtained. In some embodiments, user input may include media content uploaded by the user. For example, user input may include images, videos, audio, or other media content. In some embodiments, user input may also include text content. For example, the user may enter prompts describing generation requirements, adjusting display effects, or modifying content. In some embodiments, user input may include both media content and text content.

[0086] In step 606, the corresponding media content can be determined based on the reference media content and user input. In some embodiments, a large model can be used to process the reference media content and user input to generate the corresponding media content. It should be understood that in the embodiments of this disclosure, the corresponding media content can be generated directly based on the reference media content itself and user input, without relying on the prompts corresponding to the reference media content. Since the generation process references at least part of the display effects in the reference media content, the similarity between the generated media content and the reference media content can be improved, making the generated media content more in line with the user's expectations when creating similar content.

[0087] In some embodiments, in addition to generating media content based on reference media content and user input, prompts corresponding to the reference media content can be input into the large model as auxiliary contextual information. This provides richer contextual information for the media content generation process, thereby improving the media content generation effect. For example, prompts can be used to describe the style effects, dynamic effects, lighting effects, or other generation requirements corresponding to the reference media content. It should be understood that in this case, the prompts can serve as contextual information in the background generation process and do not need to be directly displayed to the user. Therefore, the user does not need to understand the content of the prompts to complete the creation of the corresponding content.

[0088] Figure 7A block diagram of a media content processing apparatus 700 according to some embodiments of the present disclosure is shown. The apparatus 700 includes a first display module 702 configured to display first media content and a first control configured to trigger the generation of related media content based on the first media content. The apparatus 700 also includes a second display module 704 configured to display first information in response to a first operation on the first control, the first information indicating a second operation on the first media content. Furthermore, the apparatus 700 includes a third display module 706 configured to display second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0089] Therefore, the apparatus 700 according to the embodiments of this disclosure, by generating relevant media content based on reference media content and user input, can improve the similarity between the relevant media content and the reference media content, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to the user instead of showing prompts, the operational threshold for the user can be lowered, improving the interaction efficiency and user experience when creating media content.

[0090] In some embodiments of this disclosure, the second display module 704 includes: a first determining module configured to determine an operation type corresponding to a second operation based on first media content; and a fourth display module configured to display first information based on the operation type.

[0091] In some embodiments of this disclosure, the first determining module includes: a second determining module configured to determine a content processing dimension for the first media content based on the first media content, wherein the content processing dimension includes at least one of a main processing dimension, a text processing dimension, and a content addition dimension for the first media content; and a third determining module configured to determine an operation type corresponding to the second operation based on the content processing dimension.

[0092] In some embodiments of this disclosure, the operation type includes at least one of a replacement operation, a modification operation, and an addition operation.

[0093] In some embodiments of this disclosure, the third display module 706 includes: a first acquisition module configured to acquire user input in response to an operation of type replacement, the user input including third media content for replacement; and a fifth display module configured to display second media content, the second media content being generated by replacing at least a portion of the content in the first media content with the content corresponding to the third media content.

[0094] In some embodiments of this disclosure, the third display module 706 includes: a second acquisition module configured to acquire user input in response to an operation of type modification, the user input including fourth media content for modification; and a sixth display module configured to display second media content generated by modifying at least a portion of the first media content based on the fourth media content.

[0095] In some embodiments of this disclosure, the third display module 706 includes: a third acquisition module configured to acquire user input in response to an operation of type add, the user input including fifth media content for addition; and a seventh display module configured to display second media content, the second media content being generated by adding content corresponding to the fifth media content to the first media content.

[0096] In some embodiments of this disclosure, the device 700 further includes an eighth display module configured to display an input interface corresponding to the operation type based on the operation type of the second operation.

[0097] In some embodiments of this disclosure, the device 700 further includes a ninth display module configured to display a second control in response to an operation of type replacement, the second control being configured to select media content.

[0098] In some embodiments of this disclosure, the device 700 further includes a tenth display module configured to display a third control in response to an operation of type modification, the third control being configured to input text.

[0099] In some embodiments of this disclosure, the user input corresponding to the second operation includes at least one of images, videos, text, and audio.

[0100] In some embodiments of this disclosure, the apparatus 700 further includes: a fourth acquisition module configured to acquire the user input in response to the second operation; and a first generation module configured to generate the second media content based on the user input and the first media content.

[0101] In some embodiments of this disclosure, an electronic device is provided, including: at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform an action when executed by the at least one processing unit, the action including: displaying first media content and a first control, the first control being configured to trigger the generation of related media content with reference to the first media content; displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; and displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0102] Therefore, by generating relevant media content based on reference media content and user input, this electronic device can improve the similarity between the relevant media content and the reference media content, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to the user instead of showing prompts, the operational threshold for users can be lowered, improving the interactive efficiency and user experience when creating media content.

[0103] In some embodiments of this disclosure, a computer-readable storage medium is provided having machine-executable instructions stored thereon, which, when executed by a device, cause the device to perform actions including: displaying first media content and a first control, the first control being configured to trigger the generation of related media content with reference to the first media content; displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; and displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0104] Therefore, this computer-readable storage medium, by generating relevant media content based on reference media content and user input, can improve the similarity between the relevant media content and the reference media content, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to the user instead of showing prompts, the operational threshold for users can be lowered, improving the interactive efficiency and user experience when creating media content.

[0105] In some embodiments of this disclosure, a computer program product is provided, including computer-executable instructions, wherein when executed by a processor, the computer-executable instructions cause the processor to perform actions, the actions including: displaying first media content and a first control, the first control being configured to trigger the generation of related media content with reference to the first media content; displaying first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; and displaying second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

[0106] Therefore, by generating relevant media content based on reference media content and user input, this computer program product can improve the similarity between the relevant media content and the reference media content, making the generated relevant media content more in line with the user's expectations when creating media content. Furthermore, by displaying corresponding operation information to the user instead of showing prompts, it can lower the operational threshold for users, improve the interactive efficiency and user experience when creating media content.

[0107] Figure 8 A schematic block diagram of an example device 800 suitable for implementing various embodiments of the present disclosure is illustrated. As shown, device 800 includes a processing unit 801, which can perform various appropriate actions and processes according to computer program instructions stored in random access memory (RAM) 803 and / or read-only memory (ROM) 802 or loaded from storage unit 808 into RAM 803 and / or ROM 802. Various programs and data required for the operation of device 800 may also be stored in RAM 803 and / or ROM 802. Processing unit 801 and RAM 803 and / or ROM 802 are interconnected to each other via bus 804. Input / output (I / O) interface 805 is also connected to bus 804. Although not shown in the figure... Figure 8 As shown, device 800 may also include a coprocessor.

[0108] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0109] Processing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processing unit 801 include, but are not limited to, CPUs, GPUs, various special-purpose AI computing chips, various processing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processing unit 801 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via RAM and / or ROM and / or communication unit 809. When the computer program is loaded into RAM and / or ROM and executed by processing unit 801, one or more actions of the various methods and processes described above may be performed. Alternatively, in other embodiments, processing unit 801 may be configured to perform one or more actions of the methods and processes described above by any other suitable means (e.g., by means of firmware).

[0110] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0111] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), and any suitable combination thereof. The computer-readable storage medium as used herein is not to be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0113] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0114] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0115] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0116] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0118] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or pervasive.

[0119] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this disclosure may be implemented individually or in combination with at least one other aspect. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0120] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A media content processing method, comprising: Display first media content and a first control, wherein the first control is configured to trigger the generation of related media content based on the first media content; In response to a first operation on the first control, first information is displayed, the first information indicating a second operation on the first media content; as well as In response to the second operation, second media content is displayed, which is generated based on the first media content and user input corresponding to the second operation.

2. The method of claim 1, wherein displaying the first information comprises: Based on the content of the first media, determine the operation type corresponding to the second operation; as well as Based on the operation type, the first information is displayed.

3. The method according to claim 2, wherein determining the operation type corresponding to the second operation includes: Based on the first media content, a content processing dimension for the first media content is determined, wherein the content processing dimension includes at least one of the following: a main body processing dimension, a text processing dimension, and a content addition dimension; and Based on the content processing dimension, determine the operation type corresponding to the second operation.

4. The method according to claim 2, wherein the operation type includes at least one of a replacement operation, a modification operation, and an addition operation.

5. The method of claim 4, wherein displaying the second media content comprises: In response to the operation type being the replacement operation, the user input is obtained, the user input including third media content for replacement; as well as The second media content is displayed, which is generated by replacing at least a portion of the content in the first media content with the content corresponding to the third media content.

6. The method of claim 4, wherein displaying the second media content comprises: In response to the operation type being the modification operation, the user input is obtained, the user input including fourth media content for modification; as well as The second media content is displayed, which is generated by modifying at least a portion of the first media content based on the fourth media content.

7. The method of claim 4, wherein displaying the second media content comprises: In response to the operation type being the add operation, the user input is obtained, the user input including fifth media content for adding; as well as The second media content is displayed, which is generated by adding the content corresponding to the fifth media content to the first media content.

8. The method according to claim 1, further comprising: Based on the operation type of the second operation, an input interface corresponding to the operation type is displayed.

9. The method according to claim 8, further comprising: In response to the operation type being a replacement operation, a second control is displayed, which is configured to select media content.

10. The method of claim 8, further comprising: In response to the operation type being a modification operation, a third control is displayed, which is configured to input text.

11. The method of claim 1, further comprising: In response to the second operation, the user input is obtained; as well as The second media content is generated based on the user input and the first media content.

12. The method according to any one of claims 1 to 11, wherein the user input corresponding to the second operation includes at least one of images, videos, text, and audio.

13. A media content processing apparatus, comprising: The first display module is configured to display first media content and a first control, wherein the first control is configured to trigger the generation of related media content based on the first media content; The second display module is configured to display first information in response to a first operation on the first control, the first information indicating a second operation on the first media content; as well as A third display module is configured to display second media content in response to the second operation, the second media content being generated based on the first media content and user input corresponding to the second operation.

14. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 12.

15. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 12.

Images