Media data generation method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a media data generation method and device, equipment, a medium and a product. The method comprises the steps of generating first interaction prompt information in response to a first operation; according to the first interaction prompt information, generating first media data and displaying the first media data; and obtaining second interaction prompt information, and generating and displaying second media data according to the second interaction prompt information and the first media data. According to the technical scheme provided by the embodiment of the invention, the effects of reducing the complexity of media data editing and improving the efficiency and convenience of media data generation are achieved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer processing technology, and in particular, to a method, apparatus, device, medium, and product for generating media data. Background Art

[0002] With the development of artificial intelligence technology, more and more users use related media data production software to produce media data.

[0003] The main problems with producing media data based on existing media data production software are: first, it is usually necessary to switch back and forth between platforms, software or tools for producing audio, video, images, text and other functions to generate media data that meets the needs of media data production users. In other words, media data production users need to spend a lot of time to produce different data content and then edit it, which leads to high operational complexity and time cost. Second, media data production software has high professional requirements for media data production users and has poor universality. Summary of the invention

[0004] The embodiments of the present disclosure provide a method, apparatus, device, medium, and product for generating media data, so as to reduce the complexity of media data editing, thereby improving the efficiency and convenience of media data generation.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating media data, the method comprising:

[0006] In response to the first operation, generating first interaction prompt information;

[0007] Generate and display first media data according to the first interaction prompt information;

[0008] Second interaction prompt information is obtained, and second media data is generated and displayed according to the second interaction prompt information and the first media data.

[0009] In a second aspect, an embodiment of the present disclosure further provides a media data generating device, the device comprising:

[0010] A first interaction prompt information generating module, used for generating first interaction prompt information in response to a first operation;

[0011] A first media data generating module, configured to generate and display first media data according to the first interaction prompt information;

[0012] The second media data generating module is used to obtain second interaction prompt information, and generate and display second media data according to the second interaction prompt information and the first media data.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the media data generation method as described in any one of the embodiments of the present disclosure.

[0017] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the media data generation method as described in any one of the embodiments of the present disclosure.

[0018] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the media data generation method as described in any one of the embodiments of the present disclosure.

[0019] The technical solution provided by the embodiment of the present disclosure obtains the first media data by processing the generated first interactive prompt information. On the basis of the first media data, the second interactive prompt information can be combined to generate the second media data associated with the first media data and the second interactive prompt information, so that only the relevant prompt information needs to be input to generate the second media data in combination with the previously generated content, thereby improving the efficiency and continuity of media data generation. Furthermore, the second media data can be a combination of one or more of audio, video, and text, that is, the media data can be generated based on the same media data production software or tool, thereby improving the universal applicability of media data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 A flowchart of a method for generating media data provided by an embodiment of the present disclosure;

[0022] Figure 2 A schematic diagram for representing a display interface provided by an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram for representing a display interface provided by an embodiment of the present disclosure;

[0024] Figure 4 A flowchart of a method for generating media data provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram for representing and displaying interactive guidance types provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram for representing and displaying interactive guidance types provided by an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram for representing and displaying a first category identifier provided by an embodiment of the present disclosure;

[0028] Figure 8 A flowchart of a method for generating media data provided by an embodiment of the present disclosure;

[0029] Fig. 9 A schematic diagram for representing a display interface provided by an embodiment of the present disclosure;

[0030] Fig.10 A schematic diagram for representing a display interface provided by an embodiment of the present disclosure;

[0031] Fig.11 A flowchart of a method for generating media data provided by an embodiment of the present disclosure;

[0032] Fig.12 A schematic diagram for representing generation progress provided by an embodiment of the present disclosure;

[0033] Fig.13 A schematic diagram for representing generation progress provided by an embodiment of the present disclosure;

[0034] Fig.14 A schematic diagram for representing generation progress provided by an embodiment of the present disclosure;

[0035] Fig.15 A schematic diagram for representing a display interface provided by an embodiment of the present disclosure;

[0036] Fig.16 A schematic diagram for representing an edit item provided by an embodiment of the present disclosure;

[0037] Fig.17 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0038] Fig.18 A flowchart of a method for generating media data provided by an embodiment of the present disclosure;

[0039] Fig.19 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0040] Fig. 20 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0041] Fig.21 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0042] Fig. 22 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0043] Fig.23 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0044] Fig.24 A schematic diagram for representing media data provided by an embodiment of the present disclosure;

[0045] Fig.25 A schematic diagram for representing a video clip provided by an embodiment of the present disclosure;

[0046] Fig.26 A flowchart of a method for generating media data provided by an embodiment of the present disclosure;

[0047] Fig. 27 A schematic diagram for representing a display screen provided by an embodiment of the present disclosure;

[0048] Fig.28 A schematic diagram of the structure of a media data generating device provided by an embodiment of the present disclosure;

[0049] Fig.29 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0051] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0052] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0053] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0054] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0055] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0056] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0057] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0058] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0059] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0060] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0061] Before introducing the technical solution provided by the embodiment of the present disclosure, an exemplary description of the application scenario can be given. The technical solution provided by the embodiment of the present disclosure can be applied to any platform, software or tool that can produce media data. That is, a corresponding application program or media data production platform can be developed based on the embodiment of the present disclosure. Alternatively, the solution provided by the embodiment of the present disclosure can be integrated into any existing application software that can support media data production to enrich the functions of the relevant application software and the versatility of the application software.

[0062] In actual applications, if a media data production user needs to produce media data, the application software, web page or tool provided by the embodiment of the present disclosure can be triggered. After detecting the triggering of the relevant software, web page or tool, a display interface corresponding to the embodiment of the present disclosure can be displayed to generate corresponding media data based on the editing operation in the display interface.

[0063] It should be noted that the technical solution provided in the embodiments of the present disclosure can be used on any web terminal, mobile terminal or tablet terminal that can support media data production.

[0064] Figure 1 A flow chart of a media data generation method provided in an embodiment of the present disclosure is applicable to any scenario where media data needs to be produced. The method can be executed by a media data generation device, which can be executed by a server, or by a client, or by a client and a server in cooperation.

[0065] like Figure 1 As shown, the method of this embodiment may specifically include:

[0066] S110: In response to a first operation, generate first interaction prompt information.

[0067] Among them, the first operation is an operation of generating the first interactive prompt information. Optionally, the first operation is an operation of clicking to select the prompt content corresponding to the first interactive prompt information. The first interactive prompt information is mainly used to determine the data content or content type of the first media data. The first interactive prompt information is related to the media data that the media production user wants to obtain. In this embodiment, the first interactive prompt information can be expressed by text, web page, text, audio, video, animated image, image, script or other types of information. The prompt content of the first interactive prompt information may include but is not limited to the theme, style, etc. of the media data to be produced. Specifically, the media data production user can edit the prompt information of the media data to be obtained in the display interface. After the prompt information is edited, he can click Finish to trigger the first operation. At this time, the edited prompt information can be obtained, and this prompt information can be used as the first interactive prompt information.

[0068] For example, if a media production user needs a promotional information about "Apple", he can edit the media data generation instruction in the display interface, that is, edit the text content. At this time, the edited text content can be: "I want to make an image about Apple", and accordingly, the first interactive prompt information is: "I want to make an image about Apple". For example, the schematic diagram of the display interface can be seen in Figure 2 , you can enter the text information of "a multimedia video of an apple tree" in the text input box. The schematic diagram can be seen in Figure 3 .

[0069] In this embodiment, in response to a first operation, a first interaction prompt message is generated, including: in response to a first operation of inputting text information in a text input box of a display interface, obtaining the text information in the text input box; in response to an event in which editing of the text information is completed, generating a first interaction prompt message based on the text information.

[0070] It can be understood that the display interface may include a text input area, and the text input area is referred to as a text input box. The text input box is mainly used to input related text, pictures, links, web pages and / or audio, etc. Correspondingly, the first operation may be an operation of inputting text. In the process of inputting text, text information can be obtained in real time. After the text information is edited, you can click to confirm, and the text information in the text input box when you click to confirm is used as the first interactive prompt information. That is, the media data production user can enter text information in the text input box in the display interface by keyboard input, voice input or text import. When the edit completion control is clicked, or the stop input time after entering the text information reaches a preset time, or when the user performs other operations indicating the end of editing, the edit completion event is triggered.

[0071] S120: Generate and display first media data according to the first interaction prompt information.

[0072] The first media data is media data generated based on the first interactive prompt information. The media data may be one or a combination of images, animated images, videos, and audio.

[0073] In this embodiment, the first interaction prompt information can be processed based on a pre-trained machine learning model to generate first media data that matches the first interaction prompt information. Alternatively, the first media data that matches the first interaction prompt information can be found from a resource library through a text matching algorithm.

[0074] In order to achieve the effect of visualizing the first media data and facilitate the media production user to promptly confirm whether the generated first media data meets the requirements, the first media data can be displayed. Optionally, the display interface includes a preview area, and the first media data can be displayed in the preview area.

[0075] S130: Acquire second interaction prompt information, and generate and display second media data according to the second interaction prompt information and the first media data.

[0076] Among them, the second interactive prompt information is mainly a supplement to the first interactive prompt information, or a further limitation of the first interactive prompt information. At this time, the limitation can be understood as a further concrete description of the content in the first interactive prompt information. The content type of the second interactive prompt information can be the same as that of the first interactive prompt information, which will not be repeated here. The second media data is the media data generated after processing the first media data and the second interactive prompt information. Compared with the first media data, the second media data is data that enriches the content of the first media data.

[0077] In this embodiment, the media data production user can make further adjustments based on the effect presented by the first media data, and the adjustment can be achieved through the second interactive prompt information. Optionally, you can continue to input information such as text, images, or audio in the display interface. After obtaining the second interactive prompt information, the adjustment requirements of the media data production user on the first media data can be analyzed, so as to process the first media data and generate processed second media data. The second media data can be displayed in the preview area together with the first media data, so that the media data production user can intuitively know the difference between the first media data and the second media data.

[0078] It should be noted that after determining the second media data, if there is still a need for adjustment, the solution provided by the embodiment of the present disclosure can continue to be executed. At this time, the second media data is the first media data, and the second interaction prompt information is the first interaction prompt information. The re-acquired interaction prompt information is used as the second interaction prompt information, and the step of generating the second media data is continued. The execution can be repeated multiple times to obtain media data that meets the needs of media data production users. The above method can generate richer media data under the same production environment, achieving the effect of smooth media data generation and rich content of media data.

[0079] The technical solution provided by the embodiment of the present disclosure obtains the first media data by processing the generated first interactive prompt information. On the basis of the first media data, the second interactive prompt information can be combined to generate the second media data associated with the first media data and the second interactive prompt information, so that only the relevant prompt information needs to be input to generate the second media data in combination with the previously generated content, thereby improving the efficiency and continuity of media data generation. Furthermore, the second media data can be a combination of one or more of audio, video, and text, that is, the media data can be generated based on the same media data production software or tool, thereby improving the universal applicability of media data.

[0080] Figure 4 A flowchart of a method for generating media data provided by an embodiment of the present disclosure. Based on the aforementioned embodiment, the display interface may also display interaction guidance information to prompt the user how to generate interaction prompt information and corresponding media data based on the interaction guidance information. The specific implementation method may refer to the detailed description of the embodiment of the present disclosure. The technical features that are the same or similar to the aforementioned embodiment are not described in detail here.

[0081] like Figure 4 As shown, the method of this embodiment may specifically include:

[0082] S210: Displaying interaction guidance information of at least one interaction guidance type in a first preset display format in a display interface.

[0083] Among them, the interaction guidance information is used to guide how to generate media data. The first preset display form is a pre-set presentation form of the interaction guidance information in the initial state. The interaction guidance information can be presented in different forms, optionally, text form, video form, image form, etc., and different display forms are used as interaction types. The number of at least one interaction guidance type can be one or more. If the interaction guidance type includes one, it can be a video guidance type or an image guidance type. If the interaction guidance type includes multiple types, it can be at least two of a text guidance type, a video guidance type, and an image guidance type.

[0084] In this embodiment, the content of the interactive guidance information includes but is not limited to text descriptions, reference requirement description information, reference media data templates, image examples, video tutorials, animation demonstrations, and links to guide media data production users on how to produce media data.

[0085] It should be noted that different interaction guidance types correspond to different interaction guidance information. Optionally, the interaction guidance information corresponding to the text guidance type may be instructing the media data production user in the form of text. The interaction guidance information corresponding to the video guidance type may be a pre-made video template.

[0086] In this embodiment, the first preset display form may be a folded display form of at least one interaction guide type. Optionally, it may be displayed in a fan-shaped form, and interaction guide information of different interaction guide types may be distinguished in different display forms. The schematic diagram may be referred to in Figure 5 .

[0087] At this time, the media data production user can be prompted how to produce media data through interaction guidance information of various interaction guidance types, thereby achieving richness and comprehensiveness of the guidance content, thereby facilitating the media data production user to efficiently use the solution provided by the embodiment of the present invention to produce media data.

[0088] S220: In response to a triggering operation on the interaction guidance information, display interaction guidance information of at least one interaction guidance type in a second preset display form in the display interface.

[0089] The second preset display form is a display form different from the first preset display form. For example, the second preset display form may be that the interaction guide information of at least one interaction guide type is displayed in a tiled display form.

[0090] In this embodiment, when the user needs to view specific interaction guidance information of multiple interaction guidance types, the interaction guidance information displayed in the first preset display form can be triggered. In response to the triggering operation of the interaction guidance information in the first preset display form, the interaction guidance information of at least one interaction guidance type is displayed in the display interface in the second preset display form (tiled display form). In this way, the interaction guidance information of different interaction guidance types can be displayed in a clear and intuitive manner in the display interface.

[0091] For example, when displaying the display interface, in order to facilitate the media data production user to determine how to generate media data, an intelligent agent can be displayed, and a sample description of the intelligent agent display instructions can be implemented to realize interesting visual interactive animation to convey the function usage instructions. Figure 5 When the media production user clicks on the interaction guidance information in the first preset display form, the interaction guidance information can be displayed in a preset area in the display interface in a tiled display form according to the interaction guidance type. The schematic diagram can be seen in Figure 6 .

[0092] In actual applications, in order to further facilitate the media data production user, at least one first category identifier may be displayed in the display interface, wherein the category information of the second media data is related to the first category identifier.

[0093] The category identifier is used to characterize the subject type of the media data content. For example, the category identifier may include, but is not limited to, promotion, advertising, culture and travel, comedy, food, travel, sports, film and television, music, pets, beauty, education, festivals, and other categories. The category identifier may be expressed through the category name and category image information.

[0094] In this embodiment, a plurality of template cards corresponding to the first category identifiers may be pre-made. The template card is mainly used to present text and images corresponding to the category identifier. When displaying the category identifier on the display interface, how to display it may be determined according to the number of the first category identifiers and the preset display quantity threshold. Optionally, when the number of the first category identifiers is greater than the preset display quantity threshold, the use attributes of different first category identifiers may be evaluated according to the attribute information of the media data production user (such as historical production media information, browsing information, click information, etc.) and / or the historical use attributes of the template cards of different first category identifiers (such as usage frequency, user use feedback information, etc.), and the display order of different first category identifiers may be determined according to the use attributes. The first category identifier displayed in the display interface is determined according to the display order and the preset display quantity threshold; other first category identifiers that are not displayed may be folded or hidden for display. Alternatively, the first category identifier displayed in the display interface may be dynamically updated based on the refresh operation of the media production user on the first category identifier; different first category identifiers may also be displayed by sliding in the display interface through the user's sliding operation. In actual applications, if the media data producing user has a trigger selection for the first category identifier, then the triggered first category identifier is consistent with the category information of the second media data to be generated.

[0095] In practical applications, the first category identifier may also include a play control. When a user triggers a play control of a first category identifier, the template media data associated with the first category identifier selected by the trigger may be displayed. At this time, the template media data is mainly the media data corresponding to the triggered first category identifier.

[0096] For example, see Figure 7 , the first category identifiers displayed are "promotion, advertising, culture and travel, film and television, education, and music". When the travel category identifier is detected to be triggered, it means that the second media data to be generated is related to tourism elements. The advantage of displaying the first category identifier is that it is convenient for media data production users to view template media data with different first category identifiers according to their interests and needs, and can choose the category identifier used to generate the second media data to meet the user's production needs.

[0097] The technical solution provided by the disclosed embodiment displays interaction guidance information of at least one interaction guidance type in a first preset display form in a display interface, and supports media data production users to trigger operations on the interaction guidance information, and displays interaction guidance information of at least one interaction guidance type in a second preset display form in the display interface, so that media data production users can usually view the interaction guidance information to quickly understand how to generate the media data they want, and guide media data production users to complete specific operations.

[0098] Figure 8 A flowchart of a method for generating media data provided by an embodiment of the present disclosure. The technical solution of this embodiment can further refine the generation of the first interactive prompt information based on the above embodiment, and its specific implementation method can refer to the detailed description of the embodiment of the present disclosure. Among them, the technical features that are the same or similar to the above embodiment are not repeated here.

[0099] like Figure 8 As shown, the method of this embodiment may specifically include:

[0100] S310: In response to a first operation of inputting text content in a text input box of a display interface, display a first text in a first form in the text input box and display at least one second text at a first associated position of the first text.

[0101] Among them, the first text is a text whose text content meets preset conditions. The first form is a form in which the first text is displayed in a preset color, highlight, bold, and / or a preset background color. At least one second text is related to the first text. The first text can be a first vocabulary, and the second text can be a vocabulary derived from the first vocabulary, or an associated second vocabulary. Optionally, the second text can be a text with similar meaning, similar spelling, part of speech, field, etc. to the first text. The first associated position is a position related to the first text, and optionally, the first associated position can be below or above the first text.

[0102] In this embodiment, when the media data production user enters text content in the text input box, the text content such as words, phrases or short sentences is identified. The identified adjustable text is used as the first text. For example, see Fig. 9 , "apple" in the text content is the first text, and its corresponding second text such as "banana, orange, watermelon" can be displayed below "apple". The advantage of displaying the second text is that the first text that can be replaced can be determined based on the text content, and the second text associated with the first text can be displayed, so that it is convenient for the media data production user to determine the desired first text from the second text, so that the relevant text can be determined without the media data production user's operation, further improving the efficiency of media data editing.

[0103] In this embodiment, the method for determining the second text can be: obtaining text content during the process of entering text content in the text input box; determining the first text by analyzing and processing the text content; determining the second text based on the first text, and displaying at least one second text at the first associated position of the first text.

[0104] Specifically, when a media data production user inputs text content in a text input box, the text content is obtained in real time or at a fixed time. Entity words are obtained by segmenting the text content. Entity words can be texts pointing to specific objects, such as words representing the names of specific things, people, places, projects, etc. The entity word can be used as the first text, and then, based on the first text, at least one second text associated with it can be determined, and at least one second text can be displayed at the first associated position of the first text, so that the media data production user can more easily find other texts or information related to the input content, thereby improving the media data production user experience.

[0105] In this embodiment, the manner of determining at least one second text associated with the first text includes at least one of the following:

[0106] The first method may be: determining at least one second text according to the category attribute of the first text and the preset display quantity. The category attribute may be used to characterize the category corresponding to the object in the first text, for example, if the first text is apple, its category attribute may be fruit; if the first text is seaside, its category attribute may be place. The preset display quantity is the number of second texts displayed on the display interface.

[0107] Specifically, a preset number of texts belonging to the same category attribute as the first text may be determined as the second text; or a preset number of texts under a category attribute similar to the category attribute of the first text may be determined as the second text.

[0108] Another way may be: determining at least one second text according to the category attribute, the preset number, and the call frequency attribute of the first text. The call frequency attribute refers to the number of times the second text is selected or called by the media data production user. For example, the higher the value of the call frequency attribute, the more times it is selected and used by the media data production user.

[0109] Specifically, a preset number of texts with the highest call frequency attribute can be determined from the texts with the same category attribute as the first text as the second text; or a preset number of texts with the highest call frequency attribute can be determined from the texts with category attributes similar to the category attribute of the first text and the texts with the same category attribute as the first text as the second text. The advantage of this setting is that it can accurately determine which second texts are most suitable for display to the media data production user, and give priority to displaying those second texts with higher call frequencies, which meets the expectations and needs of the media data production user.

[0110] Another way may be: determining at least one second text according to the historical replacement text of the first text, the replacement frequency attribute corresponding to the historical replacement text, the preset display quantity and the category attribute. The historical replacement text may refer to the text that selects and replaces the first text after the first text is input.

[0111] The replacement frequency attribute can be used to characterize the number of times the historical replacement text is used to replace other texts. For example, the higher the value of the replacement frequency attribute, the more times the other text is replaced. Specifically, a preset number of texts with a replacement frequency attribute ranked first can be determined from the historical replacement texts belonging to the same category attribute as the first text as the second text, so that the second text displayed to the media data production user better meets the actual needs of the media data production user.

[0112] S320: In response to the event of updating the first text based on the triggered second text, update the text content to generate first interaction prompt information based on the updated text content.

[0113] In this embodiment, when the media data production user triggers the display of the second text, the first text in the text content is replaced with the second text selected by the media data production user, and the text content in the text input box is updated synchronously. At this time, the intention or demand of the media data production user can be analyzed according to the updated text content to generate the first interactive prompt information.

[0114] In order to enable media data production users to produce media data that meets their own personalized needs, in response to a first operation of inputting text content in a text input box on a display page, a category description text of at least one second category identifier associated with the text content can be displayed at a second associated position of the text input box; in response to a triggering operation on the category description text, a third category identifier corresponding to the triggered category description text can be used as prompt content in the first interactive prompt information.

[0115] Among them, at least one second category identifier is related to the text content. The second category identifier can be a category identifier that matches the style characteristics of the text content. The category description text can be a text content used to reflect the category identifier. For example, if the category identifier is marketing, its category description text can be a promotional video. The second associated position can refer to a specific position on the display interface associated with the text input box, which is used to display the category description text related to the text content. The second associated position can be located below, next to, or at other suitable positions in the text input box. The second associated position is different from the first associated position to distinguish the information displayed at the two positions. The category information of the second media data is related to the third category identifier. That is, the third category identifier can be the category identifier used when generating the second media data, which is consistent with the category information corresponding to the generated media data.

[0116] In this embodiment, when the media data production user inputs text content in the text input box of the display interface, one or more category identifiers related to the text content can be found from at least one preset category identifier according to the text content input by the media data production user, and the found category identifier can be used as the second category identifier. And the category description text of the second category identifier is displayed at the second associated position of the text input box.

[0117] When a media data production user triggers a category description text, the second category identifier corresponding to the triggered category description text can be used as the third category identifier of the text content, and when the text content editing is completed, the third category identifier can be used as the prompt content in the first interactive prompt information.

[0118] For example, see Fig.10 The category identification of the text content is the promotion category identification. The category description texts such as "promotional film" of the promotion category identification, "animation" of the animation promotion category identification, and "three-dimensional model diagram" of the 3D category identification can be displayed in the upper left position of the text input box.

[0119] The technical solution provided by the embodiment of the present disclosure can make it easy for the media data production user to understand the core content of the category identifier by displaying the category description text of the second category identifier associated with the text content at the second associated position. At the same time, it meets the needs of the media data production user to produce media data with a specific category identifier, so that the category identifier of the generated media data is compatible with the third category identifier selected by the media data production user, thereby improving the user experience of media data production.

[0120] In this embodiment, at least one second category identifier is determined based on the following method: at least one fourth text is determined based on the text similarity attribute between the text content and at least one third text; and the second category identifier is determined based on the category information of the third media data corresponding to the at least one fourth text.

[0121] Among them, at least one fourth text is text in at least one third text, and the third text is text generated before the current moment. The third media data may refer to media data generated by the text content input by the user according to the media data. The third text refers to the text content input when generating the third media data. The text similarity attribute can be used to characterize the similarity between the third text and the text content. For example, the larger the value of the text similarity attribute, the more similar the two are, and the smaller the value of the text similarity attribute, the less similar the two are.

[0122] Specifically, a similarity calculation method can be used to calculate the text similarity attribute between the text content and the third text. For example, the similarity calculation method includes but is not limited to cosine similarity, Euclidean distance, Pearson correlation coefficient, Jaccard similarity, etc.; or, a pre-trained text similarity evaluation model can be used to analyze the text similarity attribute between the text content and the third text according to the feature information of the text content. Optionally, the text similarity evaluation model includes but is not limited to the bag-of-words model and TF-IDF, word embedding model, CNN, LSTM and BERT, etc. Further, the third text whose text similarity attribute is greater than a preset threshold can be used as the fourth text; the third text can also be sorted from high to low according to the text similarity attribute, and the third text with a set number of front positions can be used as the fourth text. Further, the category information of at least one third media data generated based on each fourth text can be determined, and the attribute information of all or part of the third media data corresponding to the same category information can be determined. Further, in combination with these attribute information, at least one category identifier corresponding to the text content can be determined from the category information as the second category identifier. The attribute information may be information associated with the third media data, such as the attribute information may include the frequency of generating the third media data, feedback information on the effect of the third media data, etc. For example, the category information whose frequency of generating the third media data is greater than the preset frequency may be used as the second category identifier; or the category information whose frequency of generating the third media data ranks first may be used as the second category identifier. The advantage of determining the second category identifier is that the second category identifier that is popular and compatible with the currently input text content can be displayed to the media data production user, so that the media data production user can select the required category identifier from it to produce media data and improve the user experience.

[0123] S330: Generate and display first media data according to the first interaction prompt information.

[0124] S340: Obtain second interaction prompt information, and generate and display second media data according to the second interaction prompt information and the first media data.

[0125] The technical solution provided by the embodiment of the present disclosure responds to a first operation of inputting text content in a text input box on a display interface by displaying a first text in a first form in the text input box and displaying at least one second text at a first associated position of the first text, thereby facilitating media data production users to quickly find replacement text, meeting the production needs of media data production users, and improving the media data production user experience.

[0126] Fig.11A flowchart of a method for generating media data provided by an embodiment of the present disclosure. The technical solution of this embodiment further refines S120 on the basis of the above embodiment, and its specific implementation method can refer to the detailed description of the embodiment of the present disclosure. Among them, the technical features that are the same or similar to the above embodiment are not repeated here.

[0127] like Fig.11 As shown, the method of this embodiment may specifically include:

[0128] S410: In response to a first operation, generate first interaction prompt information.

[0129] S420: Input the first interaction prompt information into the first content generation model to generate media data, and output and display the first media data associated with the content of the first interaction prompt information.

[0130] The first content generation model may be a pre-trained model. For example, the first content generation model may be trained based on preset text content and corresponding media data, so that the first content generation model has the ability to generate corresponding media data according to the first interaction prompt information.

[0131] In this embodiment, the first interaction prompt information can be used as the input of the first content generation model. The first content generation model analyzes the needs and intentions of the media data production user to produce media data based on the first interaction prompt information, obtains the first media data associated with the content of the first interaction prompt information, and outputs it for display.

[0132] It should be noted that the first interaction prompt information can be text, image, video, audio or other types of information, and the generated media data may also be various types of data. In order to ensure the accuracy of media data generation, models for generating different types of media data can be trained separately, and these models can be integrated into the first content generation model. For example, different types of models such as text generation model, image generation model, audio generation model, video generation model, etc. can be integrated into the first content generation model. In this way, after the first interaction prompt information is input into the first content generation model, the type of media data generated based on the first interaction prompt information can be identified, and then the model under the corresponding type can be used to process the first interaction prompt information to generate the first media data associated with the content of the first interaction prompt information. Alternatively, according to the information type of the interaction prompt information, a model of the corresponding type can be selected, analyzed, and the first media data associated with the content of the first interaction prompt information can be generated.

[0133] In order to enable the media data production user to more intuitively understand the generation status of the media data, the generation progress of the first media data based on the first interactive prompt information and the media data corresponding to the generation progress can be displayed in the display interface during the process of generating the first media data based on the first interactive prompt information. The generation progress may refer to the progress from the start of generating the first media data to the current state, which can be expressed in the form of decimals, fractions, percentages or progress bars. There are differences in the media data displayed for different generation progress.

[0134] In this embodiment, in the process of generating the first media data according to the first interactive prompt information, the generation progress of the media data and the media data corresponding to the generation progress can be displayed on the display interface at the same time. The media data displayed for different generation progress is different, and the corresponding display forms can also be different. For example, as the generation progress advances, the displayed media data gradually becomes clearer from blurry. For example, see Fig.12 , Fig.13 and Fig.14 , the transparency of media data gradually increases from 35% generation progress to 75% and then to 95%.

[0135] The advantage of this setting is that by providing real-time feedback of the generation progress of the first media data to the media data production user, the media data production user can understand the progress of the current media data generation and intuitively know how much has been generated and how much remains to be generated. At the same time, by differentially displaying the media data displayed at different generation progress, the partial content of the generated media data can be gradually displayed, thereby increasing the interest of the media data production user and enhancing the expectations of the media data production user.

[0136] In this embodiment, in the process of generating the first media data, the first interaction prompt information can be used as the historical conversation content in the display interface, and the first preset description information can be displayed in the text input box and the display content in the target control can be adjusted to the second preset description information.

[0137] The historical session content may be a record generated during the interaction between the media data production user and the system in the current session. The first preset description information is used to characterize the generation status of the first media data. For example, the generation status may include but is not limited to the status of starting generation, generation in progress, generation exception, generation pause, and generation end. The second preset description information is used to characterize that after the target control is triggered, the generation of the first media data stops. The target control may be a visual control that supports input of information to stop generating the first media data.

[0138] Specifically, after the first interaction prompt information is generated, the first interaction prompt information can be displayed in the session area of ​​the display interface as the historical session content in the session area. At this time, the historical session content is in a state that cannot be edited or modified. The session area can be an area on the display interface used to display the interaction information between the media data production user and the system. At the same time, the generation status corresponding to the first media data can be displayed in the text input box, so that the media data production user can understand the generation progress and status changes in real time. At the same time, the display content in the target control can be adjusted to the second preset description information, so that the media data production user can clearly determine whether to pause, continue, or cancel the generation of the first media data through the second preset description information, providing more control for the media data production user and making the generation process more flexible and controllable.

[0139] For example, see Fig.15 In the process of generating the first media data, "a multimedia video of an apple tree" is used as the historical session content, the first preset description information "generating" is displayed in the text input box, and the display content in the target control is adjusted from "sending" to the second preset description information "paused".

[0140] In this embodiment, in response to a trigger operation on a target control whose description information is the second preset description information, the historical conversation content is adjusted to an editable state, so as to generate first interaction prompt information based on the historical conversation content in an editable state. The editable state may refer to a state in which the content can be edited, and editing includes modifying, deleting, or adding new information.

[0141] It can be understood that: when it is detected that the media data production user triggers the target control that displays the second preset description information, the historical session content is adjusted to an editable state, and the media data production user is allowed to edit the historical session content. That is, the historical session content at this time is in an editable state, and the media data production user can edit the historical session content at this time, such as modifying, deleting or adding new information, etc., or may not modify the historical session content. If the historical session content is updated, the first interactive prompt information can be regenerated based on the updated historical session content, and S120 can be re-executed to generate the first media data. If the historical session content is not updated, the first media data can continue to be generated.

[0142] The technical solution provided by the disclosed embodiment switches the editing state of the historical conversation content by triggering the target control, allowing the media data production user to re-edit the historical conversation content inputted previously to regenerate the corresponding first media data, thereby providing the media data production user with a more intuitive and easy-to-understand and operate interaction method, improving the flexibility and convenience of media data production, and enabling the media data production user to express their needs more accurately.

[0143] S430: Acquire second interaction prompt information, and generate and display second media data according to the second interaction prompt information and the first media data.

[0144] The technical solution provided by the embodiment of the present disclosure can effectively improve the efficiency of media data generation by inputting the first interaction prompt information into the first content generation model to generate media data, and outputting the first media data associated with the content of the first interaction prompt information, so as to generate second media data that meets the user's expectations based on the second interaction prompt information and the first media data.

[0145] Based on the above embodiments, the second interaction prompt information is determined based on at least one of the following methods: one method is: based on the editing content corresponding to at least one editing item presented in the display interface, the second interaction prompt information is generated, wherein the editing content is supplementary information of the first interaction prompt information; another method is: based on the newly added text information in the display interface, the second interaction prompt information is generated, wherein the display interface includes the first interaction prompt information and the first media data; another method is: based on the device motion information of at least one second device associated with the first device that produces the second media data, the second interaction prompt information is generated; another method is: based on the drawing information on the first media data, the second interaction prompt information is generated.

[0146] In order to further clearly introduce the technical solution provided by the embodiment of the present disclosure, each method of determining the second interaction prompt information may be introduced below:

[0147] The first method of “generating second interaction prompt information based on the editing content corresponding to at least one editing item presented in the display interface” is specifically:

[0148] Among them, the editing items can be used to edit or adjust the elements in the media data and the visual controls of the element attributes. For example, these elements may include but are not limited to objects, people, animals, places, plants, food, scenery, weather, environment, etc. The number of editing items can be one or more, and the number can be related to the number of element types in the media data. For example, if the media data includes elements of scenery type and elements of thing type, the number of editing items can be two. The editing content can be the specific attribute information configured for the corresponding element. The editing content is supplementary information to the first interactive prompt information, that is, the editing content is a supplement to the first interactive prompt information, and provides more detailed guidance information for generating the second media data.

[0149] In this embodiment, at least one editing item can be presented in the display interface, and the media data production user can configure the element attributes of the corresponding element according to the editing item, and the input element attributes can be used as the editing content. For example, if the editing item corresponds to the element of apple, the editing content can be the size, color, etc. of the apple; if the editing item corresponds to the element of location, the editing content can be the seaside, park, orchard, etc. When the editing is completed, the second interactive prompt information can be generated according to the editing content in all or part of the editing items, so that the second interactive prompt information can more accurately express the production needs of the media data production user.

[0150] Optionally, at least one editing item includes a first object editing item, and the first object editing item can be used to represent a subject element such as a specific thing or object. For example, the subject element can be a person, an item, an animal, a plant, food, a building, etc. Correspondingly, based on the editing content corresponding to at least one editing item presented in the display interface, the second interactive prompt information is generated, including: in response to the triggering operation of the first object editing item in the display interface, the first object editing panel is displayed; wherein the first object editing panel includes a plurality of element editing fields; based on the editing content of the element editing field, the first object supplementary information in the second interactive prompt information is generated. The element editing field corresponds to the element attribute of the subject element. For example, the element editing field can include but is not limited to the element attribute fields such as the type, size, color, shape, and position of the subject element. The editing content is the specific attribute information entered in the element editing field. For example, if the element editing field is color, its editing content can be red; if the element editing field is type, its editing content can be red Fuji.

[0151] In this embodiment, the first object editing item can be displayed in the display interface, and when the media data production user triggers the first object editing item, the first object editing panel is displayed. The corresponding element attribute can be selected or entered in the element editing field in the first object editing panel. Furthermore, the editing contents in all element editing fields can be integrated to obtain the main supplementary information in the second interactive prompt information. The technical solution provided by the embodiment of the present disclosure supports the media data production user to enter the specific attributes of the main element in the element editing field, facilitates the media data production user to input the demand information more effectively, and helps the system to better understand the main elements that the media data production user wants, so that the second media data generated based on the main supplementary information in the second interactive prompt information is adapted to the actual main needs of the media data production user, while meeting the needs of the media data production user, improving the interactive experience and operational convenience of the media data production user.

[0152] Exemplarily, when the "first object editing item" is clicked, the first object editing panel is displayed, and its schematic diagram is shown in FIG. Fig.16 , the element editing field includes "type, size", etc. Enter "Fuji Apple" in the "Type" element editing field, that is, the editing content is "Fuji Apple".

[0153] Optionally, at least one editing item includes a second object editing item, and the second object editing item is a scene editing item, which can be used to characterize scene elements such as a specific place or environment. For example, scene elements can be scenery, atmosphere, weather, and activities, etc. For example, in a street view image, the main elements can be pedestrians and buildings, and the scene elements can be weather and street views. Accordingly, based on the editing content corresponding to at least one editing item presented in the display interface, second interaction prompt information is generated, including: in response to a triggering operation on the second object editing item in the display interface, a second object editing panel is displayed, and based on the editing content corresponding to at least one element editing field in the second object editing panel, scene supplementary information in the second interaction prompt information is generated.

[0154] The element editing field in the second object editing panel is different from the element editing field in the first object editing panel. The element editing field in the second object editing panel is the attribute editing field corresponding to the scene element; the element editing field in the first object editing panel is the attribute editing field corresponding to the main element. For example, the element editing field may include but is not limited to various element attribute fields such as location, weather, scenery, and activities. The editing content is the specific attribute information entered in the element editing field. For example, if the element editing field is location, the editing content may be an orchard; if the element editing field is weather, the editing content may be foggy day.

[0155] In this embodiment, the second object editing item can be displayed in the display interface. When the media data production user triggers the second object editing item, the second object editing panel can be displayed. The multiple element editing fields in the second object editing panel can be pre-set or can correspond to the scene elements in the first media data and / or the first interactive prompt information. Further, the media data production user can select or enter the element attribute of the corresponding scene element in the element editing field, and the specific attribute information entered is the content edited in the element editing field. Further, the content edited in all element editing fields can be integrated to obtain the scene supplementary information in the second interactive prompt information. The technical solution provided by the embodiment of the present disclosure supports the specific attributes of the scene element entered by the media data production user in the element editing field, facilitates the media data production user to more effectively enter the demand information, helps the system better understand the scene that the media data production user wants, so that the second media data generated based on the scene supplementary information in the second interactive prompt information is adapted to the actual scene requirements of the media data production user, meets the needs of the media data production user, and improves the interactive experience and operation convenience of the media data production user.

[0156] For example, when the "first object editing item" is clicked, the first object editing panel is displayed, and its schematic diagram can be continued to refer to Fig.16 , the element editing field includes "type, size", etc. Enter "Fuji Apple" in the "Type" element editing field, that is, the editing content is "Fuji Apple".

[0157] It should be noted that, regardless of whether it is the first object editing item or the second object editing item, when the input elements are different, the generated editing fields are also different, that is, the editing fields corresponding to different elements or element types are pre-set, so that when it is detected that an element is being edited, the editing field that is adapted to it can be displayed, thereby generating editing content that is adapted to the element.

[0158] In actual applications, elements can continue to be superimposed on the basis of the first interaction prompt information. At this time, it can be: in response to the trigger operation of adding the first object editing item and / or the second object editing item, the element editing field of the newly added subject and / or the element editing field of the newly added scene is displayed to update the second interaction prompt information based on the editing content of the newly added subject and the editing content of the newly added scene.

[0159] The newly added subject is a subject element newly added to the media data, and the newly added scene can be a scene element newly added to the media data.

[0160] In actual applications, a media data production user can enter a new subject element in the new element input control corresponding to the first object editing item in the display interface, and the subject element is the new subject. In response to the new creation operation on the new subject, the element editing field corresponding to the new subject is displayed. These fields allow the media data production user to configure the properties of the newly added subject, such as color, size, position, etc. Specific attribute information corresponding to the new subject can be entered or selected in the element editing field as the editing content of the new subject. Furthermore, the subject supplementary information in the second interactive prompt information can be updated based on the editing content corresponding to the new subject. Correspondingly, a new scene element can also be entered in the new element input control corresponding to the second object editing item in the display interface as a new scene. The element editing field of the new scene is displayed to update the scene supplementary information in the second interactive prompt information based on the editing content of the new subject and the editing content of the new scene. For example, continue to refer to Fig.16 When you click the "Add Subject" control, you can create a new subject and display the element editing field corresponding to the new subject; when you click the "Add Scene" control, you can create a new scene and display the element editing field corresponding to the new scene.

[0161] The technical solution provided by the embodiments of the present disclosure supports media data production users to add scene elements and main elements to media data, and supports configuring the attribute information of the elements. This not only meets the needs of media data production users for producing more complex and rich media data, but also enables the production of media data that meets the personalized needs of media data production users and improves the convenience of operation.

[0162] The second method: "Determining the second interaction prompt information based on the newly added text information in the display interface" can be:

[0163] The display interface includes the first interactive prompt information and the first media data. The newly added text information can be a question, a command or any form of input text.

[0164] In this embodiment, the media data production user can input the demand information for producing media data in the display interface according to his own needs. Each time new demand information is input, the input demand information can be used as new text information. Furthermore, the new text information can be used as the second interactive prompt information; the new text information can also be processed, such as error correction processing, polishing processing, etc., and the processed new text information can be used as the second interactive prompt information; or, natural language processing technology can be used to understand the intention and needs of the media data production user and obtain the second interactive prompt information, so as to generate the second media data based on the second interactive prompt information and the first media data, so that the second media data meets the current needs of the media data production user. In other words, the media data generated last time can be dynamically adjusted according to the real-time input of the media data production user.

[0165] For example, see Fig.17 In the display interface, the first interactive prompt information is "a little girl holding an apple" and an image of a little girl holding an apple (i.e., the first media data). When the media data production user inputs "smiling and holding an apple", "smiling and holding an apple" can be used as the newly added text information to generate an image of a little girl smiling and holding an apple (i.e., the second media data).

[0166] The third way is: "generating the second interaction prompt information based on the device motion information of at least one second device associated with the first device that produces the second media data" can specifically be:

[0167] Among them, the first device is a device for producing the second media data. The second device can be the first device or a device that communicates with the first device. The communication method includes but is not limited to Bluetooth, wired connection, Ethernet, etc. It should be noted that the first device and the second device are relative. For example, when media data is produced once, device A is the first device for producing media data, and device B is the second device associated with the first device; when media data is produced again, device B is the first device for producing media data, and device A is the second device associated with the first device; when media data is produced again, device B is the first device for producing media data, and device B is also the second device. The first device and the second device can be computers, mobile terminals, tablets, smart bracelets, smart watches or other smart devices, and the types of devices are not specifically limited. Device motion information can be relevant information used to describe device motion. For example, device motion information includes but is not limited to position change information, posture change information, direction change information, speed change information, angular velocity change information, acceleration change information, etc.

[0168] In this embodiment, during the communication between the first device producing the second media data and the second device, the device motion information generated by the second device can be received or obtained in real time or periodically, and the device motion information can be dynamically displayed on the interface of the first device. Furthermore, the generated device motion information can be used as the second interactive prompt information. The second interactive prompt information can also be updated when the device motion information generated by the second device is received or obtained again. This makes the running effect of one or some specific elements in the second media data generated based on the second interactive prompt information consistent with or similar to the motion effect of the device motion information, thereby improving the richness and fun of the media data, while enhancing the motion experience and participation of the users producing the media data.

[0169] For example, when the media data production user uses desktop computer A to produce the second media data, desktop computer A is the first device, and the mobile terminal B that communicates with desktop computer A can be used as the second device. Alternatively, the media data production user uses mobile terminal B to produce the second media data, and mobile terminal B is the first device, and mobile terminal B can also be used as the second device. The media data production user shakes mobile terminal B, and mobile terminal B moves, and its movement information can be used as device movement information. The device movement information can be obtained through the sensor of the second device (such as an accelerometer, gyroscope, GPS, etc.).

[0170] The fourth method: "generating second interaction prompt information based on the drawing information on the first media data" can be:

[0171] The drawing information may be information obtained by drawing on the first media data. For example, the drawing information may be expressed in the form of visual information such as images, graphics, and patterns.

[0172] In this embodiment, the media data production user performs drawing operations on the first media data, such as marking, drawing, adding annotations or other forms of editing, to obtain drawing information. Furthermore, the drawing information can be used as the second interactive prompt information. The second media data generated based on the second interactive prompt information carries the corresponding drawing information, thereby improving the richness and interest of the media data and meeting the production needs of the media data production user. For example, the first media data can be an image of an apple tree, and the media data production user can draw some red flower-like shapes and white cloud-like shapes on the image. The drawn information can be used as drawing information.

[0173] In actual applications, any one or more of the above methods can be selected to generate the second interaction prompt information, and the specific implementation method can be set by the media data production user according to actual needs, which is not limited in this embodiment. The technical solution provided by the embodiment of the present disclosure usually determines the second interaction prompt information by at least one of the above methods, which can meet the needs of media data production users to produce diverse and personalized media data, and improve the user experience of media data production users.

[0174] Fig.18 A flowchart of a method for generating media data provided by an embodiment of the present disclosure. The technical solution of this embodiment can further refine the generation of the second media data based on the above embodiment, and its specific implementation method can refer to the detailed description of the embodiment of the present disclosure. Among them, the technical features that are the same or similar to the above embodiment are not repeated here.

[0175] like Fig.18 As shown, the method of this embodiment may specifically include:

[0176] S510: In response to a first operation, generate first interaction prompt information.

[0177] S520: Generate and display first media data according to the first interaction prompt information.

[0178] S530: Input the second interaction prompt information and the first media data into a second content generation model to generate media data, and output and display the second media data.

[0179] Among them, the second content generation model can be a graph-to-video model or a graph-to-graph model, and its specific model can be determined according to the second interactive prompt information. If the second interactive prompt information is to generate a video, then the second content generation model is a graph-to-video model. If the second interactive prompt information is to generate an image, then. The second content generation model can be a graph-to-graph model. Of course, if the first media data is a video, then the second content generation model can be a video-to-video model, or a video-to-image model. Regardless of what kind of model the second content generation model is, it has the function of interactive prompt information and media data processing.

[0180] In this embodiment, the second interaction prompt information and the first media data can be used as inputs of the second content generation model. The second content generation model analyzes the editing intention of the media data production user on the first media data according to the second interaction prompt information, thereby processing the first media data, generating second media data associated with the content of the second interaction prompt information, and outputting the second media data for display.

[0181] It should be noted that in order to ensure the accuracy of the generation of the second media data, corresponding types of models can be generated according to different types of preset prompt information, different types of preset media data and / or different types of actual media data generated, and different types of models can be integrated into the second content generation model. For example, the model includes but is not limited to encoder-decoder structure, attention mechanism, generative adversarial network, etc. In this way, after the second interactive prompt information and the first media data are input into the second content generation model, the data type of the first media data can be identified, the data type of the second interactive prompt information can be identified, and the type of media data generated based on the second interactive prompt information and the first media data can be identified. Then, the second interactive prompt information and the first media data are processed using models corresponding to these data types to generate second media data associated with the content of the second interactive prompt information. Exemplarily, the second interactive prompt information is text, the first media data is an image, and the second media data to be generated is a video. The second interactive prompt information and the first media data can be input into the model of the image-generated video.

[0182] It should be noted that, in the embodiment of the present disclosure, the second media data may be media data generated based on the first media data and adapted to the second interaction prompt information. The following describes what the second media data corresponding to different second interaction prompt information may be.

[0183] In the case where the second interactive prompt information is device motion information, the second media data is video stream data, and the second media data is video stream data obtained after processing the first media data according to the device motion information. In other words, the second interactive prompt information can provide the motion information of the device. The device motion information is obtained based on a gyroscope deployed in the device, or it can be obtained by an inertial measurement unit in the device. The device motion information can be three-dimensional coordinate information during the device movement process. According to the three-dimensional coordinate information, the device motion trajectory and motion speed can be determined. According to the device motion trajectory and motion speed, the corresponding display object in the first media data can be controlled to move. Correspondingly, the second media data is video stream data in which the display object in the first media data is in motion. That is, the display content in the first media data is adjusted according to the device motion information, so that the display content presents a corresponding motion effect in the video stream data. The second media data shows how the display content in the first media data moves accordingly based on the device motion information.

[0184] For example, assuming that the first media data is an image of a soccer ball on a lawn, the media data creator can shake or swing the second device to generate device operation information. At this time, the device motion information is the speed and trajectory of the device during the shaking period, and it is used as the second interactive prompt information. The second media data can be: an animated video of a soccer ball being kicked on a lawn. The visual effect diagram of the second media data can be found in Fig.19 , Fig. 20 and Fig.21 Alternatively, if the second device moves to the left, the display content in the second media data may present a leftward camera movement effect. At this time, the motion information of the football in the animation video is determined based on the device motion information, that is, is related to the speed and trajectory determined by the device motion information.

[0185] In the case where the second interactive prompt information is drawing information on the first media data, the second media data is video stream data, and the second media data is video stream data that draws the drawing information on the first media data and generates a second picture corresponding to the drawing information. That is to say, the second interactive prompt information can provide the drawing information drawn on the first media data, and then combine the first media data and these drawing information to generate video stream data, so that the generated video stream data presents a video effect from drawing the corresponding drawing information on the first media data to gradually generating the second picture corresponding to the drawing information. The video stream data is the second media data, the second media data. Exemplarily, the first media data can be an image of an apple tree, and the media data production user can draw some red flower-like shapes and white cloud-like shapes on the image. These drawing information are used as the second interactive prompt information. The second media data can be a complete video stream data from starting to draw these drawing information to generating an image with flowers under the apple tree and white clouds above the apple tree. The visual effect diagram of the second media data can be seen in Fig. 22 , Fig.23 and Fig.24 .

[0186] In the case where the second interactive prompt information is newly added text information in the display interface, the data stream type of the second media data is related to the semantic information of the newly added text information. The data stream type can be image, video, text, etc. For example, the semantic information of the newly added text information is to generate an image, that is, the data stream type of the second media data is image. The semantic information of the newly added text information is to generate a video, that is, the data stream type of the second media data is video.

[0187] For example, please refer to Fig.17The first media data is an image of a little girl holding an apple, and the newly added text information is "smiling and holding an apple". The second media data can be an image of a little girl smiling and holding an apple.

[0188] In order to increase the attractiveness of the content in the media data to the media data production user, so that the media data production user can clearly and intuitively understand the content in the media data, a fifth text associated with the second media data and audio information corresponding to the fifth text can be generated; based on the fifth text and the audio information, the second media data is updated. Among them, the fifth text can be a text description or annotation of the video screen content. The language type of the audio information can be determined based on the trigger selection of the media data production user, the preset language type, or the language type used in the device's home location. The audio information can be dialogue, music, sound effects, dubbing, etc.

[0189] In this embodiment, the second media data can be input into the text content generation model, and each frame of the second media data can be extracted. A matching text, i.e., a fifth text, can be configured for each frame, or a matching text can be configured for some frames to obtain a fifth text that is compatible with the second media data. The text can also be polished based on the text polishing model to enhance the representation capability of the text and obtain a polished fifth text.

[0190] After obtaining the fifth text, the audio information input by the user can be produced according to the media data to generate audio information corresponding to the fifth text; the audio template selected by the user can also be produced according to the media data to generate audio information corresponding to the fifth text; or the fifth text can be input into the audio generation model to obtain audio information corresponding to the fifth text. After creating the relevant fifth text and the audio information corresponding to the fifth text for the second media data, the display duration corresponding to the second media data and the audio duration of the audio information can also be aligned to obtain the updated second media data, so that the second media data not only contains the picture content, but also contains the audio information synchronized with the picture content, thereby enhancing the multimodal characteristics of the media data, making it richer and more attractive to the users who produce the media data.

[0191] Exemplarily, a frame of picture in the second media data may be a sunset, and the caption text corresponding to the frame of picture input into the caption content generation model is sunset, and the caption text obtained after the caption polishing model polishes the caption text may be sunset, and the sunset glows half the sky red.

[0192] In this embodiment, the second media data may be further edited, which may be:

[0193] In response to an event that the second media data is video stream data, a video track and an audio track corresponding to the second media data are displayed. The audio track and the video track are aligned in play time. The play time corresponds to the display duration corresponding to the second media data. The audio track is used to display audio information, and the video track includes multiple video clips, which are determined based on preset clip division conditions. For example, the preset clip division conditions can be divided according to different scenes or camera movements of the video clips, or can be divided according to the length of the clips. The video track also includes a fifth text corresponding to the video clip and a clip camera movement type. The clip camera movement type refers to a specific photography method or camera movement method used in the video clip. For example, the clip camera movement type may include but is not limited to: long shot, medium shot, close shot, close-up, push-in, pull-out, pan, tilt, and follow, etc.

[0194] The method of dividing the video segments according to scenes or camera movements may be: inputting the second media data into a pre-trained scene division model, or inputting the second media data into a pre-trained camera movement type division model, to obtain the camera movement type or scene type corresponding to each video frame. Multiple video frames that are consecutive and belong to the same camera movement type or scene type are regarded as a video segment.

[0195] It should also be noted that when multiple video clips are displayed in the video track, the camera movement type and camera movement duration between the two video clips are also included. The benefit of setting this information is that it can achieve effective transition of video clips, thereby avoiding the problem of long-term black screen, and thus improving the picture display effect.

[0196] In this embodiment, when the second media data is video stream data, the second media data can be displayed in the display interface, and at the same time, the video track and audio track corresponding to the second media data can be displayed. Fig.25 The video track contains the image information, fifth text, and camera movement type of multiple video clips, and the audio track contains audio information that matches multiple video clips. This allows media data production users to edit and adjust video content according to the displayed information segmentation, thereby improving the efficiency of video stream production. In order to ensure the continuity and synchronization of the viewing experience, the audio track and the video track are precisely aligned in terms of playback time, ensuring that media data production users can hear the corresponding audio when they see a certain picture.

[0197] In this embodiment, in response to a preset interactive operation on any video segment, a content editing control is displayed at a segment display position of the video segment generating the preset interactive operation.

[0198] The content editing control is used to re-edit the segment content or second media data corresponding to the video segment after being triggered. The preset interactive operations may include but are not limited to operations such as clicking, touching, sliding, and hovering the mouse. The segment display position may be a display area in the display interface for displaying the video segment. The content editing control may be a visual control for supporting input of adjustment parameters for adjusting the video segment. For example, the adjustment parameters may be cutting parameters, resizing parameters, adding special effects parameters, or changing the segment order, etc.

[0199] Specifically, after the second media data is displayed to the media data production user in the form of a video clip, the media data production user can input adjustment parameters for any video clip according to actual needs. When the media data production user slides the mouse to the display area position of a certain video clip, or a certain video clip in the video track, or a video clip corresponding to a certain audio information in the audio track, it is considered that a preset interactive operation has been generated for the video clip. At this time, the content editing control can be displayed at the clip display position of the video clip that generates the preset interactive operation. For example, the content editing control can be a text editing control, a camera type editing control, a regeneration control, etc. The media data production user can enter the corresponding editing parameters in the content editing control, such as adjusting the segment camera type, and click the regeneration control. The system can regenerate the segment content corresponding to the video clip according to the editing parameters in the content editing control and update the second media data. The advantage of this setting is that it supports the media data production user to create video segments based on the second media data, so that the media data production user can process media data more flexibly and intuitively, improve the efficiency of media data generation, and enhance the effect of media data production user experience and multimedia content interactivity.

[0200] The technical solution provided by the embodiment of the present disclosure improves the efficiency and continuity of media data generation by inputting the second interaction prompt information and the first media data into the second content generation model to generate media data, and generates the second media data in combination with the previously generated content, thereby enhancing the user production experience and meeting the needs for complex media data production.

[0201] Fig.26 A flow chart of a method for generating media data provided by an embodiment of the present disclosure. Based on the above-mentioned embodiment, after obtaining the second media data, the present invention also supports further processing of the second media data to fine-tune the second media data. The specific implementation method can refer to the detailed description of the embodiment of the present disclosure. Among them, the technical features that are the same or similar to the above-mentioned embodiments are not repeated here.

[0202] like Fig.26 As shown, the method of this embodiment may specifically include:

[0203] S610: In response to a triggering operation on a display screen of second media data, presenting a screen adjustment area and a screen adjustment dialog box in the display screen.

[0204] Among them, the display screen can be the screen corresponding to the second media data. For example, if the second media data corresponds to an image, then the display screen is this image; if the second media data corresponds to a video, then the display screen can be a frame of the video. The screen adjustment area can be an area that allows the media data production user to adjust the screen content. The screen adjustment area can be for the entire display screen or for a specific area in the display screen. There are many ways to determine the screen adjustment area, one is intelligent recognition, and the other is based on the user's trigger selection. The screen adjustment dialog box can be a visual control for supporting the input of screen adjustment information. For example, the screen adjustment dialog box can input descriptive information for adjusting the content of the screen adjustment area; the presentation form of the screen adjustment dialog box can be a slider or a selection box. The content entered in the screen adjustment dialog box can be a description text of "the apple is a little redder".

[0205] In this embodiment, when the media data production user triggers the display screen of the second media data, or triggers the screen adjustment control, the screen adjustment area can be presented on the display interface; or in response to a trigger operation of manually selecting content on the display screen of the second media data, the screen adjustment area in the display screen can be determined based on the movement information of the mouse.

[0206] That is to say, the media data production user can use the mouse to select specific content or area in the display screen. When the media data production user moves the mouse to select a specific area in the screen, the position change of the mouse is obtained to obtain motion information. The area selected by the media data production user is determined based on the mouse motion information. This area can be the screen area that the media data production user wants to adjust. Furthermore, the screen adjustment area and the screen adjustment dialog box can be displayed according to a preset method. For example, the preset method can be: displaying the screen adjustment area in the form of edge marks, border marks or masks, and displaying the screen adjustment dialog box at the associated position of the screen adjustment area to indicate which area the media data production user can edit, and support the media data production user to enter adjustment parameters in the screen adjustment dialog box to accurately control the adjustment effect of the video screen.

[0207] For example, when you click on the picture adjustment control in the display interface, AI analyzes the picture and automatically selects the area, displays the picture adjustment area in the display picture, and pops up a picture adjustment dialog box so that you can adjust the selected picture content through language description. This allows media data production users to edit and adjust video content intuitively and flexibly, and more accurately control the final display effect of the video.

[0208] In order to further achieve an intelligent effect, the picture adjustment area can be determined intelligently. Optionally, in response to a trigger operation on the display picture of the second media data, the picture adjustment area is presented in the display picture, including: in response to a trigger operation for intelligent content selection on the display picture of the second media data, the display picture is processed based on the area selection model, and the display picture with the picture adjustment area marked is output; and at least one area adjustment selection item is displayed at a preset position.

[0209] The region selection model may be pre-trained and used to identify and mark regions in the display screen that may need to be adjusted. The region selection model may be a model for identifying salient regions in an image. A model for identifying salient regions may be pre-trained to process the display screen based on the model and present the image adjustment region in the display screen. At least one region adjustment option is used to adjust the image adjustment region after being triggered. At least one region adjustment option is an option for finely adjusting the image adjustment region.

[0210] Specifically, when the display screen of the second media data is triggered, or when the screen adjustment control is triggered, the screen adjustment area can be determined intelligently. The display screen is input into the area selection model, the specific area or specific content that needs to be adjusted in the display screen is identified, and this is marked in the display screen. The marking method may be to frame the edges with a preset color, and display the framed area in a fourth preset display form. Optionally, the fourth preset display form may be to display the screen adjustment area with a preset transparency parameter, or to display it with a mask of preset transparency. At the same time, at least one area adjustment selection item may be displayed at a preset position. The area adjustment selection item may be pre-set or may correspond to the display screen. For example, the area adjustment selection item may be drawing, optimizing edges, selecting specific objects, deleting marking information, and so on. The advantage of displaying the area adjustment selection item is that it is convenient to further adjust the marked screen adjustment area or display screen in combination with the area adjustment selection item for fine-tuning. For example, see Fig. 27 When you click the "Optimize Edge" area adjustment option, the edge of the marked image adjustment area is fitted to obtain the optimized image adjustment area and mark it.

[0211] S620: Based on the picture adjustment content input in the picture adjustment dialog box, adjust the objects in the picture adjustment area to update the second media data.

[0212] The picture adjustment contents include but are not limited to brightness, contrast, saturation, color, scaling, rotation, cropping, object change, etc. The object may refer to a specific element (such as a person, object, background, etc.) in the picture adjustment area.

[0213] In this embodiment, the media data production user can input the content of the adjustment of the picture adjustment area in the picture adjustment dialog box, and the system uses the content as the picture adjustment content. Then, according to the picture adjustment content, the objects in the picture adjustment area are adjusted accordingly, such as the adjustment of image pixels, the adjustment of image attributes, etc. After the adjustment is completed, the adjusted second media data can be used as the new second media data. For example, see Fig. 27 The green covered area is the picture adjustment area. When the user enters "the apple color is a little redder" in the picture adjustment dialog box, the picture adjustment content is "the apple color is a little redder". After processing, the color of the apple in the picture adjustment area becomes redder than before.

[0214] In order to achieve a better interactive effect, that is, to meet the user's demand for rich media data production, the acquired first video and fourth media data associated with the first video can be displayed, wherein the first video includes a preset object, and the fourth media data is part of the second media data and includes a third object associated with the preset object.

[0215] The first video includes a preset object, which may be a preset object category, and optionally, the object category is a user. For example, if the preset object is a user's expression, the first video may be a thing including the user's expression; if the preset object is a user's body movement, the first video may be a video including the user's body movement. The fourth media data may be the same as the second media data, or may be partial data in the second media data. If it is partial data, the fourth media data includes a third object associated with the preset object. The preset object may be an object that is intended to replace the third object.

[0216] The object type of the third object is the same as or similar to the object type of the preset object. For example, the first video is an expression video of the media data production user, and the fourth media data is a generated video of user B, tree, apple, and white cloud. The object type of media data production user B is the same as that of user A, that is, media data production user B is the third object. Object features may be object attribute information associated with an object. For example, object features may include but are not limited to pixel feature information, sound feature information, limb feature information, facial contour feature information, face shape feature information, part location information, and part shape information, etc. The feature information of the preset object may be applied to the third object in the fourth media data to achieve the effect of one-step full video modification.

[0217] In actual applications, in order to facilitate media data production users to clearly see the effect of data application, the first video and its associated fourth media data can be displayed in a preset manner. The preset manner can be to divide the media data display area into two areas, one area is used to display the fourth media data, and the other area is used to display the first video, so as to achieve a comparative display effect.

[0218] The method of applying the first video to the fourth media data may be: when the media data production user triggers the control for applying the first video, the object features of the preset object may be extracted based on the pre-trained feature extraction model or feature extraction algorithm. Through the object features of the preset object, a third object with similar features to the preset object is extracted from the fourth media data, and the object features of the third object may be replaced with the object features of the preset object to obtain the updated fourth media data and display it. At this time, the fourth media data includes the object features of the preset object. For example, if the preset object is the user's expression, then the expression of the preset object may be reused on the third object so that the third object displays the expression of the preset object in the fourth media data. In other words, the object features of the preset object are obtained, and in the case of the object features, the object features of the third object are adjusted while the overall content of the third object is unchanged so that the third object can reuse the object features of the preset object.

[0219] In actual applications, if the object features of the preset objects in the first video do not meet the effects desired by the media data production user, the control for re-acquiring the first video can be triggered to re-acquire the first video, and re-determine the updated second media data based on the re-acquired first video, thereby meeting the needs of the media data production user to update the objects in the media data multiple times.

[0220] On the basis of the above technical solution, in order to further enrich the screen content, it may also be that: in response to a trajectory drawing operation on the display screen corresponding to the second media data, a fourth object is displayed in the display screen moving according to the drawn trajectory.

[0221] It can be understood that: the media data production user can draw a trajectory in the display screen corresponding to the second media data. After the trajectory is drawn, the fourth object in the display screen can be controlled to move along the trajectory. It can also be that after the trajectory is drawn, the fourth object is determined according to the screen content of the display screen, and the fourth object is added to the display screen to control the fourth object to move along the trajectory.

[0222] On the basis of the above technical solution, after the trajectory is drawn, the motion attribute parameters are determined according to the screen attributes of the display screen, wherein the motion attribute parameters are used to control the motion speed and / or height of the fourth motion, etc. Optional motion attribute parameters may be air resistance parameters, etc.

[0223] The technical solution provided by the embodiment of the present disclosure can support fine-tuning of the second media data in a visual form after obtaining the second media data, thereby achieving the effect of high efficiency in adjusting the second media data, that is, achieving the effect of further post-processing the second media data.

[0224] Fig.28 A schematic diagram of the structure of a media data generating device provided by an embodiment of the present disclosure is shown in FIG. Fig.28 As shown, the device includes: a first interaction prompt information generating module 710 , a first media data generating module 720 and a second media data generating module 730 .

[0225] Among them, the first interaction prompt information generation module is used to generate first interaction prompt information in response to a first operation; the first media data generation module is used to generate and display first media data based on the first interaction prompt information; the second media data generation module is used to obtain second interaction prompt information, and generate and display second media data based on the second interaction prompt information and the first media data.

[0226] On the basis of the above technical solution, the device further comprises:

[0227] The first category identification display module is used to display at least one first category identification; wherein the category information of the second media data is related to the first category identification.

[0228] Based on the above technical solutions, the first interactive prompt information generating module includes:

[0229] A text replacement unit, configured to, in response to a first operation of inputting text content in a text input box of a display interface, display a first text in a first form in the text input box and display at least one second text at a first associated position of the first text, wherein the first text is text in the text content, and the at least one second text is associated with the first text;

[0230] The first prompt information generating unit is used to update the text content in response to an event of updating the first text based on the triggered second text, so as to generate the first interaction prompt information based on the updated text content.

[0231] On the basis of the above technical solutions, the device further includes:

[0232] A category description text generation module, configured to display, in response to a first operation of inputting text content in a text input box of a display interface, a category description text of at least one second category identifier associated with the text content at a second associated position of the text input box; wherein the at least one second category identifier is related to the text content;

[0233] A prompt content generation module is used to respond to a triggering operation on the category description text and use a third category identifier corresponding to the triggered category description text as prompt content in the first interactive prompt information; wherein the category information of the second media data is related to the third category identifier.

[0234] Based on the above technical solutions, the first media data generation module is further used to input the first interaction prompt information into the first content generation model to generate media data, and output the first media data associated with the content of the first interaction prompt information.

[0235] Based on the above technical solutions, the first media data generation module is further used to display in a display interface the generation progress of the first media data based on the first interactive prompt information, and the media data corresponding to the generation progress.

[0236] Based on the above technical solutions, the first media data generation module is further used to use the first interaction prompt information as the historical conversation content in the display interface, and to display the first preset description information in the text input box and adjust the display content in the target control to the second preset description information;

[0237] The first preset description information is used to represent the generation state of the first media data, and the second preset description information is used to represent that after the target control is triggered, the generation of the first media data stops.

[0238] Based on the above-mentioned technical solutions, the first interaction prompt information generation module is also used to adjust the historical conversation content to an editable state in response to a trigger operation on a target control whose description information is the second preset description information, so as to generate the first interaction prompt information based on the historical conversation content in an editable state.

[0239] Based on the above technical solutions, the second interaction prompt information is determined based on at least one of the following methods:

[0240] A first determining module, configured to generate the second interaction prompt information based on edit content corresponding to at least one edit item presented in the display interface, wherein the edit content is supplementary information of the first interaction prompt information;

[0241] a second determining module, configured to generate the second interaction prompt information based on the newly added text information in the display interface, wherein the display interface includes the first interaction prompt information and the first media data;

[0242] a third determining module, configured to generate the second interaction prompt information based on device motion information of at least one second device associated with the first device that produces the second media data;

[0243] The fourth determining module is configured to generate the second interaction prompt information based on the drawing information on the first media data.

[0244] Based on the above technical solution, the at least one editing item includes a first object editing item, and the first determining module includes:

[0245] A first display unit, configured to display a first object editing panel in response to a triggering operation on the first object editing item in the display interface; wherein the first object editing panel includes a plurality of element editing fields;

[0246] The first determining unit is used to generate the first object supplementary information in the second interaction prompt information based on the edit content of the element edit field.

[0247] On the basis of the above technical solutions, the first determination module includes:

[0248] a second display unit, configured to display a second object editing panel in response to a triggering operation on the second object editing item in the display interface, wherein an element editing field in the second object editing panel is different from an element editing field in the first object editing panel, and the second object editing item is a scene editing item;

[0249] The second determining unit is used to generate the scene supplementary information in the second interaction prompt information based on the editing content corresponding to at least one element editing field in the second object editing panel.

[0250] On the basis of the above technical solution, the second media data generation module is further used to input the second interaction prompt information and the first media data into a second content generation model to generate media data and output second media data.

[0251] Based on the above technical solution, the second interactive prompt information is device motion information, the second media data is video stream data, and the second media data is video stream data obtained by processing the first media data according to the device motion information.

[0252] Based on the above technical solution, the second interactive prompt information is drawing information on the first media data, the second media data is video stream data, the second media data is drawing the drawing information on the first media data, and generating video stream data of a second picture corresponding to the drawing information.

[0253] Based on the above technical solution, the second interactive prompt information is the newly added text information, and the second media data is a data stream type and media content related to the semantic information of the newly added text information.

[0254] On the basis of the above technical solution, the device further comprises:

[0255] A data generating module, configured to generate a fifth text associated with the second media data and audio information corresponding to the fifth text;

[0256] A data updating module is used to update the second media data based on the fifth text and the audio information.

[0257] On the basis of the above technical solution, the device further comprises:

[0258] The first display module is used to display a video track and an audio track corresponding to the second media data in response to an event that the second media data is video stream data; wherein the audio track and the video track are aligned in playback time, the audio track is used to display audio information, and the video track includes multiple video segments, and the multiple video segments are determined based on preset segment division conditions.

[0259] Based on the above technical solution, the device also includes: a video segment editing module, which is used to display a content editing control at the segment display position of the video segment that generates the preset interactive operation in response to a preset interactive operation on any video segment; wherein the content editing control is used to re-edit the segment content corresponding to the video segment or the second media data after being triggered.

[0260] On the basis of the above technical solution, the device further comprises:

[0261] a picture adjustment area determination module, configured to present a picture adjustment area and a picture adjustment dialog box in the display screen in response to a triggering operation on the display screen of the second media data;

[0262] The picture content adjustment module is used to adjust the objects in the picture adjustment area based on the picture adjustment content input in the picture adjustment dialog box to update the second media data.

[0263] Based on the above technical solution, the picture adjustment area determination module includes:

[0264] a first area determination unit, configured to, in response to a triggering operation of performing intelligent content selection on a display picture of the second media data, process the display picture based on a region selection model, and output a display picture with the picture adjustment area marked;

[0265] The second area determination unit is used to display at least one area adjustment option at a preset position, wherein the at least one area adjustment option is used to adjust the picture adjustment area after being triggered.

[0266] On the basis of the above technical solution, the device further comprises:

[0267] a fourth media data display module, configured to display the acquired first video and fourth media data associated with the first video, wherein the first video includes a preset object, and the fourth media data is part of the second media data and includes a third object associated with the preset object;

[0268] The fourth media data updating module is used to update the third object based on the preset object and update the fourth media data in response to a triggering operation on applying the first video.

[0269] On the basis of the above technical solution, the device further includes: a picture adjustment module, which is used to respond to the trajectory drawing operation on the display picture corresponding to the second media data, and display the fourth object moving according to the drawn trajectory in the display picture.

[0270] The technical solution of the disclosed embodiment processes the generated first interactive prompt information to obtain the first media data. Based on the first media data, the second interactive prompt information can be combined to generate the second media data associated with the first media data and the second interactive prompt information, so that only the relevant prompt information needs to be input to generate the second media data in combination with the previously generated content, thereby improving the efficiency and continuity of media data generation. Furthermore, the second media data can be a combination of one or more of audio, video, and text, that is, the media data can be generated based on the same media data production software or tool, thereby improving the universal applicability of media data.

[0271] The media data generating device provided in the embodiments of the present disclosure can execute the media data generating method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0272] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0273] Fig.29 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig.29 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0274] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.29 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0275] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0276] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0277] The electronic device provided in the embodiment of the present disclosure and the media data generation method provided in the above embodiment belong to the same inventive concept. The technical details not fully described in the embodiment of the present disclosure can be referred to the above embodiment, and the present embodiment has the same beneficial effects as the above embodiment.

[0278] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the media data generating method provided in the above embodiments is implemented.

[0279] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0280] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0281] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0282] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0283] In response to the first operation, generating first interaction prompt information;

[0284] Generate and display first media data according to the first interaction prompt information;

[0285] Second interaction prompt information is obtained, and second media data is generated and displayed according to the second interaction prompt information and the first media data.

[0286] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0287] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0288] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0289] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0290] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0291] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0292] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0293] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for generating media data, characterized in that: include: In response to the first operation, generating first interaction prompt information; Generate and display first media data according to the first interaction prompt information; Second interaction prompt information is obtained, and second media data is generated and displayed according to the second interaction prompt information and the first media data.

2. The method according to claim 1, characterized in that The method further comprises: Display at least one Category 1 logo; The category information of the second media data is related to the first category identifier.

3. The method according to claim 1, characterized in that Before generating the first interaction prompt information, the method includes: In response to a first operation of inputting text content in a text input box of a display interface, displaying a first text in a first form in the text input box and displaying at least one second text at a first associated position of the first text, wherein the first text is text in the text content, and the at least one second text is related to the first text; In response to an event of updating the first text based on the triggered second text, the text content is updated to generate the first interaction prompt information based on the updated text content.

4. The method according to claim 1 or 3, characterized in that: The method further comprises: In response to a first operation of inputting text content in a text input box of a display interface, displaying a category description text of at least one second category identifier associated with the text content at a second associated position of the text input box; wherein the at least one second category identifier is related to the text content; In response to a triggering operation on the category description text, using a third category identifier corresponding to the triggered category description text as prompt content in the first interaction prompt information; The category information of the second media data is related to the third category identifier.

5. The method according to claim 1, characterized in that The generating first media data according to the first interaction prompt information includes: The first interaction prompt information is input into a first content generation model to generate media data, and first media data associated with the content of the first interaction prompt information is output.

6. The method according to claim 1 or 5, characterized in that: In the process of generating the first media data according to the first interaction prompt information, the method further includes: The generation progress of the first media data based on the first interaction prompt information and the media data corresponding to the generation progress are displayed in the display interface.

7. The method according to claim 1, characterized in that In the process of generating the first media data, the method further includes: Using the first interaction prompt information as the historical conversation content in the display interface, and displaying the first preset description information in the text input box and adjusting the display content in the target control to the second preset description information; The first preset description information is used to represent the generation state of the first media data, and the second preset description information is used to represent that after the target control is triggered, the generation of the first media data stops.

8. The method according to claim 7, characterized in that The method further comprises: In response to a triggering operation on a target control whose description information is the second preset description information, the historical conversation content is adjusted to an editable state, so as to generate the first interaction prompt information based on the historical conversation content in the editable state.

9. The method according to claim 1, characterized in that: The second interaction prompt information is determined based on at least one of the following methods: generating the second interaction prompt information based on the edit content corresponding to at least one edit item presented in the display interface, wherein the edit content is supplementary information of the first interaction prompt information; generating the second interaction prompt information based on the newly added text information in the display interface, wherein the display interface includes the first interaction prompt information and the first media data; generating the second interaction prompt information based on device motion information of at least one second device associated with the first device that produces the second media data; The second interaction prompt information is generated based on the drawing information on the first media data.

10. The method according to claim 9, characterized in that The at least one editing item includes a first object editing item, and the generating the second interaction prompt information based on the editing content corresponding to the at least one editing item presented in the display interface includes: In response to a triggering operation on the first object editing item in the display interface, a first object editing panel is displayed; wherein the first object editing panel includes a plurality of element editing fields; Based on the edited content of the element edit field, the first object supplementary information in the second interaction prompt information is generated.

11. The method according to claim 9 or 10, characterized in that: The at least one editing item includes a second object editing item, and the generating the second interaction prompt information based on the editing content corresponding to the at least one editing item presented in the display interface includes: In response to a triggering operation on the second object editing item in the display interface, displaying a second object editing panel, wherein an element editing field in the second object editing panel is different from an element editing field in the first object editing panel, and the second object editing item is a scene editing item; The scene supplementary information in the second interaction prompt information is generated based on the editing content corresponding to at least one element editing field in the second object editing panel.

12. The method according to claim 1, characterized in that The generating the second media data according to the second interaction prompt information and the first media data includes: The second interaction prompt information and the first media data are input into a second content generation model to generate media data, and second media data is output.

13. The method according to claim 12, characterized in that The second interaction prompt information is device motion information, the second media data is video stream data, and the second media data is video stream data obtained by processing the first media data according to the device motion information.

14. The method according to claim 12, characterized in that The second interactive prompt information is drawing information on the first media data, the second media data is video stream data, and the second media data is video stream data of a second picture generated by drawing the drawing information on the first media data and generating the drawing information.

15. The method according to claim 12, characterized in that The second interaction prompt information is the newly added text information, and the second media data is a data stream type and a media content related to the semantic information of the newly added text information.

16. The method according to claim 1, characterized in that Also includes: generating a fifth text associated with the second media data and audio information corresponding to the fifth text; Based on the fifth text and the audio information, the second media data is updated.

17. The method according to claim 1 or 16, characterized in that The method further comprises: In response to an event that the second media data is video stream data, displaying a video track and an audio track corresponding to the second media data; The audio track and the video track are aligned in play time, the audio track is used to display audio information, and the video track includes a plurality of video clips, and the plurality of video clips are determined according to preset clip division conditions.

18. The method according to claim 17, characterized in that The method further comprises: In response to a preset interactive operation on any video segment, displaying a content editing control at a segment display position of the video segment generating the preset interactive operation; The content editing control is used to re-edit the segment content corresponding to the video segment or the second media data after being triggered.

19. The method according to claim 1, characterized in that In the process of displaying the second media data, the method further includes: In response to a triggering operation on a display screen of the second media data, presenting a screen adjustment area and a screen adjustment dialog box in the display screen; Based on the picture adjustment content input in the picture adjustment dialog box, the objects in the picture adjustment area are adjusted to update the second media data.

20. The method according to claim 19, characterized in that The presenting of the picture adjustment area in the display picture in response to the triggering operation on the display picture of the second media data includes: In response to a triggering operation of performing intelligent content selection on a display picture of the second media data, processing the display picture based on a region selection model, and outputting a display picture with the picture adjustment region marked; At least one area adjustment option is displayed at a preset position, wherein the at least one area adjustment option is used to adjust the image adjustment area after being triggered.

21. The method according to claim 1, characterized in that The method further comprises: Displaying the acquired first video and fourth media data associated with the first video, wherein the first video includes a preset object, and the fourth media data is part of the second media data and includes a third object associated with the preset object; In response to a triggering operation on applying the first video, the third object is updated based on the preset object, and the fourth media data is updated.

22. The method according to claim 1, characterized in that The method further comprises: In response to the trajectory drawing operation on the display picture corresponding to the second media data, the fourth object is displayed in the display picture moving according to the drawn trajectory.

23. A media data generating device, characterized in that: include: A first interaction prompt information generating module, used for generating first interaction prompt information in response to a first operation; A first media data generating module, configured to generate and display first media data according to the first interaction prompt information; The second media data generating module is used to obtain second interaction prompt information, and generate and display second media data according to the second interaction prompt information and the first media data.

24. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the media data generating method as described in any one of claims 1-22.

25. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the media data generating method according to any one of claims 1 to 22.

26. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the media data generating method according to any one of claims 1 to 22.

Images