Methods, apparatuses, devices, and products for generating video

Abstract

The present disclosure relates to methods, apparatuses, devices and products for generating a video. The method includes determining search text in a search box. The method also includes displaying search process indication information of a search process, the search process being configured as a process of determining a second material set and a third material set based on the search text and a first material set, the second material set being a subset of the first material set, the third material set being a subset of the second material set. The method further includes displaying at least a portion of materials in the second material set and at least a portion of materials in the third material set. In addition, the method also includes generating a video based on the at least a portion of materials in the second material set and / or the at least a portion of materials in the third material set.

Description

Technical Field

[0001] This disclosure relates to the field of video, and more specifically to methods, apparatus, devices and products for generating video. Background Technology

[0002] During the video creation process, users can select their favorite pictures, videos, or live photos from their album as the base material for the video. After selecting these materials, users can sort them so that they are presented in the desired order in the generated video. In addition, users can also add appropriate background music, transition effects, and filters to these materials, giving the generated video a rich audiovisual experience. Summary of the Invention

[0003] In a first aspect of the embodiments of this disclosure, a method for generating a video is provided. The method includes determining search text in a search box. The method also includes displaying search process indication information, the search process being configured to determine a second set of materials and a third set of materials based on the search text and a first set of materials, the second set of materials being a subset of the first set of materials, and the third set of materials being a subset of the second set of materials. The method further includes displaying at least a portion of the materials in the second set of materials and at least a portion of the materials in the third set of materials. Furthermore, the method also includes generating a video based on at least a portion of the materials in the second set of materials and / or at least a portion of the materials in the third set of materials.

[0004] In a second aspect of the embodiments of this disclosure, an apparatus for generating video is provided. The apparatus includes a search text determination module configured to determine search text in a search box. The apparatus also includes a process information display module configured to display search process indication information, wherein the search process is configured to determine a second set of materials and a third set of materials based on the search text and a first set of materials, wherein the second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials. The apparatus further includes a material set display module configured to display at least a portion of the materials in the second set of materials and at least a portion of the materials in the third set of materials. Furthermore, the apparatus includes a video generation module configured to generate video based on at least a portion of the materials in the second set of materials and / or at least a portion of the materials in the third set of materials.

[0005] In a third aspect of the embodiments of this disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method of the first aspect.

[0006] In a fourth aspect of the embodiments of this disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to implement the method of the first aspect.

[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or principal features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure may be implemented is shown;

[0010] Figure 2 A flowchart of a method for generating video according to some embodiments of the present disclosure is shown;

[0011] Figures 3A to 3D A schematic diagram illustrating an example interactive process of a user actively searching for materials according to some embodiments of the present disclosure is shown;

[0012] Figures 4A to 4E A schematic diagram illustrating an example interactive process from previewing materials to publishing video, according to some embodiments of this disclosure;

[0013] Figures 5A to 5B A schematic diagram illustrating an example interactive process for generating video by selecting a pre-clustered set of materials, according to some embodiments of the present disclosure;

[0014] Figures 6A to 6E A schematic diagram illustrating an example interactive process for creating a video with the same theme as the video being played, according to some embodiments of this disclosure;

[0015] Figures 7A to 7D A schematic diagram illustrating an example of adjusting the size of an area for displaying search process instructions on a search results page, according to some embodiments of the present disclosure;

[0016] Figure 8 A block diagram of an apparatus for generating video according to some embodiments of the present disclosure is shown; and

[0017] Figure 9A block diagram of a device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0018] It is understood that all user-related data involved in this technical solution should be obtained and used only after authorization from the user. This means that if it is necessary to use a user's personal information in this technical solution, the user's explicit consent and authorization are required before obtaining this data; otherwise, no related data collection and use will be carried out. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly followed in the process of data collection, use, and storage, and necessary technical measures should be taken to protect user data security and ensure the secure use of data.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0023] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0024] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0025] As mentioned above, during the video creation process, users can select their favorite pictures, videos, or live photos from their albums as the basic materials for their videos. Then, users can generate videos based on the selected materials. For example, during graduation season, a user might see videos posted by other users commemorating their campus life while browsing video apps. Inspired by these videos, the user might want to create a video about their own campus life. The user can navigate to the video creation module within the app. In this module, the user can select photos or videos taken with classmates at school from their personal albums as materials for their video creation. Then, the user can perform a series of operations such as adjusting the order of the materials, setting background music, and setting transition effects to generate a video themed "Graduation Season."

[0026] However, manually selecting materials from photo albums has several drawbacks, especially when there are many materials and the user lacks editing experience. Albums often contain a large number of photos and videos, covering multiple scenes and time periods. To find materials that fit the "graduation season" theme, users need to browse through the album one by one, filtering for materials related to campus life. This process consumes a lot of time and energy, easily leading to fatigue and even missing memorable moments. Furthermore, after initially selecting a batch of materials that fit the theme, users often face "selection difficulties." The selected materials may all carry important memories, making further selection and simplification difficult, resulting in overly long videos. In addition, since most users lack professional photography and video production knowledge, the quality of the selected materials is often poor. Moreover, selecting materials without an overall concept and coordination leads to inconsistent visual styles in the final video.

[0027] To this end, this disclosure proposes a scheme for generating video. In this scheme, a processing device can determine the search text in a search box. Then, the processing device can display search process instruction information, the search process being configured to determine a second set of materials and a third set of materials based on the search text and a first set of materials, wherein the second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials. Furthermore, the processing device can display at least a portion of the materials in the second set of materials and at least a portion of the materials in the third set of materials. Then, the processing device can generate a video based on at least a portion of the materials in the second set of materials and / or at least a portion of the materials in the third set of materials.

[0028] In this way, the processing device can automatically search for materials related to the search text from a large initial set of materials, and further filter for higher-quality materials. This reduces the time and effort users spend browsing and filtering materials one by one in the initial set, improving the efficiency of material selection. Furthermore, the second and third sets of materials filtered by the search text have a high degree of thematic consistency, reducing the potential for inconsistencies in theme or style during manual selection, thus improving the quality of the generated video. In addition, by displaying search progress indicators, users can clearly perceive the status and progress of the search, enhancing their trust in the search results and reducing negative emotions caused by waiting.

[0029] Figure 1 A schematic diagram of an example environment 100 in which various embodiments of this disclosure may be implemented is shown. For example... Figure 1 As shown, environment 100 includes processing device 102. Processing device 102 can be any device with processing capabilities. For example, processing device 102 can be a smartphone, tablet computer, laptop computer, desktop computer, workstation, or smart wearable device. In environment 100, processing device 102 can run application 104. Users can interact with application 104 through processing device 102. In application 104, users can create videos using materials such as pictures, videos, or live photos. For example, application 104 can be a content publishing platform, video editing platform, or social platform.

[0030] In environment 100, a user can initiate a request to create a video. For example, application 104 can provide a component on the user interface for creating a new video. Clicking this component triggers the request to create a video. Upon receiving the request to create a video, application 104 can obtain a media set 106 (also referred to herein as the first media set). Media set 106 may include multiple media sets 108-1, 108-2, 108-3, 108-4, and 108-5 (collectively referred to as media set 108). Media set 108 may include images, videos, or live photos. For example, media set 106 may be a photo album (e.g., a local album or a cloud album) that the user has authorized application 104 to access, containing images, videos, and live photos. Upon receiving the request to create a video, application 104 can request authorization from the user to access the local album of processing device 102 or the user's cloud album. After obtaining authorization, application 104 can obtain media set 106 from the local album or the cloud album. It should be understood that, for the sake of brevity, only a specific number of materials 108 are shown in environment 100. However, in other implementations, material set 106 may include any number of materials 108.

[0031] In environment 100, after receiving a user's request to create a video, application 104 can determine the search text 112 in search box 110. Search text 112 can be used to search for content in the media collection 106. Search text 112 can be the theme of the video to be created. For example, search text 112 could be "graduation season," "travel diary," "baby growth diary," etc. Search text 112 in search box 110 can be text actively entered by the user, or text automatically filled into search box 110 by the application in response to a specific user interaction. For example, in some implementations, application 104 can display multiple media 108 from media collection 106 and a search box 110 for text input on a media selection page (also referred to herein as the third page). The user can search for media with specific content in media collection 106 by entering search text 112 in search box 110. In some implementations, application 104 can obtain search text 112 through clickable components associated with a specific theme. For example, clickable components can include tag components, topic components, challenge components, etc. For example, when browsing videos posted on application 104, a user may discover videos about graduation season posted by other users. A "Graduation Season" tag may be displayed on the media playback page (also known as the second page) of that video. Users can create graduation-themed videos by clicking the "Graduation Season" tag. Application 104 can retrieve search text 112 related to the topic based on the user clicking the "Graduation Season" tag, without requiring the user to manually enter the search text 112 in the search box 110.

[0032] In environment 100, application 104 can perform a search process for material set 106. During this search process, application 104 can determine material set 114 (also referred to herein as a second material set) and material set 116 (also referred to herein as a third material set) based on material set 106 and search text 112. Material set 114 is a subset of material set 106 and can represent all search results for search text 112. In the example shown in environment 100, material set 114 may include material 108-1, material 108-3, and material 108-5. Material set 116 is a subset of material set 114 and can represent materials further filtered from material set 116. In the example shown in environment 100, material set 116 may include material 108-3 and 108-5. During the search process, application 104 can display a search results page (also referred to as a first page). After determining material set 114, application 104 can display at least a portion of the materials in material set 114 on the search results page. Furthermore, during the search process, while application 104 is determining the material set 116, application 104 can also display search process indication information 118, which can include the status of the search process (e.g., 56 search results found) or the action being performed (e.g., further filtering of highlights). This allows users to clearly perceive the status and progress of the search process, thereby increasing trust in the search results and reducing negative emotions caused by waiting. After determining the material set 116, application 104 can display at least a portion of the materials in the material set 116 on the search results page.

[0033] In environment 100, in some implementations, application 104 can search for content set 106 based on search text 112 and determine all search results as content set 114. In other implementations, application 104 can input content set 106 and search text 112 into a recommendation model. The recommendation model can recall multiple top-ranked content items from content set 106 based on search text 112. Application 104 can then determine the recalled content items as content set 114.

[0034] In environment 100, application 104 can utilize techniques such as image processing, natural language processing, or multimodal retrieval to search for material set 114 based on search text 112. In some implementations, application 104 can utilize natural language processing techniques to determine the semantics of search text 112. Furthermore, application 104 can utilize image recognition techniques to identify entities in material 108. Then, application 104 can utilize multimodal retrieval techniques to identify entities in material that match the semantics of search text 112, and determine search results based on this match. In other implementations, application 104 can utilize an image captioning model to generate captions for material 108. Then, application 104 can determine search results based on the match between search text 112 and the captions of material 108. In some implementations, application 104 can input material set 106 and search text 112 into a specially trained multimodal model to determine search results.

[0035] In environment 100, in some implementations, application 104 can filter for high-quality content or content featuring memorable moments based on content set 116. In other implementations, application 104 can input content set 114 and search text 112 into a recommendation model. The recommendation model can recall multiple top-ranked content pieces from content set 114 based on search text 112. Then, application 104 can define the recalled content pieces as content set 116.

[0036] In environment 100, application 104 can generate video 120 based on at least a portion of the footage in footage set 114 and / or at least a portion of the footage in footage set 116. In some implementations, application 104 can generate video 120 (also referred to herein as a second video) based on all the footage in footage set 116 after determining footage set 116. In other implementations, application 104 can display footage from footage set 116 on a search results page. The user can then add or delete footage based on footage set 116. Then, application 104 can generate video 120 based on the adjusted footage set 116. In still other implementations, the user can ignore the footage set 116 filtered by application 104 and select one or more footage from footage set 114 containing more search results for use in generating video 120.

[0037] In some implementations, when generating video 120, application 104 can utilize a pre-configured template to generate video 120 based on the media set 116. For example, the template can be configured with background music, media transition rhythms, transition effects, filters, etc. Application 104 can generate video 120 by applying this template to the media set 116. In some implementations, when generating video 120, application 104 can input the media set 116 and the video's theme (e.g., graduation season) into a multimodal model. The multimodal model can generate video 120 based on the media in the media set 116 and the input theme. For example, the generated video 120 may include generated narration, subtitles, etc. Compared to generating video using a pre-configured template, generating video using a multimodal model can improve the storytelling of the generated video.

[0038] In this way, the processing device 102 can automatically search for materials related to the search text 112 from the material set 106, which contains a large amount of material, and further filter out materials with higher quality. This reduces the time and effort spent by the user browsing and filtering materials one by one in the material set 106, and improves the efficiency of material filtering. In addition, the material set 114 and material set 116 filtered by the search text 112 have a high degree of thematic consistency, which can reduce the problems of thematic inconsistency or material style inconsistency that may occur during manual selection, and improve the effect of the generated video 120. Furthermore, by displaying search process instruction information 118 during the search process, the user can clearly perceive the status and progress of the search process, thereby enhancing trust in the search results and reducing the negative emotions caused by the waiting process.

[0039] Figure 2 A flowchart of a method 200 for generating video according to some embodiments of the present disclosure is shown. Method 200 can be performed by an apparatus for generating video, which may be, for example, a standalone device or system. The apparatus can be implemented in software and / or hardware. The method 200 will now be illustrated schematically using an apparatus for generating video as an example. Method 200 includes blocks 202, 204, 206, and 208.

[0040] like Figure 2 As shown, in box 202, the search text in the search box is defined. For example, in... Figure 1In the environment 100 shown, application 104 can determine the search text 112 in search box 110, which can be used to search for materials in material collection 106. Search text 112 can be the theme of the video to be created. For example, search text 112 could be "graduation season," "travel diary," "baby growth diary," etc. Search text 112 in search box 110 can be text actively entered by the user, or it can be text automatically filled into search box 110 by the application in response to a specific user interaction.

[0041] In box 204, search process instructions are displayed. The search process is configured to determine a second and a third set of materials based on the search text and a first set of materials. The second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials. For example, in... Figure 1 In the environment 100 shown, application 104 can perform a search process for material set 106. During this search process, application 104 can determine material sets 114 and 116 based on material set 106 and search text 112. Material set 114 is a subset of material set 106 and can represent all search results for search text 112. Material set 116 is a subset of material set 114 and can represent materials further filtered based on material set 116. In the example shown in environment 100, material set 116 may include materials 108-3 and 108-5. During the search process, application 104 can display a search results page. While application 104 is determining material set 116, application 104 can display search process indication information 118, which may include the status of the search process (e.g., 56 search results found) or the action being performed (e.g., further filtering of highlights).

[0042] Box 206 displays at least a portion of the materials from the second material set and at least a portion of the materials from the third material set. For example, in... Figure 1 In the environment 100 shown, during the search process, after determining the material set 114, application 104 can display at least a portion of the materials in material set 114 on the search results page. After determining the material set 116, application 104 can display at least a portion of the materials in material set 116 on the search results page.

[0043] In box 208, a video is generated based on at least a portion of the footage from the second footage set and / or at least a portion of the footage from the third footage set. For example, in... Figure 1In the illustrated environment 100, application 104 can generate video 120 based on at least a portion of the footage in footage set 114 and / or at least a portion of the footage in footage set 116. In some implementations, application 104 can generate video 120 based on all the footage in footage set 116 after determining footage set 116. In other implementations, application 104 can display footage from footage set 116 on a search results page. The user can then add or delete footage based on footage set 116. Application 104 can then generate video 120 based on the adjusted footage set 116. In yet another implementation, the user can ignore the footage set 116 filtered by application 104 and instead select one or more footage from footage set 114 containing more search results for use in generating video 120.

[0044] In this way, the processing device can automatically search for or recommend materials related to the search text from a first set of materials containing a large amount of content. This reduces the time and effort users spend browsing and filtering materials one by one in the first set, improving the efficiency of material selection. Furthermore, the second set of materials filtered by the search text has a high degree of thematic consistency, thus reducing the potential for inconsistencies in theme or style that may occur during manual selection, and improving the quality of the generated video.

[0045] In some embodiments, before the search process ends, the processing device may display search process instruction information and at least a portion of the materials in the second material set on the first page. After the search process ends, the processing device may display at least a portion of the materials in the second material set and at least a portion of the materials in the third material set on the first page.

[0046] In some embodiments, when generating a first video based on at least a portion of materials from a second material set and / or at least a portion of materials from a third material set, in response to an interactive operation instructing the generation of a video based on the third material set, the processing device may display a fifth page, on which the third material set is displayed. In response to a user's interactive operation regarding the first material in the third material set, the processing device may determine the fifth material set by deleting the first material from the third material set. The processing device may then generate the first video based on the fifth material set.

[0047] Figures 3A to 3D A schematic diagram illustrates an example interactive process of a user actively searching for materials according to some embodiments of this disclosure. Figure 3A The material selection page 300 is shown. Figure 3B The image shows a material search page 310 that appears in response to the user clicking the search box. Figure 3CThe search results page 320 is shown in response to the user clicking the search button. Figure 3D The search results page 330 is shown in response to the determination of the third set of materials.

[0048] like Figure 3A As shown, when the application receives a user's request to create a video, it can display a media selection page. For example, a user can click on the video creation component displayed in the application while browsing videos. The application can then display a shooting page for the user to shoot a new video. An album component can be displayed on the shooting page. When a user wants to use existing photos or videos from their album to create a video, they can click on the album component to enter the media selection page 300. After the user clicks the album component, the application can ask the user for authorization to access data from the device's local album. After obtaining the user's authorization, the application can access the media collection 304 from the device's local album (e.g., ...). Figure 1 The media collection 304 (106) is displayed on the media selection page 300, allowing users to select media from the media collection 304 for video creation. Additionally, a search box 302 may be displayed on the media selection page 300. Tooltip text may be displayed around the search box 302 (e.g., inside, above, or below the search box 302) to prompt users to search for media on a specific topic within the media collection 304 using the search box 302.

[0049] After the user clicks the search box 302, the application can display something like this. Figure 3B The resource search page 310 is shown. On the resource search page 310, the search box 302 is switched to an input state, and a keyboard component 312 can be displayed at the bottom of the page. Users can use the keyboard component 312 to enter search text in the search box 302. For example, in... Figure 3B In the example shown, a user can enter the text "graduation season" in search box 302 to search for materials related to graduation season. After the user enters the search text, the application can execute the search process. During the search process, the application can base its search on... Figure 3A The collection of materials in 304 and Figure 3B The search text entered in search box 302 determines all search results and the filtered results based on all search results.

[0050] like Figure 3C As shown, after a user enters the text "graduation season" in search box 302 and initiates a search, the application can display the search results page 320. Search results page 320 includes areas 322 and 324. Area 324 can display all search results, i.e., the material collection 326 (e.g., Figure 1The materials in material set 326 (e.g., group photos with classmates, graduation photos, videos of campus life) can be examples of such materials. However, the number of materials in material set 326 may still be quite large. If a video is generated directly based on all the materials in material set 326, the generated video may be too large or the generation may fail. Therefore, the application can further filter the materials in material set 326 to determine high-quality materials. High-quality materials may include, for example, materials with good composition, materials with high resolution, or materials that include memorable moments. Before the filtering is completed, area 322 can display search process indication information. The search process indication information may include the status of the search process and the actions being performed. For example, the search process indication information may include the number of all search results found based on the search text, and that memorable moments are being filtered.

[0051] After the search process is complete, the application can display something like this: Figure 3D The search results page 330 is shown. In search results page 330, the search process instructions are no longer displayed in area 322. For example... Figure 3D As shown, in the search results page 330, area 322 can display a filtered set of materials 334, which is a subset of material set 326. For example, material set 334 may include high-quality materials from material set 326. Furthermore, area 322 can also display the number of materials in material set 334. Additionally, area 322 can also display component 336. Upon receiving a user interaction with component 336 or with the area corresponding to material set 334, the application can display a material preview page. The material preview page can zoom in on each material used to generate the video, allowing the user to confirm the materials. Furthermore, on the material preview page, the user can further manually filter material set 334 by deleting one or more materials. This improves the quality of the generated video.

[0052] In addition, users can manually select one or more clips from clip set 326 in area 324. Then, users can click the corresponding component to generate a video based on the selected clips from clip set 326. In this case, the application can ignore the clips in the previously automatically filtered clip set 334, and the selected clips from clip set 326 can be displayed on the clip preview page.

[0053] In some embodiments, the first set of materials includes at least one of the following: images, videos, or live photographs. In some embodiments, when generating a video based on at least a portion of the materials in the second set of materials, the processing device may utilize a multimodal model to generate the video based on at least a portion of the materials in the second set of materials, wherein the video includes at least one of the following generated by the multimodal model: background music, subtitles, or narration describing at least a portion of the materials in the second set of materials. Compared to generating videos using pre-configured templates, multimodal models can dynamically generate music, subtitles, or narration that matches the materials based on the semantics of the materials, thereby improving the generated video's effectiveness in terms of emotional rendering, content understanding, and storytelling.

[0054] Figures 4A to 4E A schematic diagram illustrates an example interactive process from previewing materials to publishing video, according to some embodiments of this disclosure. Figure 4A The preview page 400 of the material is shown. Figure 4B A video generation page 420 is shown, which appears in response to determining the material to be used to generate the video. Figure 4C The video editing page 430, which appears in response to the completion of video generation, is shown. Figure 4D The video publishing page 440 is shown in response to the end of video editing. Figure 4E The media content playback page 450, which displays the published videos, is shown.

[0055] like Figure 4A As shown, the media preview page includes areas 402, 404, and 406. Area 402 can display the theme of the video to be generated. For example, the theme could be the search text in the search box mentioned above. Area 406 can display the media set 408 used to generate the video; media set 408 is displayed as thumbnails. Media set 408 could, for example, be a media set filtered by the application on the search results page (e.g., Figure 3D The number of materials in material set 408 can be displayed in area 402. When a user selects a material in material set 408 (e.g., material 410), a larger image (e.g., the original image) of the selected material can be displayed in area 404 for the user to view the details of the material. In area 406, a delete component can be displayed near each material in material set 408 to remove the corresponding material from material set 408. For example, when the user clicks the delete component corresponding to material 410, material 410 can be removed from material set 408. Then, after completing the manual material selection, the user can generate a video by clicking component 412.

[0056] After receiving the user's interaction with component 412, the application can display as follows: Figure 4B The video generation page 420 is shown. Video generation page 420 includes region 422. The progress of the video generation process can be displayed in region 422. During video generation, the application can generate a video using a multimodal model based on the set of materials determined in the material preview page 400 (also referred to herein as the fifth material set). The generated video includes at least one of the following generated by the multimodal model: background music, subtitles, or narration describing at least a portion of the materials in the second material set. The multimodal model can recognize information such as scenes (e.g., classroom, playground, etc.), people (e.g., students), actions (e.g., hugging, taking photos), temporal sequence, and emotions (e.g., laughter, crying) in the input material set. For video materials, the multimodal model can also extract keyframes, audio content, and event segments. The multimodal model can encode the recognized information into a unified multimodal vector representation. Then, the multimodal model can determine the semantic relationships and temporal logic between these vector representations. Then, the multimodal model can generate coherent narrative narration text based on the semantic relationships and temporal logic. The narration text can describe the relationship between multiple materials.

[0057] After the video is generated, the application can display something like this. Figure 4C The video editing page 430 is shown. The generated video 432 can be displayed on the video editing page 430. Users can view the effect of video 432 on the video editing page 430 and adjust video 432 using various video editing components. For example, the video editing page 430 provides an editing component 434, which allows for functions such as adding text, adding stickers, adding filters, and adding music.

[0058] After completing video editing, users can click component 438 to confirm the completion of editing and proceed to the next step. Figure 4D The video publishing page 440 is shown. Edited videos can be displayed in area 442 of the video publishing page 440. Additionally, the application can generate a tag 446 and display tag 446 in area 444. Tag 446 can include the features used to generate the video (e.g., AI-generated content) and the video's theme (e.g., graduation season). In this way, the video will be associated with tag 446.

[0059] After confirming the posting information, users can publish the video to the application for other users to view by clicking the posting component 448. Other users can then use methods such as... Figure 4EThe video is viewed on media content playback page 450. The video and its tag 446 can be displayed in area 452 of media content playback page 450. In this way, tag 446 can guide the user watching the video to use the "AI-generated video" function to create a new video on the graduation season theme. Furthermore, other users who see tag 446 can understand that they can easily create similar videos using the "AI-generated video" function. In addition, the theme in tag 446 allows other users to understand that the current video is related to existing themes, thus allowing them to infer that many users have already created videos related to this theme, and thus hoping to create videos on the same theme to participate in group activities.

[0060] In scenarios where users actively input search text, they may not be clear on how to input the accurate search text. To simplify the user's workflow and provide inspiration for video topics, the application can display clustered materials on the material search page for users to choose from.

[0061] In some embodiments, the processing device may display a search box and a fourth set of materials on a fourth page, each of the fourth set including different material themes. The processing device may obtain search text entered by the user via the search box on the fourth page. In some embodiments, in response to user interaction with the fourth set of materials, the processing device may generate a video based on at least a portion of the materials in the fourth set.

[0062] Figures 5A to 5B A schematic diagram illustrates an example interactive process for generating video by selecting a pre-clustered set of materials, according to some embodiments of the present disclosure. Figure 5A The image shows a material search page 500 displaying multiple predefined topics (the material search page is also referred to as the fourth page in this document). Figure 5B A preview page 510 of the materials is shown, which is displayed in response to the user's interactive actions on a predetermined theme.

[0063] like Figure 5AAs shown, the content search page 500 includes a search box 502 and a keyboard component 509. Users can use the keyboard component 509 to enter search text in the search box 502 to search for content related to the search text within the content collection. Furthermore, multiple content collections can be displayed in area 504 of the content search page 500. Each of these content collections corresponds to a different theme. For example, multiple content collections may include a content collection 506 with the theme "Night Streets," a content collection 508 with the theme "Baby Growth," etc. (In this document, content collections 506, 508, etc., are collectively referred to as the fourth content collection). The content in content collection 506 is associated with the theme "Night Streets," and the content in content collection 508 is associated with the theme "Baby Growth." The multiple content collections in area 504 can be determined by performing clustering on the content. For example, the application can perform a clustering algorithm on all content to generate multiple content collections. Then, the application can determine the theme of a content collection based on the content within it. In some implementations, the clustering process can be performed in advance so that the application can read the pre-clustered set of materials after entering the search results page 500, thereby reducing the latency for rendering these sets of materials.

[0064] In the resource search page 500, because the resource collection in area 504 includes clustered resources, users can directly access the resource preview page by clicking the area corresponding to the resource collection, without having to go to the search results page. For example, when a user clicks on resource collection 506 (e.g., a resource collection with the theme "Night Street"), the application can display something like this: Figure 5B The preview page 510 shows the material preview. In area 512 of the preview page 510, the theme of the material collection (e.g., "Night Street") can be displayed. The materials in the material collection can be displayed in area 514. When material 516 is selected, it can be enlarged in area 518. Each material in the material collection has a delete component nearby for removing it from the collection. For example, when the user clicks the delete component corresponding to material 516, material 516 can be removed from the material collection. After completing the manual material selection, the user can access the video generation page by clicking component 520 (e.g., as shown). Figure 4B (The material generation page shown).

[0065] The following describes the video generation process in a follow-up shooting scenario. In some embodiments, the processing device may display a first component associated with the second video on a second page. The first component is used to prompt the user to create a first video based on the theme of the second video. In response to the user's interaction with the first component, the processing device may obtain text associated with the first theme as search text.

[0066] In some embodiments, when obtaining text associated with a first topic as search text, in response to a user's interaction with a first component, the processing device may display detailed information associated with the first topic and a second component, the second component being used to prompt the user to create a new video on the first topic. In response to a user's interaction with the second component, the processing device may obtain text associated with the first topic as search text.

[0067] In some embodiments, in response to a user's interaction with the second component, the processing device may display a third page, which displays at least a portion of the materials in the first material set and a search box. If no user interaction is received, the processing device may display the first page, which displays at least a portion of the materials in the second material set and at least a portion of the materials in the third material set.

[0068] In some embodiments, the processing device may display a third component on the first page, the third component being used to return to the previous page. The processing device may display the first page in response to a user's interaction with the third component.

[0069] Figures 6A to 6E A schematic diagram illustrating an example interactive process for creating a video with the same theme as the video being played, according to some embodiments of this disclosure. Figure 6A The video playback page 600, which displays labels, is shown. Figure 6B A media content playback page 610 is displayed, showing detailed information about the topic. Figure 6C The material selection page 620 is shown. Figure 6D The search results page 630 is shown. Figure 6E The search results page 650 is shown.

[0070] like Figure 6AAs shown, the media content playback page 610 (also referred to herein as the second page) displays video 602 (also referred to herein as the second video). Furthermore, component 604 (also referred to herein as the first component) is displayed in the area associated with video 602. Component 604 may indicate that video 602 was created using a specific function in the application (e.g., the "AI-generated video" function), and that video 602 is associated with a specific theme (e.g., the "graduation season" theme, also referred to herein as the first theme). Component 604 may prompt the user to create the first video based on the theme of the second video. Normally, a user can access the page for creating videos by clicking component 606. However, in the media content playback page 610, the user can be inspired by video 602 and component 604 to click component 604 to create a video with the same theme as video 602 using the same function. Component 604 may be associated with a specific theme. For example, component 604 may be a tag, such as one found on a video publishing page (e.g., Figure 4D This is set in the video publishing page 440 shown. In other implementations, component 604 can also be a topic or challenge, etc.

[0071] When the application receives a user interaction with component 604, it can display detailed information about the topic associated with component 604, such as... Figure 6B As shown. Figure 6B A media content playback page 610 is shown. On the media content playback page 610, video 602 is displayed in a minimized state, and a panel 612 is displayed below video 602. Panel 612 may display the theme name (e.g., "Graduation Season"), images associated with the theme (e.g., images set by the application's operators), the number of users who participated in creating videos on the theme, and a video collection 614 including videos created by other users on the theme. Users can click on videos in video collection 614 to view videos created by other users. In this way, users' interest in creating videos on the theme can be further stimulated. Component 616 (also referred to herein as a second component) may be displayed on the media content playback page 610.

[0072] Upon receiving a user interaction with component 616, the application can identify text associated with that topic (e.g., "graduation season") as search text. The application can then use this search text to search the user's photo album (e.g., ...). Figure 1 The search process described above is automatically executed in the material collection 106, and the results are displayed as shown above. Figure 6D The search results page 630 is shown. Search results page 630 includes a return component 632 (also referred to herein as a third component) for returning to the previous page. However, if directly from... Figure 6BThe media content playback page 610 shown redirects to, as... Figure 6D As shown in the search results page 630, when a user wants to select other materials from the album to create a video, clicking the return component 632 in the search results page 630 will directly return the user to the media content playback page 610, requiring the user to perform multiple interactive operations to find the desired content. Figure 3A The material selection page 300 is shown. Therefore, in some embodiments, upon receiving a user's request... Figure 6B When interacting with component 616 in the application, the application can first display something like this: Figure 6C The material selection page shown is 620. Then, without receiving any user interaction, it automatically redirects to... Figure 6D The search results page shown is 630. Thus, upon receiving user requests... Figure 6D When interacting with the return component 632, it can return and display something like... Figure 6C The material selection page 620 shown allows users to access the material selection page 620 and freely select materials from the album for creating videos with just one operation.

[0073] like Figure 6D As shown, on the search results page 630, the application can automatically fill in search text (e.g., "graduation season") in the search box 634 and display all search results from the photo album based on that search text in area 638, i.e., the material collection 640 (e.g., Figure 1 (The material set 640 is mentioned in the original text). Furthermore, the application can further filter the materials in material set 640 to identify high-quality materials. Before the filtering is complete, area 636 can display search process indication information. This information may include the status of the search process and the actions being performed.

[0074] After the search process is complete, the application can display something like this: Figure 6E The search results page 650 is shown. In search results page 650, the search process instructions are no longer displayed in area 636. For example... Figure 6E As shown, in the search results page 650, area 636 can display a filtered collection of materials 652, which is a subset of material collection 640. For example, material collection 652 may include high-quality materials from material collection 640. Furthermore, area 636 can also display the number of materials in material collection 652. Additionally, area 636 can also display component 654. Upon receiving a user interaction with component 654 or with the area corresponding to material collection 652, the application can display a material preview page.

[0075] In this way, when users discover videos on a specific topic, they can quickly search for materials in their albums by clicking on tags, topics, or challenges on the page. This process eliminates the need for users to think about or manually enter search terms, thus reducing decision-making and operational costs and improving the user experience.

[0076] In some embodiments, search process instructions are displayed in a first area of ​​a first page. During the search process, in response to a user's first touch operation, the processing device may display the search process instructions in a second area of ​​the first page, the second area being smaller than the first area. In some embodiments, during the search process, in response to a user's second touch operation, the processing device may restore the display of the search process instructions in the first area of ​​the first page.

[0077] In some embodiments, at least a portion of the third set of materials is displayed in the second area. In some embodiments, in response to a second touch operation by a user, the processing device may display at least a portion of the materials in the third set of materials in the first area of ​​the first page.

[0078] In some embodiments, in response to a user's first touch operation, the processing device can move a first region upwards on a first page. The processing device can determine the distance between the bottom of the first region and the top of the first page. In response to the distance being less than a predetermined distance threshold, the processing device can form a second region by reducing the height of the first region from a first height to a second height. The processing device can then fix the second region at the top of the first page so that the second region does not move in response to the first touch operation.

[0079] Figures 7A to 7D A schematic diagram illustrating an example of adjusting the size of an area used to display the search process and search results on a search results page, according to some embodiments of the present disclosure. Figure 7A The search results page 700 is shown before the search process ends. Figure 7B The search results page 710 is shown before the search process ends. Figure 7C The search results page 720 shows the results after the search process has ended. Figure 7D The search results page 730 (also referred to as the first page in this document) is shown after the search process has ended.

[0080] like Figure 7A As shown, the search results page 700 includes areas 702, 704, and 706. The search box can be displayed in area 702, showing the search text for the current search query. Figure 7AIn the example shown, the search process has not yet ended, so search process indication information can be displayed in area 704 (also referred to herein as the first area). Figure 1 The search process instruction information 118 in the text). The material collection 708 can be displayed in area 706 (e.g., Figure 1 The material collection in 114). Figure 7A As shown, area 706 only displays a portion of the materials in material set 708. To view other materials in material set 708, the user can perform an upward swipe operation (referred to herein as the first touch operation) on the search results page 700 to move the entire search results page 700 upward, thereby expanding area 706 to display more materials. In some implementations, when the search results page 700 moves upward, area 702 can be fixed at the top of the page, with only areas 704 and 706 moving upward. In some implementations, when the search results page 700 moves upward, areas 702, 704, and 706 can move upward together. However, as the page moves upward, area 704 may be moved out of the effective display area, making it impossible for the user to know the progress of the search process. Therefore, as the page moves upward, the application can determine the distance between the bottom of area 704 and the top of the search results page. If this distance is less than a predetermined threshold, the application can reduce the height of area 704 from a first height to a second height to form a shrunken area. The application can then fix the shrunken area at the top of the page, preventing it from moving upwards in response to the user's swipe-up gesture. It should be noted that in the implementation where area 702 does not move with the page, the top of the page can be defined as the bottom of area 702. Conversely, in the viewpoint where area 702 moves upwards with the page, the top of the page can be defined as the top of area 702.

[0081] In such Figure 7B In the search results page 710 shown, region 702 does not move with the page. Therefore, the application is used to determine... Figure 7AWhen the distance between the top of region 704 and the bottom of region 702 is less than a predetermined threshold, region 714 (also referred to herein as the second region) can be displayed, and region 714 is smaller than region 704. At this time, region 714 is fixed below region 702 and no longer changes position or size in response to the user's upward swipe operation. At least a portion of the search process indication information can be displayed in region 714. In this way, the user can still see at least a portion of the search process indication information while swiping upward on region 706, thereby knowing the progress of the search process. When the user performs a downward swipe operation (also referred to herein as the second touch operation) on the search results page 710, region 706 can move downward. Region 714 can be enlarged until the distance between the top of region 706 (i.e., the top of the first row of materials in all search results) and the bottom of region 714 is less than a predetermined threshold. Figure 7A The application can restore the display of search process instructions in area 704.

[0082] After the search process is complete, the application can display something like this: Figure 7C The search results page 720 is shown. In area 704 of the search results page 720, a filtered collection of materials 722 can be displayed (e.g., Figure 1 (Materials set 116 in the document). Upon receiving an upward swipe gesture from the user on the page, regions 704 and 706 can move upward together. As the page moves upward, the application can determine the distance between the bottom of region 704 and the top of the search results page. If this distance is less than a predetermined threshold, the application can reduce the height of region 704 from a first height to a second height to form a shrunken region. The application can then fix the shrunken region at the top of the page so that it does not move upward in response to the user's upward swipe gesture.

[0083] In such Figure 7D The search results page 730 shown is used in determining... Figure 7C When the distance between the top of region 704 and the bottom of region 702 is less than a predetermined threshold, region 714 can be displayed, and region 714 is smaller than region 704. At this time, region 714 is fixed below region 702 and no longer changes position or size in response to the user's upward swipe. At least a portion of the selected materials from the set 722 can be displayed in region 714. When the user clicks on a material in region 714 or the "AI Finished Footage" component, the application can display a material preview page (e.g., as shown in the image). Figure 4AThe preview page 400 shows the materials. When the user swipes down on the search results page 730, area 706 can move down. Area 714 can be zoomed in until the distance between the top of area 706 (i.e., the top of the first row of materials in all search results) and the bottom of area 714 is less than a predetermined threshold. Figure 7A The application can restore at least a portion of the filtered content set 722 to be displayed in area 704. This allows users to see the content filtered by the application while swiping up area 706, enabling them to access the content preview page with a single operation, without needing to perform multiple swipes down to reveal area 714 and then access the content preview page. This reduces the number of operations required for users to access the content preview page, improving the user experience.

[0084] Figure 8 A block diagram of an apparatus 800 for generating video according to some embodiments of the present disclosure is shown. Figure 8 As shown, the device 800 includes a search text determination module 802, configured to determine the search text in the search box. The device 800 also includes a process information display module 804, configured to display search process indication information, wherein the search process is configured to determine a second set of materials and a third set of materials based on the search text and a first set of materials, wherein the second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials. The device 800 also includes a material set display module 806, configured to display at least a portion of the materials in the second set of materials and at least a portion of the materials in the third set of materials. Furthermore, the device 800 includes a video generation module 806, configured to generate a video based on at least a portion of the materials in the second set of materials and / or at least a portion of the materials in the third set of materials.

[0085] Figure 9 A block diagram of a device 900 capable of implementing various embodiments of the present disclosure is shown. (See diagram for example.) Figure 9 As shown, device 900 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 901, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 902 or loaded from storage unit 908 into random access memory (RAM) 903. The RAM 903 can also store various programs and data required for the operation of device 900. The CPU / GPU 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904. Although not shown in... Figure 9 As shown, device 900 may also include a coprocessor.

[0086] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] The various methods or processes described above can be executed by CPU / GPU 901. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by CPU / GPU 901, one or more steps or actions in the methods or processes described above can be performed.

[0088] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0089] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0090] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0091] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​and conventional procedural programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0092] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0093] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0096] The following are some example implementations of this disclosure.

[0097] Example 1. A method for generating a video, comprising:

[0098] Define the search text in the search box;

[0099] The search process displays search process instructions, which are configured to determine a second set of materials and a third set of materials based on the search text and a first set of materials, wherein the second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials.

[0100] Display at least a portion of the materials in the second material set and at least a portion of the materials in the third material set; and

[0101] A first video is generated based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set.

[0102] Example 2. According to the method of Example 1, wherein the search process indication information is displayed in a first area of ​​a first page, and the method further includes: during the search process, in response to a first touch operation by a user, displaying the search process indication information in a second area of ​​the first page, the second area being smaller than the first area.

[0103] Example 3. The method according to Examples 1-2 further includes: during the search process, in response to a second touch operation by the user, restoring the display of the search process indication information in the first area of ​​the first page.

[0104] Example 4. The method according to any one of Examples 1-3, wherein at least a portion of the third set of materials is displayed in the second area.

[0105] Example 5. The method according to any one of Examples 1-4 further includes: in response to a second touch operation by a user, displaying at least a portion of the materials in the third material set in the first area of ​​the first page.

[0106] Example 6. The method according to any one of Examples 1-5, wherein displaying the search process indication information in the second area of ​​the first page in response to the user's first touch operation includes:

[0107] In response to the first touch operation, the first area moves upward on the first page;

[0108] Determine the distance between the bottom of the first area and the top of the first page;

[0109] In response to the distance being less than a predetermined distance threshold, the second region is formed by reducing the height of the first region from a first height to a second height; and

[0110] The second area is fixed at the top of the first page, so that the second area does not move in response to the first touch operation.

[0111] Example 7. The method according to any one of Examples 1-6 further includes:

[0112] The second page displays a first component associated with the second video, which prompts the user to create the first video based on the theme of the second video; and

[0113] In response to a user's interaction with the first component, text associated with the first topic is retrieved as the search text.

[0114] Example 8. The method according to any one of Examples 1-7, wherein in response to a user's interaction with the first component, obtaining the text associated with the first topic as the search text includes:

[0115] In response to a user's interaction with the first component, detailed information associated with the first topic and a second component are displayed, the second component prompting the user to create a new video on the first topic; and

[0116] In response to a user's interaction with the second component, the text associated with the first topic is retrieved as the search text.

[0117] Example 9. The method according to any one of Examples 1-8 further includes:

[0118] In response to a user's interaction with the second component, a third page is displayed, the third page showing at least a portion of the materials in the first material set and the search box; and

[0119] If no user interaction is received, a first page is displayed, which displays at least a portion of the materials in the second material set and at least a portion of the materials in the third material set.

[0120] Example 10. The method according to any one of Examples 1-9 further includes:

[0121] A third component is displayed on the first page, the third component being used to return to the previous page; and

[0122] In response to the user's interaction with the third component, the first page is displayed.

[0123] Example 11. The method according to any one of Examples 1-10 further includes:

[0124] The search box and a fourth set of materials are displayed on the fourth page, each of which includes a different material theme.

[0125] The search text entered by the user is obtained via the search box on the fourth page.

[0126] Example 12. The method according to any one of Examples 1-11 further includes:

[0127] In response to a user's interactive operation on the fourth set of materials, the video is generated based on at least a portion of the materials in the fourth set of materials.

[0128] Example 13. The method according to any one of Examples 1-12 further includes:

[0129] Before the search process ends, the search process instruction information and at least a portion of the materials in the second material set are displayed on the first page; and

[0130] After the search process is completed, at least a portion of the materials in the second material set and at least a portion of the materials in the third material set are displayed on the first page.

[0131] Example 14. The method according to any one of Examples 1-13, wherein generating the first video based on at least a portion of the footage in the second set of footage and / or at least a portion of the footage in the third set of footage includes:

[0132] In response to an interactive operation instructing the generation of the video based on a third set of materials, a fifth page is displayed, on which the third set of materials is displayed;

[0133] In response to a user's interactive action on a first material in the third material set, a fifth material set is determined by deleting the first material from the third material set; and

[0134] The first video is generated based on the fifth set of materials.

[0135] Example 15. The method according to any one of Examples 1-14, wherein generating the first video based on at least a portion of the footage in the second set of footage and / or at least a portion of the footage in the third set of footage includes:

[0136] The first video is generated using a multimodal model based on at least a portion of the materials in the third material set, wherein the first video includes at least one of the following generated by the multimodal model: background music, subtitles, or narration describing the at least a portion of the materials in the second material set.

[0137] Example 16. The method according to any one of Examples 1-15, wherein the first set of materials includes at least one of the following: images, videos, or live photographs.

[0138] Example 17. An apparatus for generating video, comprising:

[0139] The search text determination module is configured to determine the search text in the search box;

[0140] The process information display module is configured to display search process indication information of the search process, wherein the search process is configured to determine a second material set and a third material set based on the search text and a first material set, wherein the second material set is a subset of the first material set and the third material set is a subset of the second material set;

[0141] The material collection display module is configured to display at least a portion of the materials in the second material collection and at least a portion of the materials in the third material collection; and

[0142] The video generation module is configured to generate a first video based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set.

[0143] Example 18. The apparatus according to Example 17, wherein the search process indication information is displayed in a first area of ​​a first page, and the apparatus further includes: a second process information display module configured to, in response to a first touch operation by a user during the search process, display the search process indication information in a second area of ​​the first page, the second area being smaller than the first area.

[0144] Example 19. The apparatus according to Example 17 or 18 further includes: a third process information display module configured to, in response to a second touch operation by a user, resume displaying the search process indication information in the first area of ​​the first page during the search process.

[0145] Example 20. The apparatus according to any one of Examples 17-19, wherein at least a portion of the third material set is displayed in the second region.

[0146] Example 21. The apparatus according to any one of Examples 17-20 further includes: a second touch operation response module configured to display at least a portion of the materials in the third material set in response to a second touch operation by a user in the first area of ​​the first page.

[0147] Example 22. The apparatus according to any one of Examples 17-21, wherein the second touch operation response module comprises:

[0148] The first touch operation response module is configured to respond to the first touch operation and cause the first area to move upward on the first page;

[0149] The distance determination module is configured to determine the distance between the bottom of the first region and the top of the first page;

[0150] A second region forming module is configured to form a second region in response to the distance being less than a predetermined distance threshold by reducing the height of the first region from a first height to a second height; and

[0151] The second area fixing module is configured to fix the second area to the top of the first page, so that the second area does not move in response to the first touch operation.

[0152] Example 23. The apparatus according to any one of Examples 17-22 further includes:

[0153] A first component display module is configured to display a first component associated with the second video on a second page, the first component being used to prompt the user to create the first video based on the theme of the second video; and

[0154] The first component interaction module is configured to, in response to a user's interaction with the first component, obtain text associated with the first topic as the search text.

[0155] Example 24. The apparatus according to any one of Examples 17-23, wherein the first component interaction module comprises:

[0156] The second component display module is configured to, in response to a user's interaction with the first component, display detailed information associated with the first topic and the second component, the second component being used to prompt the user to create a new video on the first topic; and

[0157] The second component interaction module is configured to, in response to a user's interaction with the second component, obtain the text associated with the first topic as the search text.

[0158] Example 25. The apparatus according to any one of Examples 17-24 further includes:

[0159] The third page display module is configured to display a third page in response to user interaction with the second component. The third page displays at least a portion of the materials from the first material set and the search box.

[0160] The first page display module is configured to display a first page when no user interaction is received. The first page displays at least a portion of the materials in the second material set and at least a portion of the materials in the third material set.

[0161] Example 26. The apparatus according to any one of Examples 17-25 further includes:

[0162] A return module is configured to display a third component on the first page, the third component being used to return to the previous page; and

[0163] The third component interaction module is configured to display the first page in response to user interaction with the third component.

[0164] Example 27. The apparatus according to any one of Examples 17-26 further includes:

[0165] The fourth material collection display module is configured to display the search box and the fourth material collection on the fourth page, each of the fourth material collection including a different material theme;

[0166] The search text acquisition module is configured to acquire the search text entered by the user via the search box in the fourth page.

[0167] Example 28. The apparatus according to any one of Examples 17-27 further includes:

[0168] The fourth material set interaction module is configured to generate the video based on at least a portion of the materials in the fourth material set in response to user interaction with the fourth material set.

[0169] Example 29. The apparatus according to any one of Examples 17-28 further includes:

[0170] The second process information display module is configured to display, on a first page, search process instruction information and at least a portion of the materials in the second material set before the search process ends; and

[0171] The filtering results display module is configured to display at least a portion of the materials in the second material set and at least a portion of the materials in the third material set on the first page after the search process has ended.

[0172] Example 30. The apparatus according to any one of Examples 17-29, wherein the video generation module comprises:

[0173] The third material set interaction module is configured to display a fifth page in response to an interactive operation that instructs the generation of the video based on the third material set, wherein the third material set is displayed on the fifth page;

[0174] The third material set deletion module is configured to, in response to a user's interactive operation on a first material in the third material set, determine a fifth material set by deleting the first material from the third material set; and

[0175] The fifth material set module is configured to generate the first video based on the fifth material set.

[0176] Example 31. The apparatus according to any one of Examples 17-30, wherein generating the first video based on at least a portion of the material in the second material set and / or at least a portion of the material in the third material set comprises:

[0177] A multimodal model usage module is configured to generate the first video using a multimodal model based on at least a portion of the materials in the third material set, wherein the first video includes at least one of the following generated by the multimodal model: background music, subtitles, or narration describing the at least a portion of the materials in the second material set.

[0178] Example 32. The apparatus according to any one of Examples 17-31, wherein the first set of materials includes at least one of the following: images, videos, or live photographs.

[0179] Example 33. An electronic device comprising:

[0180] Processor; and

[0181] A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform actions, the actions including:

[0182] Define the search text in the search box;

[0183] The search process displays search process instructions, which are configured to determine a second set of materials and a third set of materials based on the search text and a first set of materials, wherein the second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials.

[0184] Display at least a portion of the materials in the second material set and at least a portion of the materials in the third material set; and

[0185] A first video is generated based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set.

[0186] Example 34. The method according to Example 33, wherein the search process indication information is displayed in a first area of ​​a first page, and the method further includes: during the search process, in response to a first touch operation by a user, displaying the search process indication information in a second area of ​​the first page, the second area being smaller than the first area.

[0187] Example 35. The method according to Examples 33-34 further includes: during the search process, in response to a second touch operation by the user, restoring the display of the search process indication information in the first area of ​​the first page.

[0188] Example 36. The method according to any one of Examples 33-35, wherein at least a portion of the third set of materials is displayed in the second area.

[0189] Example 37. The method according to any one of Examples 33-36 further includes: in response to a second touch operation by a user, displaying at least a portion of the materials in the third material set in the first area of ​​the first page.

[0190] Example 38. The method according to any one of Examples 33-37, wherein displaying the search process indication information in the second area of ​​the first page in response to the user's first touch operation includes:

[0191] In response to the first touch operation, the first area moves upward on the first page;

[0192] Determine the distance between the bottom of the first area and the top of the first page;

[0193] In response to the distance being less than a predetermined distance threshold, the second region is formed by reducing the height of the first region from a first height to a second height; and

[0194] The second area is fixed at the top of the first page, so that the second area does not move in response to the first touch operation.

[0195] Example 39. The method according to any one of Examples 33-38 further includes:

[0196] The second page displays a first component associated with the second video, which prompts the user to create the first video based on the theme of the second video; and

[0197] In response to a user's interaction with the first component, text associated with the first topic is retrieved as the search text.

[0198] Example 40. The method according to any one of Examples 33-39, wherein retrieving the text associated with the first topic as the search text in response to a user interaction with the first component comprises:

[0199] In response to a user's interaction with the first component, detailed information associated with the first topic and a second component are displayed, the second component prompting the user to create a new video on the first topic; and

[0200] In response to a user's interaction with the second component, the text associated with the first topic is retrieved as the search text.

[0201] Example 41. The method according to any one of Examples 33-40 further includes:

[0202] In response to a user's interaction with the second component, a third page is displayed, the third page showing at least a portion of the materials in the first material set and the search box; and

[0203] If no user interaction is received, a first page is displayed, which displays at least a portion of the materials in the second material set and at least a portion of the materials in the third material set.

[0204] Example 42. The method according to any one of Examples 33-41 further includes:

[0205] A third component is displayed on the first page, the third component being used to return to the previous page; and

[0206] In response to the user's interaction with the third component, the first page is displayed.

[0207] Example 43. The method according to any one of Examples 33-42 further includes:

[0208] The search box and a fourth set of materials are displayed on the fourth page, each of which includes a different material theme.

[0209] The search text entered by the user is obtained via the search box on the fourth page.

[0210] Example 44. The method according to any one of Examples 33-43 further includes:

[0211] In response to a user's interactive operation on the fourth set of materials, the video is generated based on at least a portion of the materials in the fourth set of materials.

[0212] Example 45. The method according to any one of Examples 33-44 further includes:

[0213] Before the search process ends, the search process instruction information and at least a portion of the materials in the second material set are displayed on the first page; and

[0214] After the search process is completed, at least a portion of the materials in the second material set and at least a portion of the materials in the third material set are displayed on the first page.

[0215] Example 46. The method according to any one of Examples 33-45, wherein generating the first video based on at least a portion of the footage in the second set of footage and / or at least a portion of the footage in the third set of footage includes:

[0216] In response to an interactive operation instructing the generation of the video based on a third set of materials, a fifth page is displayed, on which the third set of materials is displayed;

[0217] In response to a user's interactive action on a first material in the third material set, a fifth material set is determined by deleting the first material from the third material set; and

[0218] The first video is generated based on the fifth set of materials.

[0219] Example 47. The method according to any one of Examples 33-46, wherein generating the first video based on at least a portion of the footage in the second set of footage and / or at least a portion of the footage in the third set of footage includes:

[0220] The first video is generated using a multimodal model based on at least a portion of the materials in the third material set, wherein the first video includes at least one of the following generated by the multimodal model: background music, subtitles, or narration describing the at least a portion of the materials in the second material set.

[0221] Example 48. The method according to any one of Examples 33-47, wherein the first set of materials includes at least one of the following: images, videos, or live photographs.

[0222] Although this disclosure has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in any of the appended Examples 1-48 is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementation of any of the Examples 1-48.

Claims

1. A method for generating video, comprising: Display a first component associated with the second video, the first component displaying information on a first theme; In response to the user's interaction with the first component, a second component is displayed, which prompts the user to create a new video on the first theme; In response to the user's interaction with the second component, a third page is displayed, which displays the first set of materials; If no user interaction is received with the third page, the first page is displayed. The first page displays search process instructions. The search process is configured to determine a second set of materials and a third set of materials based on the search text and the first set of materials. The search text is associated with the first topic. The second set of materials is a subset of the first set of materials, and the third set of materials is a subset of the second set of materials. Display at least a portion of the materials in the second material set and at least a portion of the materials in the third material set; as well as A first video is generated based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set.

2. The method of claim 1, wherein the search process indication information is displayed in a first area of ​​the first page, and the method further comprises: During the search process, in response to the user's first touch operation, search process instruction information is displayed in a second area of ​​the first page, where the second area is smaller than the first area.

3. The method according to claim 2, further comprising: During the search process, in response to the user's second touch operation, the search process instruction information is restored to be displayed in the first area of ​​the first page.

4. The method of claim 2, wherein at least a portion of the third set of materials is displayed in the second region.

5. The method according to claim 4, further comprising: In response to a second touch operation by the user, at least a portion of the materials in the third material set are displayed in the first area of ​​the first page.

6. The method of claim 2, wherein displaying the search process indication information in the second area of ​​the first page in response to the user's first touch operation comprises: In response to the first touch operation, the first area moves upward on the first page; Determine the distance between the bottom of the first area and the top of the first page; In response to the distance being less than a predetermined distance threshold, the second region is formed by reducing the height of the first region from a first height to a second height; as well as The second area is fixed at the top of the first page, so that the second area does not move in response to the first touch operation.

7. The method of claim 1, wherein displaying the first component associated with the second video comprises: The first component associated with the second video is displayed on the second page.

8. The method according to claim 1, further comprising: In response to user interaction with the first component, display detailed information associated with the first topic; as well as In response to a user's interaction with the second component, text associated with the first topic is retrieved as the search text.

9. The method according to claim 1, wherein the third page further displays a search box.

10. The method according to claim 1, further comprising: A third component is displayed on the first page, and the third component is used to return to the previous page; as well as The third page is displayed in response to the user's interaction with the third component.

11. The method according to claim 1, further comprising: The fourth page displays a search box and a fourth set of materials, each of which includes a different material theme. The search text entered by the user is obtained via the search box on the fourth page.

12. The method of claim 11, further comprising: In response to a user's interactive operation on the fourth set of materials, the video is generated based on at least a portion of the materials in the fourth set of materials.

13. The method according to claim 1, wherein: Before the search process ends, the search process instruction information and at least a portion of the materials in the second material set are displayed on the first page; as well as After the search process is completed, at least a portion of the materials in the second material set and at least a portion of the materials in the third material set are displayed on the first page.

14. The method of claim 1, wherein generating the first video based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set comprises: In response to an interactive operation instructing the generation of the video based on a third set of materials, a fifth page is displayed, on which the third set of materials is displayed; In response to a user's interactive operation on the first material in the third material set, the fifth material set is determined by deleting the first material from the third material set; as well as The first video is generated based on the fifth set of materials.

15. The method of claim 1, wherein generating the first video based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set comprises: The first video is generated using a multimodal model based on at least a portion of the materials in the third material set, wherein the first video includes at least one of the following generated by the multimodal model: background music, subtitles, or narration describing the at least a portion of the materials in the second material set.

16. The method of claim 1, wherein the first set of materials comprises at least one of the following: images, videos, or live photographs.

17. An apparatus for generating video, comprising: The first component display module is configured to display a first component associated with the second video, the first component displaying information about a first theme; The second component display module is configured to display the second component in response to the user's interactive operation on the first component. The second component is used to prompt the user to create a new video on the first theme. The page display module is configured to display a third page in response to user interaction with the second component, the third page displaying the first set of materials; And if no user interaction is received with the third page, the first page is displayed. The first page displays search process instructions. The search process is configured to determine a second set of materials and a third set of materials based on the search text and the first set of materials. The search text is associated with the first topic. The second set of materials is a subset of the first set of materials. The third set of materials is a subset of the second set of materials. The material collection display module is configured to display at least a portion of the materials in the second material collection and at least a portion of the materials in the third material collection; as well as The video generation module is configured to generate a first video based on at least a portion of the materials in the second material set and / or at least a portion of the materials in the third material set.

18. An electronic device comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 16.

19. A computer program product tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the method according to any one of claims 1 to 16.

Images