Method and device for acquiring video-related content, electronic equipment and medium

Abstract

The embodiment of the invention relates to a method, device and equipment for obtaining video related content and a medium. The method includes generating natural language text associated with a video based on the video. The method further includes acquiring video-related content associated with the video based on the natural language text, the video-related content including at least one of the generated video, the retrieved video and the retrieved image-text. The method further comprises the step of displaying the natural language text and the thumbnail corresponding to the video related content.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computers, and more particularly, to a method, apparatus, electronic device, and medium for acquiring video-related content. Background Art

[0002] Content acquisition refers to automatically acquiring content related to a given piece of text or image as needed. In social platforms, content generation and retrieval are widely used in the field of product recommendation, which means that when a user uploads a piece of text or image on the platform, the platform acquires content related to the uploaded text or image and recommends it to the user.

[0003] Currently, most methods for obtaining relevant content are to identify text or objects in images and then search for content related to the text or objects. The information contained in text and / or images is usually limited and cannot fully express the user's intention, resulting in a lack of accuracy in the relevant content recommended by the platform. Summary of the invention

[0004] Embodiments of the present disclosure provide a method, an apparatus, an electronic device, and a computer-readable storage medium for acquiring video-related content.

[0005] According to a first aspect of the present disclosure, a method for obtaining video-related content is provided. The method includes generating a natural language text associated with the video based on the video. The method also includes obtaining video-related content associated with the video based on the natural language text, wherein the video-related content includes at least one of the generated video, the retrieved video, and the retrieved graphic text. The method also includes displaying the natural language text and thumbnails corresponding to the video-related content.

[0006] In a second aspect of the present disclosure, a device for acquiring video-related content. The device includes a natural language text generation module configured to generate a natural language text associated with the video based on the video. The device also includes a video-related content acquisition module configured to acquire video-related content associated with the video based on the natural language text, wherein the video-related content includes at least one of the generated video, the retrieved video, and the retrieved graphic. The device also includes a video-related content display module configured to display the natural language text and thumbnails corresponding to the video-related content.

[0007] According to a third aspect of the present disclosure, an electronic device is provided, comprising a processor and a memory coupled to the processor, wherein the memory has instructions stored therein, and when the instructions are executed by the processor, the electronic device executes the method according to the first aspect.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein one or more computer instructions are stored on the computer-readable storage medium, wherein the one or more computer instructions are executed by a processor to implement the method according to the first aspect.

[0009] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram showing an example environment in which a video-related content acquisition method according to some embodiments of the present disclosure may be implemented;

[0012] Figure 2 A flowchart of a method for obtaining a video-related content model according to some embodiments of the present disclosure is shown;

[0013] Figure 3 A flowchart of a method for acquiring video-related content according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic diagram showing a process of generating multiple natural language texts according to some embodiments of the present disclosure;

[0015] Figure 5 A flowchart showing a process of generating a first expansion word according to some embodiments of the present disclosure;

[0016] Fig. 6A A schematic diagram of displaying video-related content according to some embodiments of the present disclosure is illustrated;

[0017] Figure 6B A schematic diagram of an interface for publishing a video and video-related content in combination according to certain embodiments of the present disclosure is shown;

[0018] Figure 7 A block diagram showing an apparatus for acquiring video-related content according to some embodiments of the present disclosure; and

[0019] Figure 8 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0020] Throughout the drawings, the same or similar reference numbers denote the same or similar elements. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0023] Most content acquisition methods are for text and / or images input by users. When the user inputs text and / or images, the client sends the input text and / or images to the server. The server recognizes the characters and objects in the text and / or images, then obtains the content associated with the recognized characters and objects in the database, and then sends the obtained content to the client for the user to view.

[0024] However, traditional content acquisition methods cannot identify videos. The information that images and texts can contain is limited, while videos can contain real content, specific scenes, audio and other information, and the amount of information that a video can include can be adjusted according to the length of the video. Therefore, compared with text and images, videos can more accurately reflect the user's intentions. In addition, the relevant content obtained by the platform is mostly images and text, lacking relevant content in video form, which has limitations, and users cannot find the most desired information in the relevant content obtained.

[0025] In order to solve the above problems, an embodiment of the present disclosure provides a solution for obtaining video-related content. The solution converts the content contained in the video into input text, and then uses a model (such as a pre-trained generative model) to understand the input text to generate an output text, and then obtains video-related content based on the understood output text, and the video-related content includes at least one of the generated video, the retrieved video, and the retrieved graphics and text. The solution uses a model to understand the input video text, improves the accuracy of video understanding, and thus can improve the accuracy of understanding user intent. Therefore, the video-related content obtained based on the understood output text is also more able to meet the user's expectations and give the answer the user wants. The video-related content obtained by the solution includes not only graphics and text, but also generated videos and / or retrieved videos, which can improve the comprehensiveness of the relevant content, thereby improving the user experience.

[0026] Figure 1 FIG. 1 is a schematic diagram showing an example environment 100 in which the related content acquisition method according to some embodiments of the present disclosure can be implemented. Figure 1 As shown, the example environment 100 may include a server 102, which may be a single server or a server cluster (e.g., cloud) in a centralized or distributed manner. In the embodiments of the present disclosure, video-related content may be acquired by the server. It should be understood that video-related content may also be acquired by any other device with computing capabilities. The architecture and functions of the server 102 are described here only for exemplary purposes, and do not imply any limitation on the scope of the present disclosure.

[0027] The example environment 100 may further include a client 101, which may be a user terminal, a mobile device, a computer, etc. In the embodiment of the present disclosure, after the server 102 obtains the video-related content, it transmits it to the client 101. The client 101 is a device for displaying the video-related content, which may include a display screen, and the user can view the video-related content transmitted by the server 102 through the display screen.

[0028] refer to Figure 1, the server 102 may include a video processing module 103, a pre-trained generative model 104, and a related content acquisition module 105. The video processing module 103 is used to extract information from the video input by the user, and convert the extracted information into text form to generate input text for the pre-trained generative model 104. In the embodiment of the present disclosure, the pre-trained generative model 104 may be one of the models that implement natural language processing capabilities through pre-training, for example, a large language model. The pre-trained generative model 104 is used to process the text input by the video processing module 103, and the processing method is specifically to understand and analyze the input text, and output the understood text, so as to achieve in-depth interpretation of the video information, and the output text can accurately express the user's intention. The related content acquisition module 105 is used to receive the text output by the pre-trained generative model 104, and obtain video-related content based on the output text.

[0029] In the disclosed embodiment, the acquired video-related content includes at least one of the generated video, retrieved graphics and text, and retrieved video. The diversified categories can improve the comprehensiveness of the related content, thereby increasing the probability of users acquiring target content. For example, the video-related content may include videos generated by the model based on text, videos retrieved from video websites, and graphics and text obtained from web pages and applications.

[0030] It should be understood that the architecture and functions in the example environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. Embodiments of the present disclosure may also be applied to other environments with different structures and / or functions.

[0031] The following will combine Figures 2 to 8 The process according to the embodiment of the present disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection of the present disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or the actions shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0032] Figure 2 200 according to some embodiments of the present disclosure. At block 202, a natural language text associated with the video is generated based on the video. For example, Figure 1As shown, the video processing module 103 can extract information from the video input by the user, and convert the extracted information into text form to generate a first text. In the embodiment of the present disclosure, the extracted video information may include objects included in the video, existing characters, text and / or audio information in the video, etc. The more comprehensive the extracted video information is, the more information about the video can be obtained, so that the generated first text can more accurately express the meaning included in the video. The information extracted by the video processing module 103 is mostly discrete phrases and text paragraphs, which cannot fully express the meaning that the user wants to express in the video. The pre-trained generative model 104 can be used to understand the extracted information and generate a natural language text. In some embodiments, the second text converted into a natural language form in the first text can be connected into a complete sentence by connecting the discrete phrases and text paragraphs in the first text to describe the user's intention.

[0033] For ease of understanding, an example is given here. When a user wants to know how to buy a ticket at a train station, the first text may include "train station" and "buy a ticket". After the pre-trained generative model 104 understands the first text, the output second text may be "How to use a self-service train ticket machine?" or "How to buy a train ticket online?" It should be understood that the above embodiments are exemplary and are not intended to limit the scope of protection of the present disclosure.

[0034] At block 204, video-related content associated with the video is obtained based on the natural language text, the video-related content including at least one of the generated video, the retrieved video, and the retrieved graphic text. Figure 1 After receiving the natural language text, the relevant content acquisition module 105 acquires the video-related content according to the meaning of the natural language text. In the embodiment of the present disclosure, the platform resource library can be searched to acquire the relevant content. The specific process can be to extract keywords from the natural language text and search the platform resource library according to the keywords. In addition, the relevant content can also be a video automatically generated according to the second text.

[0035] At block 206, the natural language text and the corresponding thumbnails of the video-related content are displayed. Figure 1 , both the natural language text and the video-related content can be displayed on the display screen of the client 101, and the thumbnail of the video-related content associated with the natural language text is displayed at the corresponding position of the natural language text, each thumbnail corresponds to a video-related content, and the thumbnail can be a representative image of the video-related content. The user can view the related content corresponding to the thumbnail by selecting the thumbnail.

[0036] Therefore, according to method 200 of an embodiment of the present disclosure, it is possible to obtain video-related content, and use a pre-trained generative model to achieve in-depth interpretation of video information, which can improve the understanding of user intentions. The obtained video-related content includes at least one of the generated video, retrieved graphics and text, and retrieved video. The diversified categories can improve the comprehensiveness of the related content, thereby increasing the probability of users obtaining target content.

[0037] The following will combine Figures 3 to 5 Specifically describe the process of obtaining video related content, including Figure 3 Specific steps for obtaining video-related content are shown, which are used to provide an example of processing a video and obtaining related content based on a second text; Figure 4 An example including multiple pre-trained generative models is shown for expanding the second text in multiple ways to make the acquired relevant content more comprehensive; Figure 5 The process of generating the first extended word is shown, which is used to expand the first text to make the extracted video information more comprehensive.

[0038] Figure 3 FIG. 3 is a flow chart of a method 300 for acquiring video-related content according to some embodiments of the present disclosure. At block 301, for example, Figure 1 As shown, the client 101 receives the video sent by the user, and then transmits it to the server 102. The video processing module 103 in the server 102 extracts the text in the video to generate a first text.

[0039] Boxes 302 to 310 show an example process of video processing. At box 302, the video processing module 103 may first perform frame recognition and analysis on the video, and may use different time levels for frame recognition according to actual needs. The higher the frequency of frame recognition, the more images are obtained, and the more information about the video can be obtained, which is convenient for more accurate description of the video.

[0040] At block 303, the identified video frames are deduplicated. In the disclosed embodiment, deduplication can be performed using the same image deduplication technology. For example, the similarity between the two video frames before and after the video frame portion can be compared. If the similarity is higher than a first threshold, the latter video frame is deleted from the video frame portion. Deduplication of the video and deletion of redundant video frames can reduce the amount of data processing in the video content recognition step, avoid repeated content recognition of the same video frame, and thus improve processing efficiency.

[0041] At box 304, after deduplication, the content contained in the obtained multiple deduplication frames can be identified. In the embodiment of the present disclosure, the identified video content can include image text in the video, and the image text can be a title, subtitle, etc. in the deduplication frame. The method of identifying text can adopt an image optical character recognition (OCR) service to identify text information in the image, including titles, subtitles, descriptions of key points, etc. At box 305, the identified video content can also include objects in the video, and the objects in the video can be objects such as people, houses, cars, etc. in the deduplication frame image, as well as various actions, scenes, etc.

[0042] At frame 306, after identifying the object and the image text in the video, the identified object is converted into the object text, and keywords are extracted from the object text and the image text to obtain the keyword group. In the process of generating the keyword group, the object text can be generated by a common text conversion method, and the keyword group can also be generated by a common keyword extraction method.

[0043] At box 307, in some embodiments, the method for generating the first text also includes extracting audio from the video and separating the audio portion from the video. In the disclosed embodiment, the audio can be segmented for speech with a longer duration in the video. In some embodiments, the segmentation is performed using a time-sharing method. The time-sharing method is simply to segment the audio in the video in a predetermined period. For example, the audio is segmented every 30 seconds, that is, the audio is segmented into 30-second segments. The advantage of the time-sharing method is that the operation is simple and fast.

[0044] In some embodiments, the spectrum method can be used for segmentation. The spectrum method refers to a method of determining the signal performance by observing the change of the amplitude of the signal with the frequency characteristic (amplitude-frequency characteristic). In some cases, it also includes the change of the phase of the signal with the frequency characteristic (phase-frequency characteristic). Therefore, in some embodiments, the spectrum method is used to segment by observing the characteristics of the audio signal, which can ensure that the segmented audio is a complete sentence.

[0045] At box 308, speech recognition is performed on the extracted audio. In the embodiment of the present disclosure, automatic speech recognition (Automatic Speech Recognition, ASR) technology can be used for speech recognition, or other general speech recognition technologies can be used. The specific selection can be based on actual needs, specifically to meet the purpose of recognizing video speech. In addition, speech recognition can also be performed by means of a machine learning model. At box 309, the recognized speech is converted into text to generate a text paragraph 310. In the embodiment of the present disclosure, speech-to-text conversion can be implemented based on a speech-to-text conversion algorithm or technology known in the art or developed in the future. The specific selection can be based on actual needs to meet the purpose of speech-to-text conversion.

[0046] At box 311, the keyword phrase and the text paragraph are input as the first text into the pre-trained generative model, and the pre-trained generative model understands the first text (i.e., text input) and converts the first text into a second text (i.e., text output) in natural language form, that is, the discrete phrases and text paragraphs in the first text are connected into a complete sentence to describe the user's intention. By using the pre-trained generative model to understand the text, a natural language text with richer content and more accurate semantics can be generated for subsequent video generation or retrieval.

[0047] At box 312, keywords in the second text are extracted to generate search terms. For example, the second text may be "How to use a self-service ticket purchasing machine for train tickets?", and the extracted keywords may be "train tickets" and "self-service ticket purchasing machine". At box 313, a video library is searched. The video library is searched according to the search terms extracted from the second text. The video library is searched according to the search terms, and the searched video 314 can be obtained.

[0048] Alternatively or additionally, at block 315, the picture and text library is searched. The picture and text library is searched according to the search term extracted from the second text. The videos in the picture and text library may be historical pictures and texts stored in the platform or pictures and texts stored in an external resource library connected to the platform. The picture and text library is searched according to the search term, and the searched pictures and texts 316 may be obtained.

[0049] At box 317, the video-related content obtained by the present disclosure also includes a generated video 318. The generated content model can be used to automatically generate a related video based on the input second text. The generated content model can be other models or algorithms that generate related content based on the input conditions or guidance, and the specific selection can be based on actual needs.

[0050] At box 319, the recommendation list is calculated and the relevant content is returned. When the acquired video-related content includes multiple contents, the relevance of each content in the video-related content to the video is determined, and the ranking of each content in the recommendation list is determined according to the relevance. For example, the video-related content includes two retrieved videos, and the relevance of the two retrieved videos to the video input by the user is calculated respectively, and the ranking of each content in the recommendation list is determined according to the relevance. The content with high relevance can be ranked at the top of the recommendation list, and the content with low relevance can be ranked at the back of the recommendation list.

[0051] Figure 4 A schematic diagram of a process 400 for generating multiple natural language texts according to some embodiments of the present disclosure is shown. In the embodiments of the present disclosure, multiple pre-trained generative models may be included, specifically a first pre-trained generative model 401, a second pre-trained generative model 402, and a third pre-trained generative model 403. The number of pre-trained generative models may be selected according to actual needs during implementation. The inputs of the three pre-trained generative models are all the first text, and different models may have different understandings of the first text. Therefore, the text output by the first pre-trained generative model 401 after understanding the first text is the first natural language text 404, the text output by the second pre-trained generative model 402 after understanding the first text is the second natural language text 405, and the text output by the third pre-trained generative model 403 after understanding the first text is the third natural language text 406. For ease of understanding, an example will be given below. For example, the first text includes "train station" and "ticket purchase", then the first natural language text 404 can be "How to purchase tickets at the train station?", the second natural language text 405 can be "How to use the train self-service ticket machine?", and the third natural language text 406 can be "How to purchase train tickets online?".

[0052] Setting up multiple pre-trained generative models can produce different understandings of the first text, thereby generating multiple versions of the second text including multiple different sentences, so that the acquired video-related content is also more extensive. Figure 4 In the example, the first video-related content 407 is obtained according to the first natural language text 404, the second video-related content 408 is obtained according to the second natural language text 405, and the third video-related content 409 is obtained according to the third natural language text 406. The expansion of the range of video-related content can provide users with a wider range of choices and increase the probability of obtaining the information that users want to find.

[0053] Figure 5Flowchart of process 500 for generating a first extended word according to certain embodiments of the present disclosure. At box 501, based on a predetermined template, a first extended word semantically associated with an object text and / or an image text is determined. In an embodiment of the present disclosure, an extended word for a number may include one or more numbers other than the number, and an extended word for a letter may include one or more letters other than the letter. According to an embodiment of the present disclosure, an extended word for a character may include one or more characters other than the character, and the number or type of extended words may be determined based on experience or need. The extended word may also include at least one of the following: antonyms, synonyms, hypernyms, and hyponyms.

[0054] At box 502, a phrase is generated based on the object text, the image text and the first extended word. After the first extended word is generated, keywords in the object text, the image text and the first extended word can be extracted to generate a keyword phrase. According to an embodiment of the present disclosure, the extraction of keywords can be implemented based on any suitable word segmentation algorithm known in the art. The first extended word can expand the amount of information of the first text, and the first text is the input data of the pre-trained generative model. The increase of the input data of the pre-trained generative model can improve the accuracy of the output text of the pre-trained generative model, thereby improving the accuracy of understanding the user's intention.

[0055] In some embodiments of the present disclosure, the speech-converted text paragraph can also determine a second extended word that is semantically associated with the text paragraph based on a preset semantic library, thereby expanding the amount of information in the speech text and improving the accuracy of the text output by the pre-trained generative model, thereby improving the accuracy of understanding the user's intent.

[0056] Fig. 6A FIG. 2 is a schematic diagram showing video-related content according to some embodiments of the present disclosure. Fig. 6AAs shown, the client device can display a user interface 601, and a shooting control 602 is displayed on the user interface 601. In response to the user's touch operation on the shooting control 602, the client starts shooting. After the shooting is completed, the client uploads the shot video to the server, and the server obtains relevant content based on the video and returns it to the client. The video-related content and the video playback control 613 are displayed on the user interface 601. In the embodiment of the present disclosure, the video-related content includes a first output text (such as a first question) and a second output text (such as a second question). During the specific implementation, the number of output texts can be selected according to actual needs. The first output text and the second output text represent different meanings. The first output text corresponds to a first group of thumbnails associated with the first output text, and the second output text corresponds to a second group of thumbnails associated with the second output text. Each thumbnail corresponds to a video-related content, and the thumbnail specifically refers to a representative image of the video-related content. The user can determine which group of thumbnails represents the video-related content that is closest to his or her intention based on the meaning of the first output text and the second output text, and select the closest thumbnail to view the relevant content corresponding to the thumbnail.

[0057] In the disclosed embodiment, when the user inputs a video with the intention of asking a question, the first output text may be the first question "How to buy a ticket at a train station", and the second output text may be the second question "How to buy a train ticket online". The two texts represent two questions with different meanings, and the related content represented by the first group of thumbnails is the answer to the first question, and the related content represented by the second group of thumbnails is the answer to the second question. By distinguishing the obtained video related content by different semantic expressions, the user can intuitively view the information included in the related content, improve the efficiency of user viewing and retrieval, and improve the user experience.

[0058] In some embodiments of the disclosed embodiments, at least one of a first classification control 604, a second classification control 605, and a third classification control 606 is displayed on the user interface 601, and the thumbnail displayed corresponding to the first classification control 604 represents a generated video in the related content, the thumbnail displayed corresponding to the second video classification control 605 represents a retrieved video in the related content, and the thumbnail displayed corresponding to the third classification control 606 represents a retrieved image and text in the related content. For example, in response to a user's touch operation on the first classification control 604, the user interface 601 displays a thumbnail of the generated video. The setting of the classification control is to distinguish the video-related content, which can further improve the efficiency of the user in finding the target content.

[0059] In some embodiments of the present disclosure, thumbnails associated with video-related content are arranged according to the relevance of the corresponding related content to the video. For example, the relevance of the related content corresponding to the first thumbnail 607 to the video is greater than the relevance of the related content corresponding to the second thumbnail 608 to the video.

[0060] In some embodiments of the present disclosure, the user interface 601 further displays a sixth control 603, a seventh control 609, an eighth control 610, a first control 611, and a control 612 for reshooting. In response to the user's touch operation on the sixth control 603, the video input by the user is released. In response to the user's touch operation on the seventh control 609, more video-related content is displayed. In response to the user's touch operation on the eighth control 610, the interface of manual service is displayed. In response to the user's touch operation on the first control 611, the second control, the third control, and / or the fourth control are displayed, the second control is configured to trigger the interface for displaying text questions when touched, the third control is configured to trigger the interface for displaying graphic questions when touched, and the fourth control is configured to trigger the interface for displaying video questions when touched, so that the user can ask questions through text or graphic text, or ask questions again using video. In response to the user's touch operation on the control 612, the video shooting page is returned to display, and the user can reshoot the video.

[0061] Figure 6B A schematic diagram of an interface for publishing a video and video-related content in combination according to some embodiments of the present disclosure is shown. When the user selects the first thumbnail, the user interface 601 displays Figure 6B The interface presented has a video thumbnail 614 and another thumbnail 615 displayed thereon, and in response to the user's touch operation on the fifth control 616, the video and the related content corresponding to the other thumbnail can form a content and be combined and published. In this way, the question video and the related answer video can be retrieved, and a new complete video can be quickly generated and published.

[0062] Figure 7 FIG. 7 is a block diagram of an apparatus 700 for acquiring video-related content according to some embodiments of the present disclosure. Figure 7 As shown, the device 700 includes a natural language generation module 702, a video-related content acquisition module 704, and a video-related content display module 706. The natural language text generation module 702 is configured to generate a natural language text associated with the video based on the video. The video-related content acquisition module 704 is configured to acquire video-related content associated with the video based on the natural language text, and the video-related content includes at least one of the generated video, the retrieved video, and the retrieved graphic text. The video-related content display module 706 is configured to display the natural language text and the thumbnails corresponding to the video-related content.

[0063] Figure 8 8 is a block diagram of an electronic device 800 according to some embodiments of the present disclosure. The device 800 may be a device or apparatus described in an embodiment of the present disclosure. Figure 8 As shown, the device 800 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The CPU / GPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804. Although not shown in FIG. Figure 8 As shown in FIG. 8 , device 800 may also include a co-processor.

[0064] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0065] The various methods or processes described above may be performed by the CPU / GPU 801. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU / GPU 801, one or more steps or actions in the methods or processes described above may be performed.

[0066] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0067] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0068] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0069] The computer program instructions for performing the disclosed operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, programming languages ​​including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In certain embodiments, by utilizing the state information of a computer-readable program instruction to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0070] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0071] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0072] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

[0074] Some example implementations of the present disclosure are listed below.

[0075] Example 1. A method for obtaining video-related content, comprising:

[0076] Based on the video, generating natural language text associated with the video;

[0077] Based on the natural language text, obtaining video-related content associated with the video, the video-related content including at least one of a generated video, a retrieved video, and a retrieved graphic and text; and

[0078] The natural language text and thumbnails corresponding to the video-related content are displayed.

[0079] Example 2. The method of Example 1, wherein generating natural language text associated with the video comprises:

[0080] Based on the video, generating a first text associated with the video, the first text comprising a phrase; and

[0081] Based on the first text, a second text as a natural language text is generated.

[0082] Example 3. The method of any of Examples 1-2, wherein generating a first text associated with the video comprises:

[0083] Extracting multiple video frames from the video;

[0084] Generate a plurality of deduplicated frames by deduplicating the plurality of video frames; and

[0085] A first text associated with the video is generated by identifying at least one deduplicated frame from among the plurality of deduplicated frames.

[0086] Example 4. The method of any of Examples 1-3, wherein generating a first text associated with the video comprises:

[0087] Identifying objects and image text included in the deduplication frame;

[0088] determining an object context associated with the object; and

[0089] Based on the object text and the image text, a phrase associated with the video is generated.

[0090] Example 5. The method of any one of Examples 1-4, wherein generating a first text associated with the video further comprises:

[0091] extracting audio from the video;

[0092] Performing speech recognition on the audio to generate speech text; and

[0093] Based on the keyword group and the voice text, a first text associated with the video is generated.

[0094] Example 6. The method of any of Examples 1-5, wherein generating a phrase associated with the video comprises:

[0095] Determining a first extended word semantically associated with the object text and / or the image text based on a predetermined template; and

[0096] Generate the phrase based on the object text, the image text and the first extended word

[0097] Example 7. The method of any one of Examples 1-6, wherein generating the speech text comprises:

[0098] Convert the recognized speech into text paragraphs;

[0099] Determining, based on a preset semantic library, a second extended word semantically associated with the text paragraph; and

[0100] The voice text is generated based on the text paragraph and the second expanded word.

[0101] Example 8. The method of any one of Examples 1-7, wherein generating natural language text comprises:

[0102] Generate a first output text based on the understanding of the first text by the first pre-trained generative model;

[0103] Generate a second output text based on the understanding of the first text by the second pre-trained generative model; and

[0104] The second text is generated based on the first output text and the second output text.

[0105] Example 9. The method of any one of Examples 1-8, wherein obtaining video-related content associated with the video comprises:

[0106] Based on the natural language text, the generated video is obtained by generating a content model.

[0107] Example 10. The method of any one of Examples 1-9, wherein obtaining video-related content associated with the video further comprises:

[0108] Determining search terms associated with the natural language text based on the semantics of the natural language text; and

[0109] Based on the search term, the searched video is obtained from a video library.

[0110] Example 11. The method of any one of Examples 1-10, wherein obtaining video-related content associated with the video further comprises:

[0111] Based on the search term, the searched image and text are obtained from the image and text library.

[0112] Example 12. The method according to any one of Examples 1-11, further comprising:

[0113] Determining the relevance of each of the video-related contents to the video; and

[0114] A ranking of each of the video-related contents is determined based on the relevance.

[0115] Example 13. The method of any one of Examples 1-12, wherein the natural language text includes at least a first question, the method further comprising:

[0116] displaying a first set of thumbnails of video-related content associated with the first question; and

[0117] In response to a user selecting a thumbnail in the first set of thumbnails, video-related content corresponding to the selected thumbnail is displayed.

[0118] Example 14. The method of any of Examples 1-13, wherein displaying a first set of thumbnails of video-related content associated with the first question comprises:

[0119] displaying a plurality of categories associated with video-related content; and

[0120] In response to a user selecting a first category of the plurality of categories, thumbnails of video-related content associated with the first category are displayed.

[0121] Example 15. The method of any one of Examples 1-14, further comprising:

[0122] In response to a user's touch operation on a control for reshooting, a video shooting page is displayed.

[0123] Example 16. The method of any one of Examples 1-15, further comprising:

[0124] In response to the user's touch operation on the first control, the second control, the third control and / or the fourth control are displayed; wherein the second control is configured to trigger the display of an interface for text questions when touched, the third control is configured to trigger the display of an interface for graphic questions when touched, and the fourth control is configured to trigger the display of an interface for video questions when touched.

[0125] Example 17. The method of any of Examples 1-16, further comprising:

[0126] In response to the user's touch operation on the fifth control, a combined video including the video and content related to the video is released.

[0127] Example 18. An apparatus for obtaining video-related content, comprising:

[0128] A natural language text generation module is configured to generate a natural language text associated with the video based on the video;

[0129] a video-related content acquisition module, configured to acquire video-related content associated with the video based on the natural language text, wherein the video-related content includes at least one of a generated video, a retrieved video, and a retrieved graphic; and

[0130] The video-related content display module is configured to display the natural language text and thumbnails corresponding to the video-related content.

[0131] Example 19. The apparatus of Example 18, wherein the natural language text generation module comprises:

[0132] A first text generation module is configured to generate a first text associated with the video based on the video, wherein the first text includes a phrase; and

[0133] The second text generating module is configured to generate a second text as a natural language text based on the first text.

[0134] Example 20. The apparatus of any one of Examples 18-19, wherein the natural language text generation module comprises:

[0135] A video frame extraction module, configured to extract a plurality of video frames from the video;

[0136] a video frame deduplication module, configured to generate a plurality of deduplication frames by deduplicating the plurality of video frames; and

[0137] The deduplication frame identification module is configured to generate a first text associated with the video by identifying at least one deduplication frame among the plurality of deduplication frames.

[0138] Example 21. The apparatus of any one of Examples 18-20, wherein the first text generation module further comprises:

[0139] An object and text recognition module, configured to recognize objects and image texts included in the deduplicated frame;

[0140] an object text conversion module configured to determine an object text associated with the object; and

[0141] The phrase generation module is configured to generate a phrase associated with the video based on the object text and the image text.

[0142] Example 22. According to any one of Examples 18-21, the first text generation module further includes:

[0143] An audio extraction module, configured to extract audio from the video;

[0144] A speech text generation module is configured to perform speech recognition on the audio to generate speech text; and

[0145] The associated text generation module is configured to generate a first text associated with the video based on the keyword group and the voice text.

[0146] Example 23. The apparatus of any one of Examples 18-22, wherein the phrase generation module comprises:

[0147] a first expansion word determination module configured to determine a first expansion word semantically associated with the object text and / or the image text based on a predetermined template; and

[0148] The first phrase generating module is configured to generate the phrase based on the object text, the image text and the first extended word.

[0149] Example 24. The apparatus of any one of Examples 18-23, wherein the speech-to-text generation module comprises:

[0150] A text paragraph conversion module, configured to convert the recognized speech into a text paragraph;

[0151] a second expansion word generating module configured to determine a second expansion word semantically associated with the text paragraph based on a preset semantic library; and

[0152] The first voice text generation module is configured to generate the voice text based on the text paragraph and the second expanded word.

[0153] Example 25. The apparatus of any one of Examples 18-24, wherein the natural language text generation module comprises:

[0154] A first output text generation module, configured to generate a first output text based on the understanding of the first text by the first pre-trained generative model;

[0155] A second output text generation module is configured to generate a second output text based on the understanding of the first text by a second pre-trained generative model; and

[0156] The natural language text determination module is configured to generate the second text based on the first output text and the second output text.

[0157] Example 26. The apparatus of any one of Examples 18-25, wherein the video-related content acquisition module comprises:

[0158] The generated video acquisition module is configured to obtain the generated video by generating a content model based on the natural language text.

[0159] Example 27. According to any one of Examples 18-26, the video-related content acquisition module further includes:

[0160] A search term generation module configured to determine a search term associated with the natural language text based on the semantics of the natural language text; and

[0161] The retrieved video acquisition module is configured to obtain the retrieved video from a video library based on the search term.

[0162] Example 28. The apparatus according to any one of Examples 18-27, wherein the video-related content acquisition module further comprises:

[0163] The retrieved image and text acquisition module is configured to obtain the retrieved image and text from the image and text library based on the search term.

[0164] Example 29. The apparatus of any of Examples 18-28, further comprising:

[0165] a relevance determination module, configured to determine the relevance of each of the video-related contents to the video; and

[0166] The ranking determination module is configured to determine the ranking of each content in the video-related content based on the relevance.

[0167] Example 30. The apparatus of any of Examples 18-29, wherein the natural language text includes at least a first question, the apparatus further comprising:

[0168] A first group of thumbnail display modules configured to display a first group of thumbnails of video-related content associated with the first question; and

[0169] The video-related content display module is configured to display the video-related content corresponding to the selected thumbnail in response to the user selecting a thumbnail in the first group of thumbnails.

[0170] Example 31. The apparatus of any one of Examples 18-30, wherein the first set of thumbnail display modules comprises:

[0171] a category display module configured to display a plurality of categories associated with the video-related content; and

[0172] The thumbnail display module is configured to display thumbnails of video-related content associated with a first category in response to a user selecting the first category among the multiple categories.

[0173] Example 32. The apparatus of any of Examples 18-31, further comprising:

[0174] The video shooting page display module is configured to display the video shooting page in response to a user's touch operation on a control for reshooting.

[0175] Example 33. The apparatus of any of Examples 18-32, further comprising:

[0176] The control display module is configured to display a second control, a third control and / or a fourth control in response to a user's touch operation on a first control; wherein the second control is configured to trigger a text question interface when touched, the third control is configured to trigger a graphic question interface when touched, and the fourth control is configured to trigger a video question interface when touched.

[0177] Example 34. The apparatus of any of Examples 18-33, further comprising:

[0178] The combined video publishing module is configured to publish a combined video including the video and content related to the video in response to a user's touch operation on the fifth control.

[0179] Example 35. An electronic device comprising:

[0180] Processor; and

[0181] A memory coupled to the processor, the memory having instructions stored therein, wherein the instructions, when executed by the processor, cause the electronic device to perform an action, the action being used to obtain video-related content, the action comprising:

[0182] Based on the video, generating natural language text associated with the video;

[0183] Based on the natural language text, obtaining video-related content associated with the video, the video-related content including at least one of a generated video, a retrieved video, and a retrieved graphic and text; and

[0184] The natural language text and thumbnails corresponding to the video-related content are displayed.

[0185] Example 36. The electronic device of Example 35, wherein generating natural language text associated with the video comprises:

[0186] Based on the video, generating a first text associated with the video, the first text comprising a phrase; and

[0187] Based on the first text, a second text as a natural language text is generated.

[0188] Example 37. An electronic device according to any of Examples 35-36, wherein generating a first text associated with the video comprises:

[0189] Extracting multiple video frames from the video;

[0190] Generate a plurality of deduplicated frames by deduplicating the plurality of video frames; and

[0191] A first text associated with the video is generated by identifying at least one deduplicated frame from among the plurality of deduplicated frames.

[0192] Example 38. An electronic device according to any of Examples 35-37, wherein generating a first text associated with the video comprises:

[0193] Identifying objects and image text included in the deduplication frame;

[0194] determining an object context associated with the object; and

[0195] Based on the object text and the image text, a phrase associated with the video is generated.

[0196] Example 39. An electronic device according to any of Examples 35-38, wherein generating a first text associated with the video further comprises:

[0197] extracting audio from the video;

[0198] Performing speech recognition on the audio to generate speech text; and

[0199] Based on the keyword group and the voice text, a first text associated with the video is generated.

[0200] Example 40. The electronic device of any of Examples 35-39, wherein generating a phrase associated with the video comprises:

[0201] Determining a first extended word semantically associated with the object text and / or the image text based on a predetermined template; and

[0202] The phrase is generated based on the object text, the image text, and the first extended word.

[0203] Example 41. An electronic device according to any of Examples 35-40, wherein generating the voice text comprises:

[0204] Convert the recognized speech into text paragraphs;

[0205] Determining, based on a preset semantic library, a second extended word semantically associated with the text paragraph; and

[0206] The voice text is generated based on the text paragraph and the second expanded word.

[0207] Example 42. An electronic device according to any of Examples 35-41, wherein generating natural language text comprises:

[0208] Generate a first output text based on the understanding of the first text by the first pre-trained generative model;

[0209] Generate a second output text based on the understanding of the first text by the second pre-trained generative model; and

[0210] The second text is generated based on the first output text and the second output text.

[0211] Example 43. An electronic device according to any of Examples 35-42, wherein obtaining video-related content associated with the video comprises:

[0212] Based on the natural language text, the generated video is obtained by generating a content model.

[0213] Example 44. An electronic device according to any of Examples 35-43, wherein obtaining video-related content associated with the video further comprises:

[0214] Determining search terms associated with the natural language text based on the semantics of the natural language text; and

[0215] Based on the search term, the searched video is obtained from a video library.

[0216] Example 45. The electronic device of any one of Examples 35-44, wherein obtaining video-related content associated with the video further comprises:

[0217] Based on the search term, the searched image and text are obtained from the image and text library.

[0218] Example 46. The electronic device of any of Examples 35-45, the actions further comprising:

[0219] Determining the relevance of each of the video-related contents to the video; and

[0220] A ranking of each of the video-related contents is determined based on the relevance.

[0221] Example 47. An electronic device according to any of Examples 35-46, wherein the natural language text includes at least a first question, and the method further includes:

[0222] displaying a first set of thumbnails of video-related content associated with the first question; and

[0223] In response to a user selecting a thumbnail in the first set of thumbnails, video-related content corresponding to the selected thumbnail is displayed.

[0224] Example 48. The electronic device of any of Examples 35-47, wherein displaying a first set of thumbnails of video-related content associated with the first question comprises:

[0225] displaying a plurality of categories associated with video-related content; and

[0226] In response to a user selecting a first category of the plurality of categories, thumbnails of video-related content associated with the first category are displayed.

[0227] Example 49. The electronic device of any of Examples 35-48, the actions further comprising:

[0228] In response to a user's touch operation on a control for reshooting, a video shooting page is displayed.

[0229] Example 50. The electronic device of any one of Examples 35-49, the actions further comprising:

[0230] In response to the user's touch operation on the first control, the second control, the third control and / or the fourth control are displayed; wherein the second control is configured to trigger the display of an interface for text questions when touched, the third control is configured to trigger the display of an interface for graphic questions when touched, and the fourth control is configured to trigger the display of an interface for video questions when touched.

[0231] Example 51. The electronic device of any of Examples 35-50, wherein the actions further include:

[0232] In response to the user's touch operation on the fifth control, a combined video including the video and content related to the video is released.

[0233] Example 52. A computer-readable storage medium having one or more computer instructions stored thereon, wherein the one or more computer instructions are executed by a processor to implement a method according to any one of Examples 1 to 17.

[0234] Although the disclosure has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Instead, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for obtaining video-related content, comprising: Based on the video, generating natural language text associated with the video; Based on the natural language text, obtaining video-related content associated with the video, the video-related content including at least one of a generated video, a retrieved video, and a retrieved graphic and text; and The natural language text and thumbnails corresponding to the video-related content are displayed.

2. The method of claim 1, wherein generating natural language text associated with the video comprises: Based on the video, generating a first text associated with the video, the first text comprising a phrase; and Based on the first text, a second text is generated as the natural language text.

3. The method of claim 2, wherein generating a first text associated with the video comprises: Extracting multiple video frames from the video; Deduplication is performed on the plurality of video frames to generate a plurality of deduplication frames; as well as A first text associated with the video is generated by identifying at least one deduplicated frame from among the plurality of deduplicated frames.

4. The method of claim 3, wherein generating a first text associated with the video comprises: Identifying objects and image text included in the deduplication frame; determining an object text associated with the object; as well as Based on the object text and the image text, a phrase associated with the video is generated.

5. The method of claim 4, wherein generating a first text associated with the video further comprises: extracting audio from the video; Performing speech recognition on the audio to generate speech text; as well as Based on the phrase and the voice text, a first text associated with the video is generated.

6. The method of claim 4, wherein generating a phrase associated with the video comprises: Based on a predetermined template, determining a first extended word semantically associated with the object text and / or the image text; as well as The phrase is generated based on the object text, the image text, and the first extended word.

7. The method according to claim 5, wherein generating the speech text comprises: Convert the recognized speech into text paragraphs; Based on a preset semantic library, determining a second extended word semantically associated with the text paragraph; as well as The voice text is generated based on the text paragraph and the second expanded word.

8. The method of claim 2, wherein generating natural language text associated with the video comprises: Generate a first output text based on the understanding of the first text by the first pre-trained generative model; Generate a second output text based on the understanding of the first text by the second pre-trained generative model; as well as The second text is generated based on the first output text and the second output text.

9. The method according to claim 1, wherein obtaining video-related content associated with the video comprises: Based on the natural language text, the generated video is obtained by generating a content model.

10. The method according to claim 9, wherein obtaining video-related content associated with the video further comprises: Determining search terms associated with the natural language text based on the semantics of the natural language text; as well as Based on the search term, the searched video is obtained from a video library.

11. The method according to claim 10, wherein obtaining video-related content associated with the video further comprises: Based on the search term, the searched image and text are obtained from the image and text library.

12. The method according to claim 1, further comprising: Determine the relevance of each content in the video-related content to the video; as well as A ranking of each of the video-related contents is determined based on the relevance.

13. The method of claim 1, wherein the natural language text comprises at least a first question, the method further comprising: displaying a first set of thumbnails of video-related content associated with the first question; as well as In response to a user selecting a thumbnail in the first set of thumbnails, video-related content corresponding to the selected thumbnail is displayed.

14. The method of claim 13, wherein displaying a first set of thumbnails of video-related content associated with the first question comprises: Displaying multiple categories associated with video-related content; as well as In response to a user selecting a first category of the plurality of categories, thumbnails of video-related content associated with the first category are displayed.

15. The method according to claim 13, further comprising: In response to a user's touch operation on a control for reshooting, a video shooting page is displayed.

16. The method according to claim 13, further comprising: In response to the user's touch operation on the first control, the second control, the third control and / or the fourth control are displayed, wherein the second control is configured to trigger the display of an interface for text questions when touched, the third control is configured to trigger the display of an interface for graphic questions when touched, and the fourth control is configured to trigger the display of an interface for video questions when touched.

17. The method according to claim 13, further comprising: In response to the user's touch operation on the fifth control, a combined video including the video and content related to the video is released.

18. A device for obtaining video-related content, comprising: A natural language text generation module is configured to generate a natural language text associated with the video based on the video; a video-related content acquisition module, configured to acquire video-related content associated with the video based on the natural language text, wherein the video-related content includes at least one of a generated video, a retrieved video, and a retrieved graphic; and The video-related content display module is configured to display the natural language text and thumbnails corresponding to the video-related content.

19. An electronic device comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, wherein when the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-17.

20. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1-17.

Images