Video generation method, apparatus, device, and storage medium

Abstract

Embodiments of this disclosure provide a video generation method and apparatus, device and storage medium, the method comprising: obtaining a first video and video description information for the first video; obtaining introductory text information corresponding to the first video and time node information for at least one video segment in the first video based on the video description information; dividing the first video based on the time node information for at least one video segment to obtain a video frame sequence corresponding to at least one video segment; obtaining the playback time length of the video frame sequence; converting the introductory text information into audio data of the playback time length; and generating a second video based on the video frame sequence and audio data. Since a video digest of the first video can be automatically generated using the video description information, the efficiency of generating a video digest is higher compared to manually generating a video digest of the original video.

Description

Technical Field

[0001] Cross - reference to related applications This application claims the priority of Chinese Patent Application No. 202311118817.4 filed on August 31, 2023, and hereby incorporates by reference all the content disclosed in the above - mentioned Chinese patent application as part of this application.

[0002] Embodiments of the present disclosure relate to a method and apparatus for generating videos, devices, and storage media.

Background Art

[0003] Currently, more and more users prefer to grasp the core plot of a movie video by watching its video digest. Here, the video digest is a short video obtained by extracting the core plot segments in the movie video and adding corresponding explanatory texts.

[0004] In some cases, technicians first watch and understand the movie video content, divide the movie video based on their personal understanding, and extract the core plot segments from it. Then, based on the understanding of the movie video content, they write explanatory texts related to the movie video and add the explanatory texts to the core plot segments to obtain the video digest of the movie video.

[0005] However, the inventor has found that at least the following technical problems exist in the prior art. That is, technicians need to first watch and understand the movie video content and then extract the core plot segments, which increases the time required to generate the video digest of the movie video, so the efficiency of the video generation method is low.

Summary of the Invention

Problems to be Solved by the Invention

[0006] The embodiments of this disclosure provide a video generation method, apparatus, device, and storage medium that can improve video generation efficiency. [Means for solving the problem]

[0007] Embodiments of this disclosure provide a method for generating video, the method is Obtaining the first video and the video description information of the first video, Based on the video description information, the following is obtained: introductory text information corresponding to the first video and time node information for at least one video segment in the first video; Based on the time node information of the at least one video segment, the first video is divided to obtain a video frame sequence corresponding to the at least one video segment. This includes obtaining the playback time of the video frame sequence, converting the introductory text information into audio data of the playback time, and generating a second video based on the video frame sequence and the audio data.

[0008] Embodiments of this disclosure provide a video generation apparatus, which, A first acquisition module for acquiring a first video and video description information of the first video, A second acquisition module for obtaining introductory text information corresponding to the first video and time node information for at least one video segment in the first video, based on the video description information, A splitting module for splitting the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment, The system includes a generation module for obtaining the playback time of the video frame sequence, converting the introductory text information into audio data of the playback time, and generating a second video based on the video frame sequence and the audio data.

[0009] The embodiments of this disclosure are The present invention provides an electronic device including a processor and a memory connected to the processor, wherein the memory stores computer execution instructions, and the processor executes the computer execution instructions stored in the memory to realize a video generation method according to any one embodiment of the present disclosure.

[0010] Embodiments of the present disclosure provide a computer-readable storage medium in which computer execution instructions are stored, and when a processor executes the computer execution instructions, a video generation method according to any one embodiment of the present disclosure is realized.

[0011] The embodiments of this disclosure provide a computer program which, when executed by a processor, realizes a method for generating a video as described in any one embodiment of this disclosure. [Brief explanation of the drawing]

[0012] To more clearly illustrate the embodiments of this disclosure or the technical concepts in the prior art, the following briefly introduces the drawings necessary for describing the embodiments or the prior art. Clearly, the drawings in the following description are some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these without expending any creative effort. [Figure 1] This is a schematic diagram of an application scene of the video generation method according to an embodiment of the present disclosure. [Figure 2] This is a flowchart of the video generation method according to the embodiments of this disclosure. [Figure 3] This is a schematic diagram of a video generation method according to an embodiment of the present disclosure. [Figure 4] This is a flowchart of another video generation method according to an embodiment of the present disclosure. [Figure 5] This is a block diagram of the configuration of a video generation device according to an embodiment of the present disclosure. [Figure 6]This is a schematic diagram of the hardware configuration of an electronic device according to an embodiment of the present disclosure. [Modes for carrying out the invention]

[0013] To further clarify the purpose, technical proposals, and advantages of the embodiments of this disclosure, the technical proposals of the embodiments of this disclosure are described below clearly and completely, with reference to the drawings of the embodiments. Of course, the embodiments described are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by a person skilled in the art without any creative effort based on the embodiments of this disclosure are all within the scope of protection of this disclosure.

[0014] Furthermore, the user information (including, but not limited to, user device information and user personal information) and data (including, but not limited to, data for analysis, stored data, and displayed data) relating to this application are all information and data authorized by the user or fully permitted by the parties, and the collection, use, and processing of the relevant data must comply with applicable laws, regulations, and standards, and corresponding operational entries are provided for the user to choose to permit or deny.

[0015] Currently, an increasing number of users prefer to watch video digests of movies to grasp the core plot of those movies. Here, a video digest is a short video obtained by extracting the core plot segments from a movie and adding corresponding explanatory text.

[0016] In some cases, technicians first watch and understand movie video content, divide the movie video based on their personal understanding, and extract the core plot segments from it. Then, based on the understanding of the movie video content, they write explanatory text related to the movie video and add the explanatory text to the core plot segments to obtain a video digest of the movie video. However, since technicians need to first watch and understand the movie video content and then extract the core plot segments, the time required to generate the video digest of the movie video becomes long, so the efficiency of the method for generating the video digest is low.

[0017] Therefore, how to improve the generation efficiency of the video digest of movie videos is one technical problem that urgently needs to be solved currently.

[0018] To solve the above problems, embodiments of the present disclosure provide the following technical ideas. First, video identifier of the video file can be used to obtain introduction text information of the video digest and time node information to important video segments, and based on the time node information of the important video segments, extract important video segments from the original video file. Finally, convert the introduction text information into audio data and add it to the audio track corresponding to the video frame sequence to obtain a video digest. Since the video digest corresponding to the video identifier can be automatically generated by the video identifier, the efficiency of generating the video digest is higher compared with manually generating the video digest of the original video.

[0019] Hereinafter, the application scenarios of embodiments of the present disclosure will be described.

[0020] The video generation method according to an embodiment of the present disclosure may be used in various video generation scenarios. For example, a video digest of a movie video is generated, and for another example, a video digest of a game video is generated. FIG. 1 is a schematic diagram of an application scenario of the video generation method according to an embodiment of the present disclosure. As shown in FIG. 1, data transmission is performed between the terminal 101 and the service side 102 in a wired or wireless connection manner. The terminal 101 can send a request to the service side 102 to generate a video digest of a movie video. When the service side 102 receives the request, based on the video identifier carried in the request, it generates a video digest of the movie video and returns the video digest of the movie video to the terminal 101. The terminal 101 can play the video digest of the movie video. Hereinafter, the video generation method according to an embodiment of the present disclosure will be described in detail using specific embodiments.

[0021] FIG. 2 is a flowchart of the video generation method according to an embodiment of the present disclosure. The execution subject of the video generation method may be a terminal device or a server device. In this embodiment, it is described by taking the execution subject as a terminal device as an example. Referring to FIG. 2, the method includes the following.

[0022] In S201, a first video and video description information of the first video are obtained.

[0023] In an embodiment of the present disclosure, the first video may be an original video of any one video material. For example, the first video may be an original video of a movie, and the first video may be an original video recorded by a user.

[0024] In some embodiments, the original video may be obtained from the local of the terminal device. A correspondence relationship between a video name and the original video is stored in the terminal device. Accordingly, the step of obtaining the first video is as follows. Based on the video name, the original video corresponding to the video name is obtained from the correspondence relationship between the video name and the original video, and the first video is obtained.

[0025] In some other embodiments, the original video may be downloaded from the server. Accordingly, the step of obtaining the first video is as follows: The terminal device sends a video acquisition request to the server, which carries the video name; the server receives the video acquisition request, extracts the original video corresponding to the video name based on the video name, returns the original video to the terminal, and the terminal obtains the first video.

[0026] Video description information may include the video name, images in the video, or a specific video segment in the video. Optionally, as shown in Figure 3, the first video is the original video of a movie, and the video description information for the first video is the movie name. Accordingly, the steps for obtaining the video description information for the first video are as follows: A "Movie Intelligent Digest" control is displayed within the client-side interface. In response to a click on the "Movie Intelligent Digest" control, a video name input box is displayed. In response to detecting the movie name entered by the user via the video name input box, the terminal obtains the video description information for the first video.

[0027] In S202, based on the video description information, introductory text information corresponding to the first video and time node information for at least one video segment in the first video are obtained.

[0028] Optionally, a machine learning model may be used to obtain introductory text information corresponding to the first video and time node information for at least one video segment in the first video. Accordingly, this step is as follows: Video description information is input to a pre-configured machine learning model, and introductory text information corresponding to the first video and time node information for at least one video segment in the first video are output, where the video segment is an important plot segment in the first video, and the machine learning model is used to obtain the video description information of the video and the time node information for at least one video segment in the video.

[0029] As an example, as shown in Figure 3, the video description information is the movie title, and the movie title is input into a machine learning model, which outputs the movie's introductory text information and time node information for important plot points.

[0030] Optionally, the introductory text information may be a textual introduction about the key plot points of the original video. Here, the time node information includes the start time node and end time node of the video segment. Exemplarily, at least one video segment includes video segment 1 and video segment 2, where the time node information for video segment 1 is 00:12:50~00:15:50 and the time node information for video segment 2 is 00:35:50~00:36:50.

[0031] In S203, the first video is divided based on the time node information of at least one video segment to obtain a video frame sequence corresponding to at least one video segment.

[0032] Optionally, the time node information includes the start time node and end time node of the video segment, and accordingly, this step is as follows: Based on the start time node and end time node of each video segment, the first video is divided to extract at least one video segment containing multiple video frames, and the multiple video frames contained in at least one video segment are combined to obtain a video frame sequence corresponding to at least one video segment.

[0033] Here, a video segment includes multiple video frames, and a video frame sequence corresponding to at least one video segment is obtained by rearranging the multiple video frames of at least one video segment.

[0034] In some embodiments, the video frames may be sorted chronologically, and accordingly, the steps of combining multiple video frames contained in at least one video segment to obtain a video frame sequence corresponding to at least one video segment are as follows: Combine multiple video frames contained in at least one video segment according to the chronological order of the video segments to obtain a video frame sequence corresponding to at least one video segment.

[0035] In some other embodiments, the video segments may be sorted according to their plotting importance, and accordingly, the step of combining multiple video frames contained in at least one video segment to obtain a video frame sequence corresponding to at least one video segment is as follows: Combining multiple video frames contained in at least one video segment according to the importance level of the video segments to obtain a video frame sequence corresponding to at least one video segment, where the importance level is used to indicate the importance of the plotting of the video segments.

[0036] Furthermore, as shown in Figure 3, in the process of combining multiple video frames contained in at least one video segment, introductory text information (i.e., video subtitles) may be added to the video frame sequence.

[0037] In S204, the playback time of the video frame sequence is obtained, the introductory text information is converted into audio data of the same playback time, and a second video is generated based on the video frame sequence and audio data.

[0038] In the embodiments of this disclosure, the second video may be a video that introduces the main plot of the first video. Exemplarily, the second video is a video digest of the first video.

[0039] In one embodiment of this disclosure, a character recognition model may be used to convert introductory text information into audio data. As shown in Figure 3, the audio data may be the narration of the original video.

[0040] Optionally, a video frame sequence may be displayed on a video track in the video editing interface, and the playback duration of the video frame sequence can be obtained from the video track's timeline. Accordingly, audio data may be displayed on an audio track in the video editing interface, and the audio track's timeline can be synchronized with the video track's timeline to obtain audio data of the same playback duration.

[0041] Before generating the second video, the video frame sequence and audio data may be previewed or adjusted. Accordingly, generating the second video based on the video frame sequence and audio data, as shown in Figure 4, includes the following steps S401 to S402.

[0042] In S401, a video frame sequence is displayed on the video track in the video editing interface, and audio data is displayed on the audio track in the video editing interface.

[0043] Here, the video editing interface includes video tracks and audio tracks. The video track allows you to adjust the video frame sequence. For example, you may adjust the playback duration and playback speed of the video frame sequence. The audio track allows you to adjust the audio data. For example, you may adjust the playback duration and playback speed of the audio data.

[0044] In some embodiments, a video editing interface may first adjust a video frame sequence and / or audio data before generating a second video. Here, the step of adjusting the video frame sequence and / or audio data includes the step of adjusting the video frame sequence to obtain a new video frame sequence in response to receiving a first editing operation on the video frame sequence, and the step of adjusting the audio data to obtain new audio data in response to receiving a second editing operation on the audio data.

[0045] The embodiments of this disclosure are not specifically limited to the first and second editing operations. Optionally, the first editing operation may include adding effects, adjusting the playback speed of video frames, etc. The second editing operation may include adding background music, adjusting the playback duration, adjusting the playback speed, etc.

[0046] In S402, in response to receiving a second video export operation, the audio data in the audio track and the video frame sequence in the video track are combined to generate the second video.

[0047] Optionally, an export control is displayed within the video editing interface. Accordingly, this step is as follows: In response to receiving a trigger operation on the export control, the system confirms that an export operation for a second video has been received, and combines the audio data in the audio track and the video frame sequence in the video track to generate the second video. Exemplarily, the export control may be a "Video Digest Export" control, and the trigger operation may be a click on the "Video Digest Export" control.

[0048] Furthermore, during the process of exporting the second video, an export information interface for the second video may be displayed. This export information interface may include information such as the video name, duration, data size (storage space occupied), resolution, format, and frame rate.

[0049] The video generation method according to this embodiment, in response to receiving a video identifier, obtains a first video and video description information for the first video; based on the video description information, obtains introductory text information corresponding to the first video and time node information for at least one video segment in the first video; based on the time node information for at least one video segment, divides the first video to obtain a video frame sequence corresponding to at least one video segment; obtains the playback time length of the video frame sequence; converts the introductory text information into audio data of the playback time length; and generates a second video based on the video frame sequence and audio data. Since a video digest of the first video can be automatically generated using the video description information, the efficiency of generating a video digest is higher compared to manually generating a video digest of the original video.

[0050] Figure 5 is a block diagram of the configuration of a video generation device according to an embodiment of the present disclosure. Referring to Figure 5, the device includes a first acquisition module 501, a second acquisition module 502, a splitting module 503, and a generation module 504.

[0051] The first acquisition module 501 is used to acquire the first video and the video description information of the first video.

[0052] The second acquisition module 502 is used to acquire introductory text information corresponding to the first video and time node information for at least one video segment in the first video, based on the video description information.

[0053] The splitting module 503 is used to split the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

[0054] The generation module 504 is used to obtain the playback time length of the video frame sequence, convert the introductory text information into audio data of the playback time length, and generate a second video based on the video frame sequence and the audio data.

[0055] According to one or more embodiments of the present disclosure, the acquisition of introductory text information corresponding to the first video and time node information for at least one video segment in the first video by the second acquisition module 502 based on the video description information specifically includes inputting the video description information into a pre-configured machine learning model and outputting introductory text information corresponding to the first video and time node information for at least one video segment in the first video, wherein the video segment is an important plot segment in the first video, and the machine learning model is used to obtain introductory text information for the video and time node information for at least one video segment in the video from the video description information of the video.

[0056] According to one or more embodiments of the present disclosure, the time node information includes a start time node and an end time node of a video segment, and accordingly, the splitting module 503 splitting the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment specifically includes splitting the first video based on the start time node and end time node of each video segment to extract at least one video segment containing multiple video frames, and combining the multiple video frames contained in the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

[0057] According to one or more embodiments of the present disclosure, the division module 503 combining a plurality of video frames contained in at least one video segment to obtain a video frame sequence corresponding to the at least one video segment specifically includes combining a plurality of video frames contained in at least one video segment in chronological order of the video segments to obtain a video frame sequence corresponding to the at least one video segment, or combining a plurality of video frames contained in at least one video segment in order of importance level of the video segments to obtain a video frame sequence corresponding to the at least one video segment, wherein the importance level is used to indicate the importance of the plot of the video segment.

[0058] According to one or more embodiments of the present disclosure, the generation module 504 generating a second video based on the video frame sequence and the audio data specifically includes displaying the video frame sequence on a video track in a video editing interface and displaying the audio data on an audio track in the video editing interface, and generating the second video by combining the audio data on the audio track and the video frame sequence on the video track in response to receiving an export operation for the second video.

[0059] According to one or more embodiments of the present disclosure, the present invention further includes an adjustment module used to adjust the video frame sequence in response to receiving a first editing operation on the video frame sequence to obtain a new video frame sequence, and to adjust the audio data in response to receiving a second editing operation on the audio data to obtain new audio data.

[0060] Here, the first acquisition module 501, the second acquisition module 502, the splitting module 503, and the generation module 504 are connected in sequence. The video generation device according to this embodiment can implement the technical proposal of the above-described method embodiment, and its implementation principle and technical effects are similar, so this embodiment will not be described further here.

[0061] Referring to Figure 6, which shows a schematic configuration of an electronic device 600 suitable for realizing an embodiment of the present disclosure, the electronic device 600 may be a terminal device or a server. Here, terminal devices may include, but are not limited to, mobile devices such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (abbreviated as PDAs), tablet computers (abbreviated as Portable Android Devices, PADs), portable media players (abbreviated as PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in Figure 6 is merely an example and does not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0062] As shown in Figure 6, the electronic device 600 may include a processing unit (e.g., a central processor, graphics processor, etc.) 601, which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage device 608 into random access memory (RAM) 603. RAM 603 further stores various programs and data necessary for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are connected to each other via bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0063] Generally, devices such as input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, and gyroscopes; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, and vibrators; storage devices 608 including, for example, magnetic tapes and hard disks; and communication devices 609 can be connected to the I / O interface 605. The communication device 609 can allow the electronic device 600 and other devices to exchange data via wireless or wired communication. Figure 6 shows an electronic device 600 with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have all of them instead.

[0064] In particular, according to embodiments of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program on a computer-readable medium, the computer program including program code for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-described functions, limited by the methods of embodiments of the present disclosure, are performed.

[0065] The computer-readable medium in this disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of both. The computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage mediums may include, but are not limited to, an electrical connection having one or more wires, a portable computer magnetic disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact magnetic disk read-only memory (CD-ROM), optical memory device, magnetic memory device, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, the program may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, the computer-readable signal medium may include data signals propagated in the baseband or as part of a carrier, which contain computer-readable program code. The data signals propagated in this manner may be in various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may transmit, propagate, or transmit programs for use by or in combination with instruction execution systems, apparatus, or devices. The program code contained in the computer-readable medium may be transmitted by any suitable medium, including but not limited to electric wires, optical cables, RF (radio frequency), or any suitable combination thereof.

[0066] The computer-readable medium described above may be included in the electronic device described above, or it may be a standalone entity not incorporated into the electronic device.

[0067] The computer-readable medium contains one or more programs, and when these one or more programs are executed by the electronic device, the electronic device is made to perform the method shown in the above embodiment.

[0068] Computer program code for performing the operations of the Disclosure can be organized in one or a variety of programming languages ​​or a combination thereof, and such programming languages ​​include object-oriented programming languages, such as Java, Smalltalk, and C++, and further include general procedural programming languages, such as the "C" language or similar programming languages. The program code may run entirely on a user computer, partially on a user computer, run as a single standalone software package, run partially on a user computer and partially on a remote computer, or run entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it may be connected to an external computer (for example, connected via the Internet using an Internet service provider).

[0069] The flowcharts and block diagrams in the drawings illustrate the systematic architecture, functions, and operations that can be implemented according to the systems, methods, and computer programs of each embodiment of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a defined logic function. It should be noted that in some implementations as alternatives, the functions described in the blocks may occur in a different order than those shown in the drawings. For example, two blocks shown consecutively may actually be executed almost in parallel, or in some cases in reverse order, depending on the function. Furthermore, it should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the defined function or operation, or by a combination of dedicated hardware and computer instructions.

[0070] The units / modules referred to in the descriptions of the embodiments of this disclosure may be implemented in software or in hardware. Herein, the names of the units / modules are not limited in any case to the units / modules themselves; for example, the first acquisition module may be described as a "module for acquiring video and video description information."

[0071] The functions described above in this specification may be performed, at least in part, by one or more hardware logic components. For example, typical types of hardware logic components that can be used include, but are not limited to, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), and composite programmable logic circuits (CPLDs).

[0072] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores a program for use by or in combination with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0073] According to one or more embodiments of this disclosure, a method for generating a video is provided, and the method is Obtaining the first video and the video description information of the first video, Based on the video description information, the following is obtained: introductory text information corresponding to the first video and time node information for at least one video segment in the first video; Based on the time node information of the at least one video segment, the first video is divided to obtain a video frame sequence corresponding to the at least one video segment. This includes obtaining the playback time of the video frame sequence, converting the introductory text information into audio data of the playback time, and generating a second video based on the video frame sequence and the audio data.

[0074] According to one or more embodiments of the present disclosure, obtaining introductory text information corresponding to a first video and time node information for at least one video segment in the first video based on the video description information includes inputting the video description information into a pre-configured machine learning model and outputting introductory text information corresponding to the first video and time node information for at least one video segment in the first video, wherein the video segment is an important plot segment in the first video, and the machine learning model is used to obtain introductory text information for a video and time node information for at least one video segment in a video based on the video description information of the video.

[0075] According to one or more embodiments of the present disclosure, the time node information includes a start time node and an end time node of a video segment, and accordingly, dividing the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment includes dividing the first video based on the start time node and end time node of each video segment to extract at least one video segment containing a plurality of video frames, and combining the plurality of video frames contained in the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

[0076] According to one or more embodiments of the present disclosure, obtaining a video frame sequence corresponding to at least one video segment by combining a plurality of video frames contained in at least one video segment includes obtaining a video frame sequence corresponding to at least one video segment by combining a plurality of video frames contained in at least one video segment in chronological order of the video segments, or by combining a plurality of video frames contained in at least one video segment in order of importance level of the video segments, wherein the importance level is used to indicate the importance of the plot of the video segment.

[0077] According to one or more embodiments of the present disclosure, generating a second video based on the video frame sequence and the audio data includes displaying the video frame sequence on a video track in a video editing interface and displaying the audio data on an audio track in the video editing interface, and generating the second video by combining the audio data on the audio track and the video frame sequence on the video track in response to receiving an export operation for the second video.

[0078] According to one or more embodiments of the present disclosure, the following are further included: adjusting the video frame sequence to obtain a new video frame sequence in response to receiving a first editing operation on the video frame sequence; and adjusting the audio data to obtain new audio data in response to receiving a second editing operation on the audio data.

[0079] According to one or more embodiments of this disclosure, a video generating apparatus is provided, the apparatus is A first acquisition module for acquiring a first video and video description information of the first video, A second acquisition module for obtaining introductory text information corresponding to the first video and time node information for at least one video segment in the first video, based on the video description information, A splitting module for splitting the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment, The system includes a generation module for obtaining the playback time of the video frame sequence, converting the introductory text information into audio data of the playback time, and generating a second video based on the video frame sequence and the audio data.

[0080] According to one or more embodiments of the present disclosure, the second acquisition module acquires introductory text information corresponding to the first video and time node information for at least one video segment in the first video based on the video description information, which specifically includes inputting the video description information into a pre-configured machine learning model and outputting introductory text information corresponding to the first video and time node information for at least one video segment in the first video, wherein the video segment is an important plot segment in the first video, and the machine learning model is used to obtain introductory text information for the video and time node information for at least one video segment in the video based on the video description information of the video.

[0081] According to one or more embodiments of the present disclosure, the time node information includes a start time node and an end time node of a video segment, and accordingly, the splitting module splitting the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment specifically includes splitting the first video based on the start time node and end time node of each video segment to extract at least one video segment containing multiple video frames, and combining the multiple video frames contained in the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

[0082] According to one or more embodiments of the present disclosure, the division module combining a plurality of video frames contained in at least one video segment to obtain a video frame sequence corresponding to the at least one video segment specifically includes combining a plurality of video frames contained in at least one video segment in chronological order of the video segments to obtain a video frame sequence corresponding to the at least one video segment, or combining a plurality of video frames contained in at least one video segment in order of importance level of the video segments to obtain a video frame sequence corresponding to the at least one video segment, wherein the importance level is used to indicate the importance of the plot of the video segment.

[0083] According to one or more embodiments of the present disclosure, the generation module generating a second video based on the video frame sequence and the audio data specifically includes displaying the video frame sequence on a video track in a video editing interface and displaying the audio data on an audio track in the video editing interface, and generating the second video by combining the audio data on the audio track and the video frame sequence on the video track in response to receiving an export operation for the second video.

[0084] According to one or more embodiments of the present disclosure, the present invention further includes an adjustment module used to adjust the video frame sequence in response to receiving a first editing operation on the video frame sequence to obtain a new video frame sequence, and to adjust the audio data in response to receiving a second editing operation on the audio data to obtain new audio data.

[0085] According to one or more embodiments of the present disclosure, an electronic device is provided that includes a processor and a memory that is communicated to the processor. The aforementioned memory stores computer execution instructions. The processor executes computer execution instructions stored in the memory to realize the video generation method described in any one embodiment of the present disclosure.

[0086] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, the computer-readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions, a video generation method described in any one embodiment of the present disclosure is realized.

[0087] The embodiments of this disclosure provide a computer program which, when executed by a processor, realizes a method for generating a video as described in any one embodiment of this disclosure.

[0088] The above description is merely an explanation of preferred embodiments and operational technical principles of the present disclosure. Those skilled in the art will understand that the scope of the disclosure is not limited to technical solutions formed by specific combinations of the above technical features, but encompasses other technical solutions formed by any combination of the above technical features or their equivalents, without departing from the disclosed concepts. For example, technical solutions formed by substituting the above features with similar functional technical features disclosed in this disclosure.

[0089] While the operations are described in a specific order, this should not be understood as requiring them to be performed in a particular order or sequence. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while the above discussion includes some specific implementation details, these should not be understood as limiting the scope of this disclosure. Some features described in the context of a single embodiment may be implemented in combination within a single embodiment. Conversely, various features described in the context of a single embodiment may be implemented in multiple embodiments, either individually or in any suitable subcombination.

[0090] Although this topic has been described using terminology specific to structural features and / or the logical operation of the method, it should be understood that the topic limited to the attached claims is not limited to the specific features or operations described above. Rather, the specific features and operations described above are merely illustrative forms of realizing the claims.

Claims

1. A method for generating a video, To obtain the first video and the video description information of the first video, Based on the video description information, the following is obtained: introductory text information corresponding to the first video and time node information for at least one video segment in the first video; Based on the time node information of the at least one video segment, the first video is divided to obtain a video frame sequence corresponding to the at least one video segment. A method for generating a video, comprising: obtaining the playback time length of the video frame sequence; converting the introductory text information into audio data of the playback time length; and generating a second video based on the video frame sequence and the audio data.

2. Based on the video description information, obtaining introductory text information corresponding to the first video and time node information for at least one video segment in the first video is: The method according to claim 1, comprising inputting the video description information into a pre-configured machine learning model and outputting introductory text information corresponding to the first video and time node information for at least one video segment in the first video, wherein the video segment is an important plot segment in the first video, and the machine learning model is used to obtain introductory text information for the video and time node information for at least one video segment in the video from the video description information of the video.

3. The time node information for each video segment includes the start time node and end time node of the video segment. Based on the time node information of the at least one video segment, the first video is divided to obtain a video frame sequence corresponding to the at least one video segment. Based on the start time node and end time node of each video segment, the first video is divided to extract at least one video segment containing multiple video frames. The method according to claim 1 or 2, comprising combining a plurality of video frames contained in the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

4. Combining multiple video frames contained in the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment is: To obtain a video frame sequence corresponding to the at least one video segment by combining multiple video frames contained in the at least one video segment in the chronological order of the video segments, or The method according to claim 3, comprising combining a plurality of video frames contained in the at least one video segment in order of importance levels of the video segments to obtain a video frame sequence corresponding to the at least one video segment, wherein the importance level is used to indicate the importance of the plot of the video segment.

5. Generating a second video based on the aforementioned video frame sequence and audio data is: Display the video frame sequence on the video track in the video editing interface, and display the audio data on the audio track in the video editing interface. The method according to any one of claims 1 to 4, comprising generating the second video by combining audio data in the audio track and a video frame sequence in the video track in response to receiving the export operation for the second video.

6. The method according to claim 5, further comprising: adjusting the video frame sequence to obtain a new video frame sequence in response to receiving a first editing operation on the video frame sequence; and adjusting the audio data to obtain new audio data in response to receiving a second editing operation on the audio data.

7. A video generation device, A first acquisition module configured to acquire a first video and video description information of the first video, A second acquisition module is configured to acquire introductory text information corresponding to the first video and time node information for at least one video segment in the first video, based on the video description information. A splitting module configured to split the first video based on the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment, A video generation apparatus comprising a generation module configured to obtain the playback time length of the video frame sequence, convert the introductory text information into audio data of the playback time length, and generate a second video based on the video frame sequence and the audio data.

8. An electronic device including a processor and a memory that is communicated to the processor, The aforementioned memory stores computer execution instructions. The processor executes computer execution instructions stored in the memory to realize the video generation method according to any one of claims 1 to 6, in an electronic device.

9. A computer-readable storage medium, wherein computer execution instructions are stored in the computer-readable storage medium, and when a processor executes the computer execution instructions, the video generation method according to any one of claims 1 to 6 is realized.

10. A computer program that, when executed by a processor, realizes the video generation method described in any one of claims 1 to 6.

Images