Video generation method and apparatus, device, and storage medium

Abstract

Embodiments of the present disclosure provide a video generation method and apparatus, a device, and a storage medium. The method comprises: acquiring a first video and video description information of the first video; on the basis of the video description information, acquiring introduction text information corresponding to the first video and time node information of at least one video clip in the first video; on the basis of the time node information of the at least one video clip, segmenting the first video to obtain a video frame sequence corresponding to the at least one video clip; and acquiring a playback duration of the video frame sequence, converting the introduction text information into audio data of the playback duration, and generating a second video on the basis of the video frame sequence and the audio data. Automatic generation of a video summary of the first video can be achieved by means of the video description information, and compared with manual generation of a video summary of an original video, improves the video summary generation efficiency.

Description

Video generation method, device, equipment and storage medium

[0001] Cross-references

[0002] This application claims priority to Chinese patent application No. 202311118817.4 filed on August 31, 2023. The contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0003] Embodiments of the present disclosure relate to a method, apparatus, device, and storage medium for generating a video. Background Art

[0004] Currently, more and more users like to watch video summaries of movie videos to understand the key plots of the movie videos. Among them, the video summary is a short video that extracts the key plot fragments in the movie video and adds corresponding text explanations.

[0005] In some cases, technicians first watch and understand the film video content. Based on their personal understanding, they segment the film video and extract key plot segments. Then, based on their understanding of the film video content, they write a text commentary related to the film video and add the text commentary to the key plot segments to obtain a video summary of the film video.

[0006] However, the inventors found that the existing technology has at least the following technical problems: since technicians are required to first watch and understand the movie video content and then extract key plot clips, it takes a long time to generate a video summary of the movie video, so the efficiency of this video generation method is low.

[0007] Summary of the Invention

[0008] Embodiments of the present disclosure provide a method, apparatus, device, and storage medium for generating a video, which can improve the efficiency of generating a video.

[0009] An embodiment of the present disclosure provides a method for generating a video, comprising:

[0010] Obtaining a first video and video description information of the first video;

[0011] Obtaining, according to the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video;

[0012] Segmenting the first video according to time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment;

[0013] The playing time of the video frame sequence is obtained, the introduction text information is converted into audio data of the playing time, and a second video is generated according to the video frame sequence and the audio data.

[0014] An embodiment of the present disclosure provides a video generation device, comprising:

[0015] A first acquisition module is used to acquire a first video and video description information of the first video;

[0016] A second acquisition module is configured to acquire, based on the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video;

[0017] a segmentation module, configured to segment the first video according to time node information of the at least one video segment, to obtain a video frame sequence corresponding to the at least one video segment;

[0018] The generating module is used to obtain the playing time of the video frame sequence, convert the introduction text information into audio data of the playing time, and generate a second video according to the video frame sequence and the audio data.

[0019] An embodiment of the present disclosure provides an electronic device, including:

[0020] A processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the video generation method described in any embodiment of the present disclosure.

[0021] An embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the video generation method described in any embodiment of the present disclosure is implemented.

[0022] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video generation method described in any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] FIG1 is a schematic diagram of an application scenario of a video generation method provided by an embodiment of the present disclosure;

[0025] FIG2 is a flow chart of a method for generating a video according to an embodiment of the present disclosure;

[0026] FIG3 is a schematic diagram of a method for generating a video according to an embodiment of the present disclosure;

[0027] FIG4 is a flowchart of another method for generating a video according to an embodiment of the present disclosure;

[0028] FIG5 is a structural block diagram of a video generation device provided by an embodiment of the present disclosure; and

[0029] FIG6 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present disclosure.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] Currently, more and more users like to watch video summaries of movie videos to understand the key plots of the movie videos. Among them, the video summary is a short video that extracts the key plot fragments in the movie video and adds corresponding text explanations.

[0033] In some cases, technicians first watch and understand the film video content. Based on their personal understanding, they segment the film video and extract key plot segments. Then, based on their understanding of the film video content, they write a text commentary related to the film video and append the text commentary to the key plot segments, thereby generating a video summary of the film video. However, because technicians need to watch and understand the film video content before extracting the key plot segments, generating a video summary of the film video takes a long time, making this video summary generation method inefficient.

[0034] Therefore, how to improve the generation efficiency of video summaries of movie videos is a technical problem that needs to be solved urgently.

[0035] To address the aforementioned issues, the disclosed embodiments provide the following technical concepts: First, the video file's video identifier is used to obtain the video summary's introductory text and time information for important video segments. Then, based on the time information for these important video segments, the important video segments are extracted from the original video file. Finally, the introductory text is converted into audio data and added to the audio track corresponding to the video frame sequence to obtain the video summary. Because the video identifier can be used to automatically generate a video summary for the original video corresponding to the video identifier, the efficiency of generating video summaries is improved compared to manually generating video summaries for the original video.

[0036] The following explains the application scenarios of the embodiments of the present disclosure:

[0037] The video generation method provided by the embodiment of the present disclosure can be applied to scenarios of generating various types of videos. For example, generating a video summary of a movie video, and for another example, generating a video summary of a game video. Figure 1 is a schematic diagram of an application scenario of a video generation method provided by an embodiment of the present disclosure. As shown in Figure 1, data is transmitted between the terminal 101 and the server 102 via a wired or wireless connection. The terminal 101 can send a request to generate a video summary of a movie video to the server 102. The server 102 receives the request, generates a video summary of the movie video according to the video identifier carried in the request, and returns the video summary of the movie video to the terminal 101. The terminal 101 can play the video summary of the movie video. The video generation method provided by the embodiment of the present disclosure is described in detail using a detailed embodiment below.

[0038] FIG2 is a flow chart of a method for generating a video according to an embodiment of the present disclosure. The execution subject of the method for generating a video can be a terminal device or a server device. This embodiment is described using the terminal device as an example. Referring to FIG2 , the method includes:

[0039] S201: Obtain a first video and video description information of the first video.

[0040] In the embodiment of the present disclosure, the first video may be an original video of any video material. For example, the first video may be an original video of a movie, or the first video may be an original video recorded by a user.

[0041] In some embodiments, the original video can be obtained locally from the terminal device. The terminal device stores a correspondence between video names and original videos. Accordingly, the step of obtaining the first video is: based on the video name, from the correspondence between the video name and the original video, obtain the original video corresponding to the video name to obtain the first video.

[0042] In other embodiments, the original video can be downloaded from a server. Accordingly, the step of obtaining the first video is as follows: the terminal device sends a video acquisition request to the server, the video acquisition request carrying the video name; the server receives the video acquisition request, extracts the original video corresponding to the video name based on the video name, and returns the original video to the terminal, which then obtains the first video.

[0043] The video description information may be the video title, an image in the video, a certain video clip in the video, etc. Optionally, as shown in FIG3 , the first video is the original video of a movie, and the video description information of the first video is the movie title; accordingly, the steps for obtaining the video description information of the first video are: a "movie smart summary" control is displayed in the client interface, and in response to a click operation on the "movie smart summary" control, a video title input box is displayed; in response to detecting the movie title input by the user through the video title input box, the terminal obtains the video description information of the first video.

[0044] S202: Obtain introduction text information corresponding to the first video and time node information of at least one video segment in the first video according to the video description information.

[0045] Optionally, a machine learning model can be used to obtain introductory text information corresponding to the first video and time node information of at least one video segment in the first video. Accordingly, this step is as follows: inputting the video description information into a preset machine learning model, outputting the introductory text information corresponding to the first video and time node information of at least one video segment in the first video, where the video segment is an important plot segment in the first video; wherein the machine learning model is used to obtain the introductory text information of the video and the time node information of at least one video segment in the video through the video description information of the video.

[0046] For example, as shown in FIG3 , the video description information is the movie title. The movie title is input into the machine learning model, and the movie introduction text information and the time node information of important plots are output.

[0047] Optionally, the introductory text information can be a text introduction to important plot points in the original video. The time node information includes the start and end time nodes of the video segments. For example, the at least one video segment includes video segment 1 and video segment 2; the time node information for video segment 1 is: 00:12:50-00:15:50; and the time node information for video segment 2 is: 00:35:50-00:36:50.

[0048] S203: Segment the first video according to time node information of at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

[0049] Optionally, the time node information includes the start time node and the end time node of the video clip; accordingly, this step is: according to the start time node and the end time node of each video clip, the first video is segmented to extract at least one video clip, wherein the video clip includes multiple video frames; the at least one video clip including multiple video frames is combined to obtain a video frame sequence corresponding to the at least one video clip.

[0050] The video segment includes multiple video frames, and a video frame sequence corresponding to the at least one video segment is obtained by sorting the multiple video frames of the at least one video segment.

[0051] In some embodiments, the sorting can be performed according to the time sequence; accordingly, the step of combining at least one video clip including multiple video frames to obtain a video frame sequence corresponding to the at least one video clip is: combining at least one video clip including multiple video frames according to the time sequence of the video clips to obtain a video frame sequence corresponding to the at least one video clip.

[0052] In other embodiments, the video clips can be sorted according to the degree of plot importance; accordingly, the step of combining at least one video clip including multiple video frames to obtain a video frame sequence corresponding to the at least one video clip is: combining at least one video clip including multiple video frames in order of the importance levels of the video clips to obtain a video frame sequence corresponding to the at least one video clip, wherein the importance level is used to represent the degree of plot importance of the video clip.

[0053] It should be noted that, as shown in FIG3 , in the process of combining at least one video segment including multiple video frames, introductory text information (ie, video subtitles) may also be added to the video frame sequence.

[0054] S204: Obtain the playback duration of the video frame sequence, convert the introduction text information into audio data of the playback duration, and generate a second video based on the video frame sequence and the audio data.

[0055] In an embodiment of the present disclosure, the second video may be a video that introduces the main plot of the first video. For example, the second video is a video summary of the first video.

[0056] According to one embodiment of the present disclosure, the introductory text information can be converted into audio data through a text recognition model. As shown in FIG3 , the audio data can be the commentary of the original video.

[0057] Alternatively, a video frame sequence can be displayed on a video track in a video editing interface, and the playback duration of the video frame sequence can be obtained through the timeline of the video track. Correspondingly, audio data can be displayed on an audio track in a video editing interface, and the timeline of the audio track can be aligned with the timeline of the video track to obtain audio data with the same playback duration.

[0058] It should be noted that, before generating the second video, the video frame sequence and the audio data may be previewed or adjusted. Accordingly, as shown in FIG4 , generating the second video according to the video frame sequence and the audio data includes the following steps S401 to S402 .

[0059] S401: Display a video frame sequence on a video track in a video editing interface, and display audio data on an audio track in the video editing interface.

[0060] The video editing interface includes video and audio tracks. The video track allows you to adjust the video frame sequence. For example, you can adjust the playback duration and speed of the video frame sequence. The audio track allows you to adjust the audio data. For example, you can adjust the playback duration and speed of the audio data.

[0061] In some embodiments, the video frame sequence and / or audio data can be adjusted first through the video editing interface, and then the second video can be edited. The step of adjusting the video frame sequence and / or audio data includes: in response to receiving a first editing operation on the video frame sequence, adjusting the video frame sequence to obtain a new video frame sequence; and in response to receiving a second editing operation on the audio data, adjusting the audio data to obtain new audio data.

[0062] In the embodiments of the present disclosure, the first editing operation and the second editing operation are not specifically limited. Optionally, the first editing operation can be adding special effects, adjusting the playback speed of video frames, etc. The second editing operation can be adding background music, adjusting the playback duration, adjusting the playback speed, etc.

[0063] S402: In response to receiving an export operation for a second video, combine the audio data on the audio track with the video frame sequence on the video track to generate a second video.

[0064] Optionally, an export control is displayed within the video editing interface. Accordingly, this step involves: in response to receiving a trigger operation on the export control, determining that an export operation for the second video has been received, combining the audio data on the audio track with the video frame sequence on the video track to generate the second video. Exemplarily, the export control may be an "Export Video Summary" control, and the trigger operation may be a click operation on the "Export Video Summary" control.

[0065] It should be noted that, during the process of exporting the second video, an export information interface of the second video may also be displayed, which may include information such as video name, duration, data size (size of occupied storage space), resolution, format, and frame rate.

[0066] The video generation method provided in this embodiment, in response to receiving a video identifier, obtains a first video and video description information of the first video; based on the video description information, obtains introductory text information corresponding to the first video and time node information of at least one video segment in the first video; based on the time node information of the at least one video segment, segments the first video to obtain a video frame sequence corresponding to the at least one video segment; obtains the playback duration of the video frame sequence, converts the introductory text information into audio data indicating the playback duration, and generates a second video based on the video frame sequence and audio data. Because a video summary of the first video can be automatically generated using the video description information, the efficiency of generating video summaries is improved compared to manually generating video summaries of the original video.

[0067] FIG5 is a structural block diagram of a video generation device provided by an embodiment of the present disclosure. Referring to FIG5 , the device includes: a first acquisition module 501 , a second acquisition module 502 , a segmentation module 503 , and a generation module 504 .

[0068] The first acquisition module 501 is configured to acquire a first video and video description information of the first video.

[0069] The second acquisition module 502 is configured to acquire, according to the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video.

[0070] The segmentation module 503 is configured to segment the first video according to the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment.

[0071] The generating module 504 is configured to obtain the playing duration of the video frame sequence, convert the introduction text information into audio data of the playing duration, and generate a second video based on the video frame sequence and the audio data.

[0072] According to one or more embodiments of the present disclosure, the second acquisition module 502 obtains the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video based on the video description information, specifically including: inputting the video description information into a preset machine learning model, outputting the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video, wherein the video segment is an important plot segment in the first video; wherein the machine learning model is used to obtain the introductory text information of the video and the time node information of at least one video segment in the video through the video description information of the video.

[0073] According to one or more embodiments of the present disclosure, the time node information includes the start time node and the end time node of the video segment; accordingly, the segmentation module 503 segments the first video according to the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment, specifically including: segmenting the first video according to the start time node and the end time node of each video segment to extract at least one video segment, wherein the video segment includes multiple video frames; combining at least one video segment including multiple video frames to obtain a video frame sequence corresponding to the at least one video segment.

[0074] According to one or more embodiments of the present disclosure, the segmentation module 503 combines at least one of the video clips including multiple video frames to obtain a video frame sequence corresponding to the at least one video clip, specifically including: combining at least one of the video clips including multiple video frames in chronological order of the video clips to obtain a video frame sequence corresponding to the at least one video clip; or combining at least one of the video clips including multiple video frames in order of importance levels of the video clips to obtain a video frame sequence corresponding to the at least one video clip, wherein the importance level is used to represent the degree of plot importance of the video clip.

[0075] According to one or more embodiments of the present disclosure, the generation module 504 generates a second video based on the video frame sequence and the audio data, specifically including: displaying the video frame sequence on a video track in a video editing interface, and displaying the audio data on an audio track in the video editing interface; in response to receiving an export operation for the second video, combining the audio data on the audio track with the video frame sequence on the video track to generate the second video.

[0076] According to one or more embodiments of the present disclosure, it also includes: an adjustment module; the adjustment module is used to adjust the video frame sequence in response to receiving a first editing operation on the video frame sequence to obtain a new video frame sequence; and, in response to receiving a second editing operation on the audio data, adjust the audio data to obtain new audio data.

[0077] The first acquisition module 501, the second acquisition module 502, the segmentation module 503 and the generation module 504 are connected in sequence. The video generation device provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.

[0078] Referring to FIG6 , a schematic diagram of the structure of an electronic device 600 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 600 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG6 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0079] As shown in Figure 6, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0080] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 6 shows an electronic device 600 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0081] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0082] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0083] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0084] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0085] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0087] The units / modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the names of the units / modules do not limit the units / modules themselves. For example, the first acquisition module may also be described as a "module for acquiring video and video description information."

[0088] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0089] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] According to one or more embodiments of the present disclosure, a method for generating a video is provided, comprising:

[0091] Obtaining a first video and video description information of the first video;

[0092] Obtaining, according to the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video;

[0093] Segmenting the first video according to time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment;

[0094] The playing time of the video frame sequence is obtained, the introduction text information is converted into audio data of the playing time, and a second video is generated according to the video frame sequence and the audio data.

[0095] According to one or more embodiments of the present disclosure, obtaining the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video based on the video description information includes: inputting the video description information into a preset machine learning model, outputting the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video, wherein the video segment is an important plot segment in the first video; wherein the machine learning model is used to obtain the introductory text information of the video and the time node information of at least one video segment in the video through the video description information of the video.

[0096] According to one or more embodiments of the present disclosure, the time node information includes the start time node and the end time node of the video segment; accordingly, the first video is segmented according to the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment, including: segmenting the first video according to the start time node and the end time node of each video segment to extract at least one video segment, wherein the video segment includes multiple video frames; combining at least one of the video segments including multiple video frames to obtain a video frame sequence corresponding to the at least one video segment.

[0097] According to one or more embodiments of the present disclosure, wherein at least one of the video clips includes a plurality of video frames and is combined to obtain a video frame sequence corresponding to the at least one video clip, including: combining at least one of the video clips including a plurality of video frames in chronological order of the video clips to obtain a video frame sequence corresponding to the at least one video clip; or combining at least one of the video clips including a plurality of video frames in order of importance levels of the video clips to obtain a video frame sequence corresponding to the at least one video clip, wherein the importance level is used to represent the degree of plot importance of the video clip.

[0098] According to one or more embodiments of the present disclosure, generating a second video based on the video frame sequence and the audio data includes: displaying the video frame sequence on a video track in a video editing interface, and displaying the audio data on an audio track in the video editing interface; in response to receiving an export operation for the second video, combining the audio data on the audio track with the video frame sequence on the video track to generate the second video.

[0099] According to one or more embodiments of the present disclosure, it also includes: in response to receiving a first editing operation on the video frame sequence, adjusting the video frame sequence to obtain a new video frame sequence; and in response to receiving a second editing operation on the audio data, adjusting the audio data to obtain new audio data.

[0100] According to one or more embodiments of the present disclosure, a video generation apparatus is provided, including:

[0101] A first acquisition module is used to acquire a first video and video description information of the first video;

[0102] A second acquisition module is configured to acquire, based on the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video;

[0103] a segmentation module, configured to segment the first video according to time node information of the at least one video segment, to obtain a video frame sequence corresponding to the at least one video segment;

[0104] The generating module is used to obtain the playing time of the video frame sequence, convert the introduction text information into audio data of the playing time, and generate a second video according to the video frame sequence and the audio data.

[0105] According to one or more embodiments of the present disclosure, the second acquisition module obtains the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video based on the video description information, specifically including: inputting the video description information into a preset machine learning model, outputting the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video, wherein the video segment is an important plot segment in the first video; wherein the machine learning model is used to obtain the introductory text information of the video and the time node information of at least one video segment in the video through the video description information of the video.

[0106] According to one or more embodiments of the present disclosure, the time node information includes the start time node and the end time node of the video clip; accordingly, the segmentation module segments the first video according to the time node information of the at least one video clip to obtain a video frame sequence corresponding to the at least one video clip, specifically comprising: segmenting the first video according to the start time node and the end time node of each video clip to extract at least one video clip, wherein the video clip includes multiple video frames; combining at least one of the video clips including multiple video frames to obtain a video frame sequence corresponding to the at least one video clip.

[0107] According to one or more embodiments of the present disclosure, the segmentation module combines at least one of the video clips including multiple video frames to obtain a video frame sequence corresponding to the at least one video clip, specifically including: combining at least one of the video clips including multiple video frames in chronological order of the video clips to obtain a video frame sequence corresponding to the at least one video clip; or combining at least one of the video clips including multiple video frames in order of importance levels of the video clips to obtain a video frame sequence corresponding to the at least one video clip, wherein the importance level is used to represent the degree of plot importance of the video clip.

[0108] According to one or more embodiments of the present disclosure, the generation module generates a second video based on the video frame sequence and the audio data, specifically including: displaying the video frame sequence on a video track in a video editing interface, and displaying the audio data on an audio track in the video editing interface; in response to receiving an export operation for the second video, combining the audio data on the audio track with the video frame sequence on the video track to generate the second video.

[0109] According to one or more embodiments of the present disclosure, it also includes: an adjustment module; the adjustment module is used to adjust the video frame sequence in response to receiving a first editing operation on the video frame sequence to obtain a new video frame sequence; and, in response to receiving a second editing operation on the audio data, adjust the audio data to obtain new audio data.

[0110] According to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0111] The memory stores computer-executable instructions;

[0112] The processor executes the computer-executable instructions stored in the memory to implement the video generation method described in any embodiment of the present disclosure.

[0113] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the video generation method described in any embodiment of the present disclosure is implemented.

[0114] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video generation method described in any embodiment of the present disclosure.

[0115] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0116] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0117] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for generating a video, comprising: Obtaining a first video and video description information of the first video; According to the video description information, obtaining introduction text information corresponding to the first video and time node information of at least one video segment in the first video; Segmenting the first video according to time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment; The playing time of the video frame sequence is obtained, the introduction text information is converted into audio data of the playing time, and a second video is generated according to the video frame sequence and the audio data.

2. The method according to claim 1, wherein: The acquiring, according to the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video includes: The video description information is input into a preset machine learning model, and the introductory text information corresponding to the first video and the time node information of at least one video segment in the first video are output, wherein the video segment is an important plot segment in the first video; wherein the machine learning model is used to obtain the introductory text information of the video and the time node information of at least one video segment in the video through the video description information of the video.

3. The method according to claim 1 or 2, wherein: The time node information of each video segment includes a start time node and an end time node of the video segment; The step of segmenting the first video according to the time node information of the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment includes: Segmenting the first video according to the start time node and the end time node of each video segment to extract the at least one video segment, wherein the video segment includes a plurality of video frames; The multiple video frames included in the at least one video segment are combined to obtain a video frame sequence corresponding to the at least one video segment.

4. The method according to claim 3, wherein: Combining the multiple video frames included in the at least one video segment to obtain a video frame sequence corresponding to the at least one video segment includes: Combining multiple video frames included in the at least one video segment in a time sequence of the video segments to obtain a video frame sequence corresponding to the at least one video segment; or, According to the order of importance of the video segments, the plurality of The video frames are combined to obtain a video frame sequence corresponding to the at least one video segment, wherein the importance level is used to indicate the importance of the plot of the video segment.

5. The method according to any one of claims 1 to 4, wherein: The step of generating a second video according to the video frame sequence and the audio data comprises: Displaying the video frame sequence on a video track in a video editing interface, and displaying the audio data on an audio track in the video editing interface; In response to receiving an export operation of the second video, the audio data on the audio track is combined with the video frame sequence on the video track to generate the second video.

6. The method according to claim 5, further comprising: In response to receiving a first editing operation on the video frame sequence, adjusting the video frame sequence to obtain a new video frame sequence; And, in response to receiving a second editing operation on the audio data, adjusting the audio data to obtain new audio data.

7. A video generation device, comprising: A first acquisition module is configured to acquire a first video and video description information of the first video; A second acquisition module is configured to acquire, according to the video description information, introduction text information corresponding to the first video and time node information of at least one video segment in the first video; a segmentation module, configured to segment the first video according to time node information of the at least one video segment, to obtain a video frame sequence corresponding to the at least one video segment; The generating module is configured to obtain the playing time of the video frame sequence, convert the introduction text information into audio data of the playing time, and generate a second video according to the video frame sequence and the audio data.

8. An electronic device comprising: A processor, and a memory communicatively connected to the processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the video generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method for generating a video as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method for generating a video according to any one of claims 1 to 6 is implemented.

Images