Short video generation method and device

Abstract

The embodiment of the invention provides a short video generation method, and the method comprises the steps: automatically recognizing a character dialogue in a to-be-processed long video through a large model of automatic voice recognition and deep learning, and automatically intercepting a wonderful segment in the long video according to the semantic analysis of the character dialogue; and combining the wonderful clips, and automatically adding a transition special effect at a switching position between different clips to obtain an edited short video. According to the technical scheme provided by the embodiment of the invention, the bright spot of the video can be more accurately captured when the short video is generated, and the visual effect of the short video is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio and video processing technology, and in particular, to a short video generation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rise of short video platforms, short videos have become one of the main ways for users to obtain information and entertainment. In contrast, although long videos are rich in content and information, they are often too long to meet the fragmented viewing needs of users. Therefore, converting long video content into exciting clips suitable for dissemination on short video platforms has become an important means to increase the exposure of long videos and user stickiness.

[0003] In the prior art, the wonderful clips in the long video are usually selected manually and then edited to generate short videos. However, this method is time-consuming and labor-intensive, and it is difficult to ensure the accuracy and coherence of the video content.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a short video generation method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a short video generation method, the method comprising: Through automatic speech recognition and a large model of deep learning, the dialogues of characters in the long video to be processed are automatically identified, and the highlights of the long video are automatically captured based on the semantic analysis of the dialogues of the characters; The wonderful clips are combined, and transition effects are automatically added at the switching points between different clips to obtain a edited short video.

[0007] Optionally, the method of automatically identifying character dialogues in a long video to be processed by using a large model of automatic speech recognition and deep learning, and automatically capturing highlights in the long video based on semantic analysis of the character dialogues, includes: Automatically perform speech recognition on the long video to be processed, extract the conversation content of the characters in the long video and the timestamps corresponding to the conversation content of the characters, and obtain a recognition result; Inputting the recognition result into a large model of deep learning, and automatically selecting at least one wonderful dialogue from the dialogue content of the characters by performing semantic analysis on the dialogue content of the characters in the recognition result; According to the wonderful dialogue selected by the large model, by matching the timestamp in the recognition result, the video content corresponding to the wonderful dialogue is extracted from the long video to obtain the wonderful clip.

[0008] Optionally, after extracting the video content corresponding to the wonderful conversation from the long video, the method further includes: Obtaining the start and end timestamps of each character dialogue in the video content corresponding to the wonderful dialogue according to the recognition result; Determine, according to the start and end timestamps, a portion of the video content in which the density of character dialogue is lower than a preset condition; The portion where the character dialogue density is lower than a preset condition is deleted from the video content to obtain the wonderful segment.

[0009] Optionally, the method further comprises: Automatically adjust the edited short video to convert the horizontal screen to the vertical screen ratio.

[0010] Optionally, the automatically adjusting the edited short video picture includes: Performing person detection on the short video screen, and determining whether the short video screen is a close-up screen of a person or a large scene screen according to the person detection result; The character close-up picture is converted into a vertical screen ratio by automatic cropping, and the large scene picture is converted into a vertical screen ratio by adding blurring effects on the upper and lower sides.

[0011] Optionally, performing person detection on the short video picture, and determining whether the short video picture is a close-up picture of a person or a large scene picture according to the person detection result includes: Detecting the position and size of a person in each frame of the short video to obtain a person recognition frame; When the size of the person recognition frame in the current frame exceeds a preset threshold, determining that the current frame is a close-up picture of the person; When the size of the person recognition frame in the current frame does not exceed a preset threshold, the current frame is determined to be a large scene picture.

[0012] Optionally, the method further comprises: In the process of automatic cropping, the jitter during picture cropping is eliminated by introducing an inter-frame smoothing algorithm.

[0013] Another aspect of an embodiment of the present application provides a short video generation device, the device comprising: A screening module is used to automatically identify character dialogues in a long video to be processed through automatic speech recognition and a deep learning model, and automatically capture highlights from the long video based on semantic analysis of the character dialogues; The editing module is used to combine the wonderful clips and automatically add transition effects at the switching points between different clips to obtain an edited short video.

[0014] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0015] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0016] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0017] The above technical solution adopted in the embodiment of the present application may have the following advantages: By introducing ASR recognition technology and a large model of deep learning, character dialogues and highlights in long videos can be automatically identified. By extracting key information from the video, the highlights of the video can be captured more accurately when generating short videos. In addition, the embodiment of the present application also improves the visual effect of the short video through character detection and screen cropping. In particular, when processing vertical screen conversion, by introducing an inter-frame smoothing algorithm, the jitter problem during screen cropping is effectively eliminated, making the short video more in line with the viewing experience of mobile devices. The embodiment of the present application can significantly improve production efficiency while ensuring the quality of short videos, providing strong technical support for the promotion of long videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0019] Figure 1 A diagram schematically shows an operating environment of a short video generation method according to Embodiment 1 of the present application; Figure 2 The flowchart of the short video generation method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 A detailed flow chart of step S200; Figure 4 Schematically shows Figure 2 Another detailed flow chart of step S200; Figure 5 A flowchart of an optional embodiment of the short video generation method proposed in Embodiment 1 of the present application is schematically shown; Figure 6 Schematically shows Figure 5 A detailed flow chart of step S304; Figure 7 Schematically shows Figure 6 A detailed flow chart of step S3040; Figure 8 A block diagram of a short video generation device according to Embodiment 2 of the present application is schematically shown; and Fig. 9 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0021] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0022] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0023] First, the following terms are explained: Long video: usually refers to video content that is longer than 15 minutes, commonly seen in movies, TV series, or complete documentaries.

[0024] Short videos: Video content with a shorter length, generally 30 seconds to 5 minutes, suitable for rapid dissemination on social media platforms.

[0025] Automatic Speech Recognition (ASR): Identify the conversation content in a video by converting the speech in the video into text.

[0026] Big model: refers to an artificial intelligence model trained through deep learning that can handle complex tasks, such as selecting highlights of video content.

[0027] Transition effects: effects used in video editing to make the switching between different scenes more natural and reduce visual abruptness.

[0028] Vertical video conversion: The process of converting a horizontal video into a vertical video suitable for mobile phone viewing, achieved through cropping and image adjustment.

[0029] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: The existing technology for generating short videos from long videos mainly relies on manual editing, which is inefficient and the understanding and refinement of video content is not accurate enough. In addition, although the existing automation technology has improved efficiency to a certain extent, it is often difficult to balance the rhythm and information density of short videos, resulting in low quality of generated short videos. In addition, when converting horizontal videos to vertical videos, the cropping and adjustment of the picture are also prone to problems such as unreasonable character composition or poor visual effects. Especially when dealing with large scene pictures, it is difficult to maintain the continuity and naturalness of the picture. Therefore, how to convert long videos into exciting short videos more efficiently and intelligently has become a technical problem that needs to be solved urgently.

[0030] Aims to overcome the deficiencies in existing technologies. To this end, the embodiment of the present application provides a short video generation technology solution. In this technical solution, by introducing ASR recognition technology and a large model of deep learning, character dialogues and highlights in long videos can be automatically identified, and the highlights of the video can be captured more accurately when generating short videos. The visual effect of the short video is also improved by character detection and screen cropping. See below for details.

[0031] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0032] See also Figure 1In this operating environment diagram, the present application can be applied to application environments including, but not limited to, a service platform 2, a client 4, and a network 6.

[0033] The client 4 is used to display the interface of the current application to the user and receive the user's operation. The client 4 can be a terminal device such as a PC (Personal Computer), a mobile phone, a tablet computer, a portable computer, a wearable device, etc.

[0034] The service platform 2 is used to provide data and technical support for the client 4. The service platform 2 can be a computing device such as a rack server, a blade server, a tower server or a cabinet server, and can be an independent server or a server cluster composed of multiple servers.

[0035] The network 6 can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), the Wideband Code Division Multiple Access (WCDMA), the 4G network, the 5G network, Bluetooth, Wi-Fi, etc. The service platform 2 and one or more of the clients 4 are connected to each other through the network 6 for data transmission and interaction.

[0036] Embodiment 1 Figure 2 The flowchart of the short video generation method according to the first embodiment of the present application is schematically shown. It can be understood that the flowchart in the embodiment of the present method is not used to limit the order of executing the steps. As needed, some steps in the flowchart can also be added or deleted. The execution subject of the short video generation method can be the service platform or the client.

[0037] like Figure 2 As shown, the short video generation method may include steps S200 to S202, wherein: Step S200, automatically identifying the character dialogues in the long video to be processed through the large model of ASR recognition and deep learning, and automatically capturing the wonderful clips in the long video based on the semantic analysis of the character dialogues.

[0038] In this embodiment, ASR recognition is first used to automatically identify the dialogues between characters in the long video to be processed, and then a large model of deep learning is used to perform semantic analysis on the dialogues between characters based on the results of the above recognition, thereby automatically capturing the wonderful clips in the long video.

[0039] Step S202, combining the wonderful clips and automatically adding transition effects at the switching points between different clips to obtain a edited short video.

[0040] Considering the fast tempo of short videos, in order to connect the switching between different clips in the previous step, this embodiment will also automatically add transition effects at the switching point. These special effects can reduce the sense of fragmentation when the scene switches, making the entire video smoother and more natural.

[0041] The short video generation method provided in this embodiment automatically recognizes character dialogues and highlights in long videos by introducing ASR recognition technology and a large model of deep learning. By extracting key information from the video, the highlights of the video can be captured more accurately when generating a short video.

[0042] Further reading Figure 3 , is a detailed flow chart of the above step S200. In this embodiment, the step S200 specifically includes: Step S2000, performing ASR recognition on the long video to be processed, extracting the conversation content of the characters in the long video and the timestamps corresponding to the conversation content of the characters, and obtaining a recognition result.

[0043] First, perform ASR recognition on the long video to extract the conversation content (i.e., text) and the timestamps corresponding to the conversation content. ASR technology can convert the speech in the long video into text and mark the timestamp of each conversation. These timestamps will be used for subsequent highlight clip selection and video editing.

[0044] Step S2002: input the recognition result into a large model of deep learning, and automatically select at least one wonderful dialogue from the character dialogue content by performing semantic analysis on the character dialogue content in the recognition result.

[0045] Next, the ASR recognition results are input into the deep learning big model, which automatically selects the wonderful dialogues (i.e., text fragments) from the character dialogues by semantically analyzing the text content of the character dialogues in the recognition results. When selecting the wonderful dialogues, the big model will combine multiple factors such as the dialogue content, the character's emotions, and the overall plot to ensure that the selected wonderful dialogues are both interesting and can attract the audience's attention.

[0046] Step S2004, based on the wonderful dialogue selected by the large model, by matching the timestamp in the recognition result, the video content corresponding to the wonderful dialogue is extracted from the long video to obtain the wonderful clip.

[0047] According to the wonderful dialogue selected by the large model, the corresponding video content can be intercepted by matching the timestamp in the ASR recognition result. Specifically, the ASR recognition algorithm will mark the start time and end time of each dialogue, and the video screen during this period can be extracted according to the marking. By intercepting the video content corresponding to the wonderful dialogue in the above manner and obtaining the wonderful clip (i.e., video clip), it can be ensured that the generated short video contains complete dialogues and key scenes.

[0048] Further reading Figure 4 , is another detailed flow chart of the above step S200. In this embodiment, the step S200 further includes: Step S2006, obtaining the start and end timestamps of each character dialogue in the video content corresponding to the wonderful dialogue according to the recognition result.

[0049] Since the ASR recognition algorithm marks the start and end time of each dialogue, the start and end timestamps of each character dialogue identified in the video content corresponding to the wonderful dialogue can be obtained from the recognition result. The start and end timestamps can be used to determine which parts of the wonderful dialogue have character dialogues and which parts do not.

[0050] Step S2007, determining a portion of the video content where the character dialogue density is lower than a preset condition according to the start and end timestamps.

[0051] In this embodiment, in order to make the wonderful clips highlight key information, it is necessary to further determine whether there is a part without dialogue or less dialogue in the video content corresponding to the wonderful dialogue. Specifically, a dialogue density condition can be preset, such as a dialogue word count threshold in a certain time period. Then, the dialogue density in each time window in the video content corresponding to the wonderful dialogue is counted using the time period as a time window, and then compared with the preset condition. If the dialogue density does not reach the preset condition, it means that the dialogue density in the time period is low, that is, the video content in the time period is not very important.

[0052] Step S2008, deleting the portion where the character dialogue density is lower than a preset condition from the video content to obtain the wonderful segment.

[0053] According to the ASR recognition results and the above comparison, if there is no character dialogue or less dialogue for a long time in a certain time period, it is considered that this part contributes little to the information transmission and can be deleted. While retaining the wonderful content, the length of the short video is compressed as much as possible. The specific deletion method can be set according to the actual situation and is not limited here. For example, the end timestamp of the previous dialogue in the time period and the start timestamp of the next dialogue can be obtained, and then the part without character dialogue in the time period can be deleted according to these two timestamps, and so on.

[0054] The above processing can improve the information density of the short video, highlight key information, and further optimize the rhythm and duration of the short video.

[0055] like Figure 5 As shown, it is a flowchart of an optional embodiment of the short video generation method proposed in the first embodiment of the present application. It can be understood that the flowchart is not used to limit the order of executing the steps. Some steps in the flowchart can also be added or deleted as needed.

[0056] In an optional embodiment, the method Figure 2-4 On the basis of, step S304 is also included. Specifically, the method includes: Step S300, automatically identifying the character dialogues in the long video to be processed through automatic speech recognition and a large model of deep learning, and automatically capturing the wonderful clips in the long video based on the semantic analysis of the character dialogues.

[0057] Step S302, combining the wonderful clips and automatically adding transition effects at the switching points between different clips to obtain a edited short video.

[0058] The implementation process of the above steps S300-S302 is the same as Figure 2 Steps S200-S202 in are similar and will not be repeated here.

[0059] Step S304, automatically adjusting the edited short video screen to convert the horizontal screen into a vertical screen ratio.

[0060] Since the main playback device for short videos is currently a mobile phone, in order to adapt to the viewing experience of mobile phone screens, this embodiment also introduces a technology for vertical video conversion. This technology can automatically convert the horizontal video screen into a vertical video screen suitable for mobile phone viewing through screen cropping and screen adjustment.

[0061] Further reading Figure 6 , is a detailed flow chart of the above step S304. In this embodiment, the step S304 specifically includes: Step S3040: Perform person detection on the short video frame, and determine whether the short video frame is a close-up frame of the person or a large scene frame according to the person detection result.

[0062] Step S3042: The character close-up image is converted into a vertical screen ratio by automatic cropping, and the large scene image is converted into a vertical screen ratio by adding blurring effects on the upper and lower sides.

[0063] Further reading Figure 7 , is a detailed flow chart of the above step S3040. In this embodiment, the step S3040 specifically includes: Step S400, detecting the position and size of a person in each frame of the short video to obtain a person recognition frame; In this embodiment, the detection may adopt any existing feasible person detection algorithm, which is not limited here. The person identification frame is the smallest rectangular frame that completely contains the person in the picture of the current frame.

[0064] Step S402, compare the size of the person identification frame in the current frame with a preset threshold. If the size of the person identification frame in the current frame exceeds the preset threshold, execute step S404; otherwise, if the size of the person identification frame in the current frame does not exceed the preset threshold, execute step S406.

[0065] Step S404, determining that the current frame is a close-up picture of a person.

[0066] Step S406, determining that the current frame is a large scene picture.

[0067] In this embodiment, different processing methods are used for the character close-up picture or the large scene picture to achieve vertical screen video conversion. Among them, for the character close-up picture, the picture will be automatically cropped according to the character's position and size, while retaining the main part of the character and making it conform to the vertical screen ratio. For the large scene picture, a sandwich form is adopted, that is, a blur effect is added to the upper and lower sides of the original picture to convert it into a vertical screen video picture.

[0068] In other optional embodiments, different thresholds may be used to determine the character close-up image and the large scene image, which will not be described in detail here.

[0069] In addition, in an optional embodiment, in order to avoid jitter problems when cropping the screen during the vertical screen conversion process, this embodiment adopts an inter-frame smoothing algorithm. Specifically, the algorithm adjusts the position and size of the cropping frame by analyzing the changes in the character recognition frame detected between consecutive frames, so that the close-up picture of the character in the cropped short video is smoother and more stable, thereby improving the user's viewing experience.

[0070] Through the above steps, the embodiment of the present application can automatically convert long videos into short videos with compact rhythm and excellent visual effects, greatly improving the dissemination efficiency of video content.

[0071] The short video generation method provided in this embodiment can automatically identify character dialogues and highlights in long videos by introducing ASR recognition technology and a large model of deep learning. By extracting key information from the video, the highlights of the video can be captured more accurately when generating a short video. In addition, this method also improves the visual effect of the short video by detecting characters and cropping the screen. In particular, when processing vertical screen conversion, by introducing an inter-frame smoothing algorithm, the jitter problem during screen cropping is effectively eliminated, making the short video more suitable for the viewing experience of mobile devices. This method can significantly improve production efficiency while ensuring the quality of short videos, providing strong technical support for the promotion of long videos.

[0072] Embodiment 2 Figure 8 The block diagram of the short video generation device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 8 As shown, the device 60 may include: a screening module 600 and a clipping module 602, wherein: The screening module 600 is used to automatically identify character dialogues in a long video to be processed through automatic speech recognition and a large model of deep learning, and automatically capture wonderful clips in the long video based on semantic analysis of the character dialogues.

[0073] The editing module 602 is used to combine the wonderful clips and automatically add transition effects at the switching points between different clips to obtain an edited short video.

[0074] In an optional embodiment, the screening module 600 is specifically used for: Automatically perform speech recognition on the long video to be processed, extract the conversation content of the characters in the long video and the timestamps corresponding to the conversation content of the characters, and obtain a recognition result; Inputting the recognition result into a large model of deep learning, and automatically selecting at least one wonderful dialogue from the dialogue content of the characters by performing semantic analysis on the dialogue content of the characters in the recognition result; According to the wonderful dialogue selected by the large model, by matching the timestamp in the recognition result, the video content corresponding to the wonderful dialogue is extracted from the long video to obtain the wonderful clip.

[0075] In an optional embodiment, after extracting the video content corresponding to the wonderful dialogue from the long video, the screening module 600 further includes: Obtaining the start and end timestamps of each character dialogue in the video content corresponding to the wonderful dialogue according to the recognition result; Determine, according to the start and end timestamps, a portion of the video content in which the density of character dialogue is lower than a preset condition; The portion where the character dialogue density is lower than a preset condition is deleted from the video content to obtain the wonderful segment.

[0076] In an optional embodiment, the editing module 602 is further configured to: Automatically adjust the edited short video to convert the horizontal screen to the vertical screen ratio.

[0077] In an optional embodiment, the automatically adjusting the edited short video picture includes: Performing person detection on the short video screen, and determining whether the short video screen is a close-up screen of a person or a large scene screen according to the person detection result; The character close-up picture is converted into a vertical screen ratio by automatic cropping, and the large scene picture is converted into a vertical screen ratio by adding blurring effects on the upper and lower sides.

[0078] In an optional embodiment, performing person detection on the short video picture, and determining whether the short video picture is a close-up picture of a person or a large scene picture according to the person detection result includes: Detecting the position and size of a person in each frame of the short video to obtain a person recognition frame; When the size of the person recognition frame in the current frame exceeds a preset threshold, determining that the current frame is a close-up picture of the person; When the size of the person recognition frame in the current frame does not exceed a preset threshold, the current frame is determined to be a large scene picture.

[0079] In an optional embodiment, the editing module 602 is further configured to: In the process of automatic cropping, the jitter during picture cropping is eliminated by introducing an inter-frame smoothing algorithm.

[0080] The specific implementation process of the above-mentioned functions of each module can be found in the description of the above-mentioned embodiment 1, and will not be repeated here.

[0081] Embodiment 3 Fig. 9 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the short video generation method according to the third embodiment of the present application is schematically shown. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig. 9 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed on the computer device 10000, such as program code of a short video generation method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0082] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0083] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0084] It should be pointed out that Fig. 9 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0085] In this embodiment, the short video generation method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0086] Embodiment 4 An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the short video generation method in the embodiment are implemented.

[0087] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the short video generation method in the embodiment, etc. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0088] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0089] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0090] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A short video generation method, characterized in that: The method comprises: Through automatic speech recognition and a large model of deep learning, the dialogues of characters in the long video to be processed are automatically identified, and the highlights of the long video are automatically captured based on the semantic analysis of the dialogues of the characters; The wonderful clips are combined, and transition effects are automatically added at the switching points between different clips to obtain a edited short video.

2. The method according to claim 1, characterized in that The large model of automatic speech recognition and deep learning is used to automatically identify the dialogues of characters in the long video to be processed, and the wonderful clips in the long video are automatically intercepted according to the semantic analysis of the dialogues of the characters, including: Automatically perform speech recognition on the long video to be processed, extract the conversation content of the characters in the long video and the timestamps corresponding to the conversation content of the characters, and obtain a recognition result; Inputting the recognition result into a large model of deep learning, and automatically selecting at least one wonderful dialogue from the dialogue content of the characters by performing semantic analysis on the dialogue content of the characters in the recognition result; According to the wonderful dialogue selected by the large model, by matching the timestamp in the recognition result, the video content corresponding to the wonderful dialogue is extracted from the long video to obtain the wonderful clip.

3. The method according to claim 2, characterized in that After extracting the video content corresponding to the wonderful dialogue from the long video, the method further includes: Obtaining the start and end timestamps of each character dialogue in the video content corresponding to the wonderful dialogue according to the recognition result; Determine, according to the start and end timestamps, a portion of the video content in which the density of character dialogue is lower than a preset condition; The portion where the character dialogue density is lower than a preset condition is deleted from the video content to obtain the wonderful segment.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Automatically adjust the edited short video to convert the horizontal screen to the vertical screen ratio.

5. The method according to claim 4, characterized in that The automatic adjustment of the edited short video screen includes: Performing person detection on the short video screen, and determining whether the short video screen is a close-up screen of a person or a large scene screen according to the person detection result; The character close-up picture is converted into a vertical screen ratio by automatic cropping, and the large scene picture is converted into a vertical screen ratio by adding blurring effects on the upper and lower sides.

6. The method according to claim 5, characterized in that The performing person detection on the short video picture and determining whether the short video picture is a person close-up picture or a large scene picture according to the person detection result comprises: Detecting the position and size of a person in each frame of the short video to obtain a person recognition frame; When the size of the person recognition frame in the current frame exceeds a preset threshold, determining that the current frame is a close-up picture of the person; When the size of the person recognition frame in the current frame does not exceed a preset threshold, the current frame is determined to be a large scene picture.

7. The method according to claim 5, characterized in that The method further comprises: In the process of automatic cropping, the jitter during picture cropping is eliminated by introducing an inter-frame smoothing algorithm.

8. A short video generation device, characterized in that: The device comprises: A screening module is used to automatically identify the dialogues of characters in the long video to be processed through automatic speech recognition and a large model of deep learning, and automatically capture the highlights of the long video based on the semantic analysis of the dialogues of the characters; The editing module is used to combine the wonderful clips and automatically add transition effects at the switching points between different clips to obtain an edited short video.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images