Video editing method and device

Abstract

The embodiment of the invention provides a video editing method. The video editing method comprises the following steps: acquiring background music according to a video editing demand; on the basis of drumbeat recognition and emotion analysis on the background music by an expert template, selecting a key drumbeat for video picture switching, and generating an editing configuration file on the basis of the key drumbeat; segmenting and classifying the to-be-processed video through multi-modal analysis to obtain different types of video clips; and automatically selecting a video clip combination based on the editing configuration file, and adding a transition special effect to obtain an edited video. According to the embodiment of the invention, by introducing the intelligent editing scheme based on the expert template, the editing result can be ensured to be highly matched with the music rhythm, the scene segment selected in the editing process meets the preset emotion and theme requirements, and the logicality and ornamental value of the video are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio and video processing technology, and in particular, to a video editing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the popularity of short video platforms, user-generated content has exploded, and automated video editing technology has emerged, which can automatically generate video content according to specific rules or algorithms, reducing the time and cost of manual editing. However, existing intelligent editing solutions have obvious deficiencies in the quality and control of generated videos, especially when facing diversified user needs, and often fail to match user expectations. Therefore, it is of great significance to develop a new video editing solution that can both ensure video quality and improve the level of intelligence.

[0003] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0004] The embodiments of the present application provide a video editing method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0005] One aspect of an embodiment of the present application provides a video editing method, the method comprising: Get background music according to video editing requirements; Based on the expert template, the drum beat recognition and emotion analysis of the background music are performed to select the key drum beats for switching the video images, and a clip configuration file is generated based on this; The video to be processed is segmented and classified through multimodal analysis to obtain video clips of different categories; A video clip combination is automatically selected based on the editing configuration file, and a transition effect is added to obtain an edited video.

[0006] Optionally, the drum beat recognition and emotion analysis of the background music based on the expert template and the selection of key drum beats for switching video images include: Analyze the background music using a drum beat recognition algorithm to identify all drum beats; The expert template analyzes the rhythm and emotion of the background music according to the drum beat recognition result to configure a set of key drum beats for screen switching.

[0007] Optionally, generating a clip configuration file includes: Generate video clip selection rules and transition special effect settings between the key drum beats.

[0008] Optionally, classifying the video to be processed by multimodal analysis includes: Automatically identifying the person appearing in the video to be processed by a person retrieval algorithm, and determining the identity information of the person and the position and size of the person in the picture by person detection and matching; Based on the position and size, different shot types are distinguished according to a set threshold, including close-up shots and long shots.

[0009] Optionally, classifying the video to be processed by multimodal analysis further includes: The emotional state of the character is analyzed through an expression recognition algorithm.

[0010] Optionally, classifying the video to be processed by multimodal analysis further includes: Combining the multimodal model and the scene segmentation algorithm, the video to be processed is segmented into different scenes, including close-up scenes of specific characters, long shots of specific characters, close shots of combined characters, long shots of combined characters, specific emotional scenes of specific characters, scenes with specific features, and empty shots.

[0011] Optionally, classifying the video to be processed by multimodal analysis further includes: The multiple frames of the video segments after scene segmentation in the same scene are voted through an inter-frame voting mechanism to further determine the scene category corresponding to the video segment.

[0012] Another aspect of an embodiment of the present application provides a video editing device, the device comprising: The acquisition module is used to obtain background music according to the video editing requirements; An analysis module, for performing drum beat recognition and emotion analysis on the background music based on an expert template, selecting key drum beats for switching video images, and generating a clip configuration file based on this; The classification module is used to segment and classify the video to be processed through multimodal analysis to obtain video clips of different categories; The editing module is used to automatically select a combination of video clips based on the editing configuration file and add transition effects to obtain an edited video.

[0013] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0014] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0015] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0016] The above technical solution adopted in the embodiment of the present application may have the following advantages: By introducing an intelligent editing solution based on expert templates, the shortcomings of existing technologies in video quality and controllability are overcome. Based on the drum beat recognition and emotion analysis of background music based on expert templates, the solution selects the key drum beats for video screen switching, and on this basis, formulates a configuration file for video editing, which can ensure that the editing results are highly consistent with the rhythm of the music. The solution combines multimodal analysis such as character retrieval algorithm, expression recognition algorithm, and scene segmentation algorithm to classify video content according to character identity, emotion, scene type, etc., and automatically generates a combination of video clips that conform to emotional expression based on the preset editing configuration file, ensuring that the scene clips selected during the editing process meet the preset emotional and theme requirements, improving the logic and viewing of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 A diagram schematically shows an operating environment of a video editing method according to Embodiment 1 of the present application; Figure 2 The flowchart of the video editing method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 A detailed flow chart of step S202; Figure 4 Schematically shows Figure 2 A detailed flow chart of step S204; Figure 5 Schematically shows Figure 2 Another detailed flow chart of step S204; Figure 6 Schematically shows Figure 2Another detailed flow chart of step S204; Figure 7 Schematically shows Figure 2 Another detailed flow chart of step S204; Figure 8 A block diagram schematically shows a video editing device according to the second embodiment of the present application; and Fig. 9 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0022] First, the following terms are explained: Expert Templates: Pre-set video editing rules and profiles based on the knowledge and experience of domain experts to guide the video editing process.

[0023] Editing profile: A collection of video editing rules developed by experts based on music and video content to guide the selection and arrangement of video clips.

[0024] Multimodal editing technology: a technology that combines multiple data modalities (such as audio, visual, text, etc.) for video editing to improve the accuracy and expressiveness of editing.

[0025] Drum beat recognition algorithm: An algorithm used to detect rhythm points or strong drum beats in music, such as the Beat-Transformer algorithm, to help determine the key points of video switching.

[0026] Character retrieval algorithm: used to identify specific characters appearing in the video, and determine the character's identity and its position and size in the picture through, for example, the Insightface algorithm.

[0027] Expression recognition algorithm: an algorithm used to analyze the facial expressions of people in video images to identify their emotional state.

[0028] Scene segmentation algorithm: An algorithm used to segment a video into different scenes, helping to identify different situations or picture categories in the video.

[0029] InsightFace algorithm: is a face recognition algorithm based on deep learning. The algorithm has high accuracy and real-time performance, and is widely used in face recognition, face detection, face attribute analysis and other fields.

[0030] CLIP (Contrastive Language–Image Pre-training) model: is a multimodal pre-training neural network, an effective and scalable method for learning from natural language supervision. The core idea of ​​this model is to use a large amount of paired data of images and texts for pre-training to learn the alignment relationship between images and texts. This model has the ability of multimodal learning, which can understand information of two different modalities, image and text, at the same time and establish connections between them.

[0031] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: The emerging multimodal intelligent editing technology uses multiple input modes for comprehensive analysis, such as audio, video, text, etc. Although it provides a richer information basis for intelligent editing, the technology is still not fully mature, and there are problems such as low recognition accuracy and inadequate video emotion transmission.

[0032] In this context, music-driven video editing has gradually attracted attention. By combining the mood and rhythm of music with the rhythm and picture content of video editing, the viewing and appeal of the video can be significantly improved. However, existing intelligent editing technologies usually rely on simple rules or basic machine learning algorithms, resulting in uncontrollable generated video effects, especially in complex music and diverse video scenes, which are prone to problems such as weak sense of rhythm and inaccurate emotional communication. In addition, although multimodal technology has broad prospects, due to the lack of maturity of the algorithm, the existing solutions still have a lot of room for improvement in recognition accuracy and scene matching.

[0033] To this end, an embodiment of the present application provides a video editing technical solution. In this technical solution, the shortcomings of the prior art in terms of video quality and controllability can be overcome by introducing expert templates. First, the rhythm points in the music are accurately identified through the drum tracking algorithm, achieving perfect synchronization between the video screen and the music rhythm. Secondly, by combining the multimodal model with expert knowledge, specific editing profiles are set for different music types and emotions to ensure that the editing effect is more artistic and emotionally expressive. In addition, through technical means such as scene segmentation and inter-frame voting, the accuracy of scene recognition is greatly improved, so that the generated video clips are more in line with human aesthetic needs in terms of logic and coherence. Finally, the present application can automatically generate high-quality videos that meet user expectations, and has high operability and flexibility. See below for details.

[0034] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0035] See also Figure 1 In this operating environment diagram, the present application can be applied to application environments including, but not limited to, a service platform 2, a client 4, and a network 6.

[0036] The client 4 is used to display the interface of the current application to the user and receive the user's operation. The client 4 can be a terminal device such as a PC (Personal Computer), a mobile phone, a tablet computer, a portable computer, a wearable device, etc.

[0037] The service platform 2 is used to provide data and technical support for the client 4. The service platform 2 can be a computing device such as a rack server, a blade server, a tower server or a cabinet server, and can be an independent server or a server cluster composed of multiple servers.

[0038] The network 6 can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), the Wideband Code Division Multiple Access (WCDMA), the 4G network, the 5G network, Bluetooth, Wi-Fi, etc. The service platform 2 and one or more of the clients 4 are connected to each other through the network 6 for data transmission and interaction.

[0039] Embodiment 1 Figure 2 The flowchart of the video editing method according to the first embodiment of the present application is schematically shown. It can be understood that the flowchart in the embodiment of the present method is not used to limit the order of executing the steps. As needed, some steps in the flowchart can also be added or deleted. The execution subject of the video editing method can be the service platform or the client.

[0040] like Figure 2 As shown, the video editing method may include steps S200 to S206, wherein: Step S200, obtaining background music according to video editing requirements.

[0041] First, through audio analysis, various music styles can be classified and style labels can be set for each piece of music, such as ancient style, high-energy, lyrical, rock, etc. The style labels can help quickly match suitable background music during video editing.

[0042] In this embodiment, according to the style label of each piece of music and the requirements of this video editing, matching music can be obtained as the background music of the video editing.

[0043] Step S202, based on the expert template, the drum beats of the background music are recognized and the emotion is analyzed, key drum beats for switching the video images are selected, and a video editing configuration file is generated on this basis.

[0044] Next, the background music is analyzed using a drum beat recognition algorithm (e.g., the Beat-Transformer algorithm) to identify all drum beats (rhythm points), especially strong drum beats. Based on the recognition results, the expert template can select key drum beats suitable for screen switching based on the rhythm and emotion analysis of the music. Subsequently, the switching of video clips is precisely synchronized with the key drum beats, so that the generated video has a high degree of consistency in rhythm and emotional expression.

[0045] It is worth noting that each piece of music can have multiple editing profiles, each of which corresponds to a different selection of key drum beat groups, so that it can be flexibly applied in different editing scenarios.

[0046] Then, experts develop editing profiles based on the characteristics of the background music and video. The profiles contain rules for selecting video clips between key drum beats, transition effects settings, and other editing parameters to ensure that the editing results are highly consistent with the music rhythm.

[0047] Step S204, segmenting and classifying the video to be processed through multimodal analysis to obtain video clips of different categories.

[0048] In this embodiment, the multimodal analysis combines multiple algorithms such as character retrieval algorithm, expression recognition algorithm and scene segmentation algorithm to classify the video content according to character identity, emotion, scene type, etc., so as to automatically generate a video clip combination that conforms to the emotional expression based on the editing configuration file in the future.

[0049] Specifically, the character retrieval algorithm (such as the Insightface algorithm) is used to automatically identify the characters in the video, and the character's identity information and their position and size in the picture are determined through character detection and matching. Then, different types of shots, such as close-ups and long shots, are distinguished based on the set threshold. In addition, the expression recognition algorithm can be used to analyze the emotional state of the character, so as to select video clips that match the mood of the music.

[0050] For the recognition of specific scenes and empty shots (such as landscapes, buildings, etc.), we can combine multimodal models (such as CLIP models) and scene segmentation algorithms to segment videos into different scenes, and improve the accuracy of scene classification through the inter-frame voting mechanism. These processed video clips will be used as materials for video editing.

[0051] Based on these algorithms, the types of video segments that can be selected in this patent may include but are not limited to: (1) close-up scenes of specific characters; (2) long shots of specific characters; (3) close shots of combined characters; (4) long shots of combined characters; (5) specific emotional scenes of specific characters, such as sadness, happiness, anger, etc.; (6) scenes with specific characteristics, such as high-energy fighting scenes, intimate shots, etc.; (7) empty shots, such as landscapes, buildings, etc.

[0052] Step S206, automatically selecting a video clip combination based on the editing configuration file, and adding transition effects to obtain an edited video.

[0053] In this embodiment, by reading the editing configuration file, suitable video clips are automatically selected and arranged and combined according to preset rules to form an edited video. Specifically, the video clip selection rule between every two adjacent key drum beats is obtained from the editing configuration file, and according to the rule and the video segmentation and classification results, suitable video clips are automatically selected to fill in between the two key drum beats.

[0054] In the video, transition effects such as fade in and fade out, white flash, etc. can be automatically added according to the transition effect settings of the key drum beats preset in the editing profile, so that the video switching is more natural and smooth. The final output editing video will have a high sense of rhythm and emotional expression, which can better attract the audience's attention.

[0055] Through the above steps, the present application realizes intelligent editing based on expert templates, so that the generated video not only retains the efficiency of intelligence, but also can fully integrate human ingenuity and artistry.

[0056] Further reading Figure 3 , is a detailed flow chart of the above step S202. In this embodiment, the step S202 specifically includes: Step S2020: Use a drum beat recognition algorithm to analyze the background music and identify all drum beats.

[0057] The drum beat recognition algorithm, such as the Beat-Transformer algorithm, is used to detect drum beats (rhythm points) or strong drum beats in the background music to help determine key points of video switching.

[0058] Step S2022: the expert template analyzes the rhythm and emotion of the background music according to the drum beat recognition result to configure a set of key drum beats for screen switching.

[0059] After all the drum beats in the background music are identified, it is not necessary to use all the drum beats to switch the screen, which will result in too high a switching frequency. In this embodiment, the expert template analyzes the rhythm and emotion of the background music according to the drum beat recognition result, and further determines a group of key drum beats from all the drum beats for subsequent video screen switching. The key drum beats can be strong drum beats or drum beats at positions with large emotional changes.

[0060] Step S2024: Generate a clipping configuration file according to the key drum beats.

[0061] The configuration file includes video clip selection rules between key drum beats, transition effect settings, and other editing parameters.

[0062] Further reading Figure 4, is a detailed flow chart of the above step S204. In this embodiment, the step S204 specifically includes: Step S2040, automatically identifying the person appearing in the video to be processed through a person retrieval algorithm, and determining the identity information of the person and the position and size of the person in the picture through person detection and matching.

[0063] Step S2042: based on the position and size, different types of shots are distinguished according to a set threshold, including close-up shots and long-shot shots.

[0064] For example, if the size ratio of the character in the picture exceeds the threshold, it means that the picture is a close-up shot. Otherwise, if the size ratio of the character in the picture is less than the threshold, it means that the picture is a long shot.

[0065] Further reading Figure 5 , is another detailed flow chart of the above step S204. In this embodiment, the step S204 also includes: Step S2044, analyzing the emotional state of the character through an expression recognition algorithm.

[0066] Based on the characters in each video frame that have been identified, the expression recognition algorithm can further analyze the facial expressions of the characters in each video frame to identify their emotional states, so that a video clip that matches the emotion of the background music can be selected later.

[0067] Further reading Figure 6 , is another detailed flow chart of the above step S204. In this embodiment, the step S204 also includes: Step S2046, combining the multimodal model and the scene segmentation algorithm to segment the video to be processed into different scenes.

[0068] Specifically, the video clips obtained after segmentation include close-up scenes of specific characters, long shots of specific characters, close shots of combined characters, long shots of combined characters, specific emotional scenes of specific characters, scenes with specific features, empty shots, etc.

[0069] Further reading Figure 7 , is another detailed flow chart of the above step S204. In this embodiment, the step S204 also includes: Step S2048: voting on multiple frames of the video segments in the same scene after scene segmentation through an inter-frame voting mechanism to further determine the scene category corresponding to the video segment.

[0070] In this embodiment, the inter-frame voting mechanism is used to optimize the scene recognition results. The mechanism determines the category of the scene by voting on multiple frames of the same scene in the video clips after scene segmentation, thereby improving the accuracy of scene recognition. This method ensures that the scene clips selected during the editing process meet the preset emotional and theme requirements, and improves the logic and viewing quality of the video.

[0071] The video editing method provided in this embodiment can select key drum beats for switching video images based on drum beat recognition and emotion analysis of expert template scene music, and formulate a configuration file for video editing on this basis; through multimodal analysis, combined with a character retrieval algorithm, an expression recognition algorithm, and a scene segmentation algorithm, the video content is classified according to character identity, emotion, scene type, etc., and a video clip combination that conforms to emotional expression is automatically generated based on a preset editing configuration file; a drum beat tracking algorithm is used to accurately identify the rhythm points in the music, and the switching of the video clips is precisely synchronized with the drum beats, so that the generated video has a high degree of consistency in rhythm and emotional expression; a multiple frames of the video clips after scene segmentation are voted on in the same scene through an inter-frame voting mechanism to determine the category of the scene, thereby improving the accuracy of scene recognition.

[0072] Embodiment 2 Figure 8 The block diagram of the video editing device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 8 As shown, the device 60 may include: an acquisition module 600, an analysis module 602, a classification module 604, and a clipping module 606, wherein: The acquisition module 600 is used to acquire background music according to the video editing requirements.

[0073] The analysis module 602 is used to perform drum beat recognition and emotion analysis on the background music based on the expert template, select key drum beats for switching video images, and generate a video editing configuration file based on this.

[0074] The classification module 604 is used to segment and classify the video to be processed through multimodal analysis to obtain video clips of different categories.

[0075] The editing module 606 is used to automatically select a video clip combination based on the editing configuration file and add transition effects to obtain an edited video.

[0076] In an optional embodiment, the drum beat recognition and emotion analysis of the background music based on the expert template and the selection of key drum beats for switching video images include: Analyze the background music using a drum beat recognition algorithm to identify all drum beats; The expert template analyzes the rhythm and emotion of the background music according to the drum beat recognition result to configure a set of key drum beats for screen switching.

[0077] In an optional embodiment, generating a clip configuration file includes: Generate video clip selection rules and transition special effect settings between the key drum beats.

[0078] In an optional embodiment, classifying the video to be processed by multimodal analysis includes: Automatically identifying the person appearing in the video to be processed by a person retrieval algorithm, and determining the identity information of the person and the position and size of the person in the picture by person detection and matching; Based on the position and size, different shot types are distinguished according to a set threshold, including close-up shots and long shots.

[0079] In an optional embodiment, classifying the video to be processed by multimodal analysis further includes: The emotional state of the character is analyzed through an expression recognition algorithm.

[0080] In an optional embodiment, classifying the video to be processed by multimodal analysis further includes: Combining the multimodal model and the scene segmentation algorithm, the video to be processed is segmented into different scenes, including close-up scenes of specific characters, long shots of specific characters, close shots of combined characters, long shots of combined characters, specific emotional scenes of specific characters, scenes with specific features, and empty shots.

[0081] In an optional embodiment, classifying the video to be processed by multimodal analysis further includes: The multiple frames of the video segments after scene segmentation in the same scene are voted through an inter-frame voting mechanism to further determine the scene category corresponding to the video segment.

[0082] The specific implementation process of the above-mentioned functions of each module can be found in the description of the above-mentioned embodiment 1, and will not be repeated here.

[0083] Embodiment 3 Fig. 9The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing a video editing method according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig. 9 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed in the computer device 10000, such as program codes of a video editing method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0084] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0085] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0086] It should be pointed out that Fig. 9 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0087] In this embodiment, the video editing method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0088] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the video editing method in the embodiment are implemented.

[0089] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the video editing method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0090] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0091] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0092] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video editing method, characterized in that: The method comprises: Get background music according to video editing requirements; Based on the expert template, the drum beat recognition and emotion analysis of the background music are performed to select the key drum beats for switching the video images, and a clip configuration file is generated based on this; The video to be processed is segmented and classified through multimodal analysis to obtain video clips of different categories; A video clip combination is automatically selected based on the editing configuration file, and a transition effect is added to obtain an edited video.

2. The method according to claim 1, characterized in that The drum beat recognition and emotion analysis of the background music based on the expert template and the selection of key drum beats for switching video images include: Analyze the background music using a drum beat recognition algorithm to identify all drum beats; The expert template analyzes the rhythm and emotion of the background music according to the drum beat recognition result to configure a set of key drum beats for screen switching.

3. The method according to claim 1 or 2, characterized in that: Generating a clip configuration file comprises: Generate video clip selection rules and transition special effect settings between the key drum beats.

4. The method according to claim 1, characterized in that: The method of classifying the video to be processed by multimodal analysis includes: Automatically identifying the person appearing in the video to be processed by a person retrieval algorithm, and determining the identity information of the person and the position and size of the person in the picture by person detection and matching; Based on the position and size, different shot types are distinguished according to a set threshold, including close-up shots and long shots.

5. The method according to claim 4, characterized in that The method of classifying the video to be processed by multimodal analysis further includes: The emotional state of the character is analyzed through an expression recognition algorithm.

6. The method according to claim 1, 4 or 5, characterized in that: The classifying the video to be processed by multimodal analysis also includes: Combining the multimodal model and the scene segmentation algorithm, the video to be processed is segmented into different scenes, including close-up scenes of specific characters, long shots of specific characters, close shots of combined characters, long shots of combined characters, specific emotional scenes of specific characters, scenes with specific features, and empty shots.

7. The method according to claim 6, characterized in that The classifying the video to be processed by multimodal analysis also includes: The multiple frames of the video segments after scene segmentation in the same scene are voted through an inter-frame voting mechanism to further determine the scene category corresponding to the video segment.

8. A video editing device, characterized in that: The device comprises: The acquisition module is used to obtain background music according to the video editing requirements; An analysis module, for performing drum beat recognition and emotion analysis on the background music based on an expert template, selecting key drum beats for switching video images, and generating a clip configuration file based on this; The classification module is used to segment and classify the video to be processed through multimodal analysis to obtain video clips of different categories; The editing module is used to automatically select a combination of video clips based on the editing configuration file and add transition effects to obtain an edited video.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images