Video dubbling method and device, computer equipment and storage medium

Abstract

The invention provides a video dubbing method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a video to be dubbed with music, and obtaining the video beat data, video emotion data and video content data of the video to be dubbed with music according to the video to be dubbed with music; obtaining a plurality of pieces of music to be matched, obtaining music beat data, music style data and music content data of each piece of music to be matched according to the plurality of pieces of music to be matched, and comparing the music beat data, the music style data and the music content data with video beat data, video emotion data and video content data corresponding to the video to be matched with music; and according to a comparison result, determining candidate matching music from the plurality of pieces of to-be-matched music. According to the invention, music matching can be rapidly carried out on the video, and the video music matching efficiency is effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of video editing, and specifically to a method, apparatus, computer equipment and storage medium for composing music for a video. Background Art

[0002] With the continuous development of video editing technology, the consumption demand for short videos is also growing. More and more users are constantly enriching the form and content of short videos through video editing technology. Among the many video editing technologies, the automatic matching technology around video background music is also sought after by the majority of video enthusiasts.

[0003] Traditionally, the matching of video background music still relies mainly on manual selection, where the target music is matched according to the video material and one's own feelings. This often requires users to spend more time and energy to find suitable target music, resulting in low efficiency in video editing. Summary of the invention

[0004] The embodiments of the present application provide a video music matching method, apparatus, computer equipment and storage medium. The technical problem to be solved is that, in view of the defects of the prior art, the matching of the existing video background music still mainly relies on manual selection, which causes users to spend more time and energy to find suitable target music, resulting in low efficiency of video editing.

[0005] In order to solve the above technical problems, a technical solution adopted by the present application is to provide a video music matching method, device, computer equipment and storage medium, wherein the video music matching method comprises:

[0006] Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music;

[0007] Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched;

[0008] Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video;

[0009] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0010] In a specific implementation, the step of obtaining the video beat data, video emotion data, and video content data of the video to be matched with music includes:

[0011] Acquire inertial sensor data of the video to be matched with music and viewing angle data of each video frame to obtain the video beat data of the video to be matched with music, wherein the video beat data represents the video rhythm of the video to be matched with music;

[0012] Performing human body detection and scene detection on the video to be matched with music, and obtaining the video emotion data of the video to be matched with music according to the human body detection results and the scene detection results;

[0013] The content of each video frame of the video to be matched with music is analyzed by a video understanding algorithm, and the content of each video frame is trained and analyzed according to a large language model to obtain the video content data of the video to be matched with music.

[0014] In a specific implementation, the step of acquiring a plurality of music to be matched, and obtaining music beat data, music style data, and music content data of each of the music to be matched according to the plurality of music to be matched, includes:

[0015] Analyze the audio waveform of the music to be matched to obtain the music beat data of the music to be matched;

[0016] Detecting the label data of the music to be matched, or analyzing the music to be matched based on an AI algorithm to obtain the music style data of the music to be matched;

[0017] The title data, the tag data and the lyrics data of the music to be matched are analyzed to obtain the music content data of the music to be matched.

[0018] In a specific implementation, the music beat data, the music style data, and the music content data of each of the to-be-matched music are compared with the video beat data, the video emotion data, and the video content data corresponding to the to-be-matched video, respectively, including:

[0019] Performing a beat comparison on the music beat data of the music to be matched and the video beat data of the video to be matched with the music, to obtain a beat comparison score;

[0020] Matching the music style data of the music to be matched with the video emotion data of the video to be matched with music according to a preset mapping relationship to obtain an emotion comparison score;

[0021] The title data, the tag data and the lyrics data in the music content data of the music to be matched are respectively compared with the video content data of the video to be matched with music, and a content comparison score is obtained according to the comparison weights of the title data, the tag data and the lyrics data.

[0022] In a specific embodiment, the comparison weight of the title data and the label data is greater than the comparison weight of the lyrics data, the lyrics data is trained and analyzed according to a large language model, and the training and analysis results are compared with the video content data of the video to be matched with music.

[0023] In a specific implementation, determining candidate matching music from the plurality of music to be matched according to the comparison result includes:

[0024] Calculate the comprehensive matching score of each of the to-be-matched music according to the weights of the beat comparison score, the emotion comparison score and the content comparison score;

[0025] The comprehensive matching scores of the music to be matched are sorted to obtain N music to be matched with the highest comprehensive matching scores, and the N music to be matched are determined as candidate matching music.

[0026] In a specific implementation, the video music matching method further includes:

[0027] In response to the user's selection operation, target matching music is determined from the candidate matching music, and the target matching music is merged with the video to be matched with music to obtain a target video.

[0028] In a specific embodiment, a video music matching device is also provided, and the video music matching device comprises:

[0029] A module for obtaining a video to be matched with music, wherein the module is used to obtain a video to be matched with music, and obtain video beat data, video emotion data and video content data of the video to be matched with music according to the video to be matched with music;

[0030] A module for acquiring multiple music to be matched, wherein the module for acquiring multiple music to be matched is used to acquire multiple music to be matched, and obtain music beat data, music style data and music content data of each of the music to be matched according to the multiple music to be matched;

[0031] A comparison module, the comparison module is used to compare the music beat data, the music style data and the music content data of each of the music to be matched with the video beat data, the video emotion data and the video content data corresponding to the video to be matched with music;

[0032] A determination module, the determination module is used to determine candidate matching music from the plurality of music to be matched according to the comparison result;

[0033] A merging module is used to respond to a user's selection operation, determine target matching music from the candidate matching music, merge the target matching music with the video to be matched with music, and obtain a target video.

[0034] In a specific embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the video soundtrack method described in any of the above specific embodiments are implemented.

[0035] In a specific embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video music matching method described in any of the above specific embodiments are implemented.

[0036] The beneficial effect of the present application is that, different from the prior art, the present application provides a video music matching method, device, computer equipment and storage medium, wherein the video music matching method includes:

[0037] Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music;

[0038] Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched;

[0039] Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video;

[0040] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0041] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0042] The corresponding matching is performed by decoupling the video to be accompanied by music and the music to be matched into three parts: beat data, emotion data and content data. According to the matching comparison results, candidate matching music is selected from the music to be matched, thereby realizing fast and automatic music matching of the video to be accompanied by music. Without the need for users to manually select music, candidate matching music that is compatible with the beat, emotion and plot content of the video to be accompanied by music can be automatically matched according to the beat, emotion and plot content of the video to be accompanied by music, thereby improving the user's video background music matching efficiency and video editing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the implementation modes of the present application, the drawings required for use in the description of the implementation modes will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 This is a schematic diagram of the implementation process of a video music matching method provided in Example 1 of the present application;

[0045] Figure 2 This is a schematic diagram of the module composition of a video music matching device provided in the second embodiment of the present application;

[0046] Figure 3 It is a schematic diagram of the internal structure of a computer device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0047] The present application is further described in detail below in conjunction with the accompanying drawings and implementation methods. It is particularly noted that the following implementation methods are only used to illustrate the present application, but do not limit the scope of the present application. Similarly, the following implementation methods are only some implementation methods of the present application rather than all implementation methods. All other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0048] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to these processes, methods, products or devices.

[0049] The above terms are only for the convenience of description and should not be construed as limitations on the present technical solution.

[0050] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] The specific implementation of the present application is described in detail below in conjunction with specific embodiments.

[0052] Embodiment 1.

[0053] Figure 1 The implementation process of a video music matching method provided in Example 1 of the present application is shown. For the convenience of explanation, only the part related to the embodiment of the present application is shown, which is described in detail as follows.

[0054] A video to be matched with music is obtained, and according to the video to be matched with music, video beat data, video emotion data and video content data of the video to be matched with music are obtained.

[0055] It should be noted that the video to be matched with music is analyzed from three aspects: beat, emotion and content. The beat of the video reflects the overall rhythm of the video, the emotion of the video reflects the emotion of the target character in the video and the color tone of the scene, and the content of the video reflects the plot of the video.

[0056] One specific implementation method is that obtaining the video beat data, video emotion data and video content data of the video to be matched with music includes:

[0057] Acquire inertial sensor data of the video to be matched with music and viewing angle data of each video frame to obtain the video beat data of the video to be matched with music, wherein the video beat data represents the video rhythm of the video to be matched with music;

[0058] Performing human body detection and scene detection on the video to be matched with music, and obtaining the video emotion data of the video to be matched with music according to the human body detection results and the scene detection results;

[0059] The content of each video frame of the video to be matched with music is analyzed by a video understanding algorithm, and the content of each video frame is trained and analyzed according to a large language model to obtain the video content data of the video to be matched with music.

[0060] It should be noted that the video beat data of the video to be matched with music is obtained by performing a comprehensive calculation on the inertial sensor data of the video to be matched with music and the perspective data of each video frame when the user is editing. One feasible specific implementation method is to obtain the three-axis translational degrees of freedom in the inertial sensor data and the perspective data to obtain a data sequence of the rotational degrees of freedom of the video to be matched with music, normalize the data sequence, and then perform a weighted calculation to obtain the average beat score of the video to be matched with music. The video beat data characterizes the video rhythm of the video to be matched with music. For example, when a user shoots a video while riding, the translation speed in the inertial sensor data is fast, the picture changes quickly, and the video rhythm is strong.

[0061] By performing human body detection and scene detection on the video to be matched with music, the facial expressions of the human body and the light and dark colors of the scene are identified, and the video emotion data of the video to be matched with music is obtained according to the facial expressions of the human body and the light and dark colors of the scene. The video emotion data can be represented by a video emotion curve, and the video emotions include depression, stability, excitement, etc.

[0062] The content of each video frame of the video to be matched with music, including the recognized text information, is analyzed by a conventional video understanding algorithm, and the content of each video frame is trained and analyzed according to a large language model to obtain the video content data of the video to be matched with music.

[0063] Acquire multiple music to be matched, and obtain music beat data, music style data and music content data of each music to be matched based on the multiple music to be matched.

[0064] It should be noted that the multiple music to be matched are also analyzed from three aspects: beat, style and content, and the obtained analysis results are matched with the analysis results of the video to be matched with music, the music beat data is matched with the video beat data, the music style data is matched with the video mood data, and the music content data is matched with the video content data.

[0065] One specific implementation method is that the method of obtaining a plurality of music to be matched, and obtaining music beat data, music style data, and music content data of each of the music to be matched according to the plurality of music to be matched, includes:

[0066] Analyze the audio waveform of the music to be matched to obtain the music beat data of the music to be matched;

[0067] Detecting the label data of the music to be matched, or analyzing the music to be matched based on an AI algorithm to obtain the music style data of the music to be matched;

[0068] The title data, the tag data and the lyrics data of the music to be matched are analyzed to obtain the music content data of the music to be matched.

[0069] It should be noted that, by analyzing the audio waveform of the music to be matched, the rhythm points of the music to be matched are determined according to the peak values ​​in the audio waveform, so as to obtain the music beat data of the music to be matched.

[0070] The music style data of the music to be matched is obtained by analyzing the music tag of the music to be matched, or analyzing the music to be matched based on an AI algorithm. The music style data corresponds to the video emotion data of the video to be matched with music. The music style data characterizes the classification attributes of the music to be matched, including sadness, pop, rock, happiness, sports, etc.

[0071] By analyzing the title data, the tag data and the lyrics data of the music to be matched, the music content data of the music to be matched is comprehensively obtained, and the music content data further characterizes the classification attributes of the music to be matched.

[0072] The music beat data, the music style data and the music content data of each of the music to be matched are respectively compared with the video beat data, the video emotion data and the video content data corresponding to the video to be matched with music.

[0073] It should be noted that the music beat data of the music to be matched is compared with the video beat data of the video to be matched with music, the music style data of the music to be matched is compared with the video emotion data of the video to be matched with music, and the music content data of the music to be matched is compared with the video content data of the video to be matched with music.

[0074] One specific implementation method is to compare the music beat data, the music style data and the music content data of each of the music to be matched with the video beat data, the video emotion data and the video content data corresponding to the video to be matched with music, respectively, including:

[0075] Performing a beat comparison on the music beat data of the music to be matched and the video beat data of the video to be matched with the music, to obtain a beat comparison score;

[0076] Matching the music style data of the music to be matched with the video emotion data of the video to be matched with music according to a preset mapping relationship to obtain an emotion comparison score;

[0077] The title data, the tag data and the lyrics data in the music content data of the music to be matched are respectively compared with the video content data of the video to be matched with music, and a content comparison score is obtained according to the comparison weights of the title data, the tag data and the lyrics data.

[0078] It should be noted that the beat comparison score represents the degree of overlap between the rhythm points of the music beat data and the video beat data in the time series; the emotion comparison score represents the degree of correlation between the music style data and the video emotion data, and the preset mapping relationship can be: the video emotion corresponding to the video emotion data is depressed, and the classification attribute of the music style data is mapped to sadness; the video emotion corresponding to the video emotion data is stable, and the classification attribute of the music style data is mapped to pop, pure music, classical, etc.; the video emotion corresponding to the video emotion data is excited, and the classification attribute of the music style data is mapped to rock, happiness, sports, etc.; the content comparison score represents the degree of overlap between the music style data and the video content data, such as the degree of overlap between the text content of the music content data and the text content of the video content data, and the content comparison score is obtained according to the comparison weights of the title data, the tag data and the lyrics data.

[0079] One specific implementation method is that the comparison weight of the title data and the label data is greater than the comparison weight of the lyrics data, the lyrics data is trained and analyzed based on a large language model, and the training and analysis results are compared with the video content data of the video to be matched with music.

[0080] It should be noted that, for music content, the title and label can better reflect the content of a piece of music. Therefore, when calculating the content comparison score, the comparison weight of the title data and the label data is greater than the comparison weight of the lyrics data. For the lyrics data, the lyrics data can be first trained and analyzed according to a large language model, and compared with the video content data of the video to be matched with music according to the training and analysis results.

[0081] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0082] It should be noted that, based on the comparison results, N candidate matching music are determined from the multiple music to be matched, and the N candidate matching music are provided for user selection, and one user is selected from the N candidate matching music as the target matching music for the video to be matched.

[0083] One specific implementation method is that, according to the comparison result, determining candidate matching music from the plurality of music to be matched includes:

[0084] Calculate the comprehensive matching score of each of the to-be-matched music according to the weights of the beat comparison score, the emotion comparison score and the content comparison score;

[0085] The comprehensive matching scores of the music to be matched are sorted to obtain N music to be matched with the highest comprehensive matching scores, and the N music to be matched are determined as candidate matching music.

[0086] It should be noted that by calculating the comprehensive matching scores of each piece of music to be matched by weighted sum, one feasible specific implementation is that the weight of the content comparison score is greater than the weight of the emotion comparison score, which is greater than the weight of the beat comparison score.

[0087] In one specific implementation, the video music matching method further includes:

[0088] In response to the user's selection operation, target matching music is determined from the candidate matching music, and the target matching music is merged with the video to be matched with music to obtain a target video.

[0089] Embodiment 2.

[0090] Figure 2 A schematic diagram of the module composition of a video music matching device provided in the second embodiment of the present application is shown. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0091] A video music matching device according to an embodiment of the present application includes:

[0092] A module for obtaining a video to be matched with music, wherein the module is used to obtain a video to be matched with music, and obtain video beat data, video emotion data and video content data of the video to be matched with music according to the video to be matched with music;

[0093] A module for acquiring multiple music to be matched, wherein the module for acquiring multiple music to be matched is used to acquire multiple music to be matched, and obtain music beat data, music style data and music content data of each of the music to be matched according to the multiple music to be matched;

[0094] A comparison module, the comparison module is used to compare the music beat data, the music style data and the music content data of each of the music to be matched with the video beat data, the video emotion data and the video content data corresponding to the video to be matched with music;

[0095] A determination module, the determination module is used to determine candidate matching music from the plurality of music to be matched according to the comparison result;

[0096] A merging module is used to respond to a user's selection operation, determine target matching music from the candidate matching music, merge the target matching music with the video to be matched with music, and obtain a target video.

[0097] A video music matching device of an embodiment of the present application is used to implement a video music matching method provided in any specific implementation manner of the first embodiment of the present application:

[0098] Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music;

[0099] Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched;

[0100] Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video;

[0101] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0102] It should be noted that each module in the above-mentioned video music matching device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0103] It should be noted that, for technical details not fully described in this embodiment, reference may be made to the video music matching method provided in any specific implementation manner in the above-mentioned embodiment 1, and will not be repeated here.

[0104] Embodiment three.

[0105] In an embodiment of the present application, a computer device is provided. Figure 3 A schematic diagram of the internal structure of a computer device.

[0106] The computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the video music matching method provided by any specific implementation method in the first embodiment of the present application, specifically including:

[0107] Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music;

[0108] Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched;

[0109] Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video;

[0110] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0111] It should be noted that the computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a video generation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0112] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] It should be noted that, for technical details that are not described in detail in this embodiment, reference can be made to the video music matching method provided in any specific implementation manner in the above-mentioned embodiment 1, and will not be repeated here.

[0114] Embodiment 4.

[0115] In an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the video soundtrack method provided by any specific implementation method in the first embodiment of the present application is implemented, specifically including:

[0116] Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music;

[0117] Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched;

[0118] Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video;

[0119] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0120] It should be noted that, for technical details that are not described in detail in this embodiment, reference can be made to the video music matching method provided in any specific implementation manner in the above-mentioned embodiment 1, and will not be repeated here.

[0121] Different from the prior art, the embodiments of the present application provide a video music matching method, device, computer equipment and storage medium, wherein the video music matching method includes:

[0122] Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music;

[0123] Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched;

[0124] Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video;

[0125] According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

[0126] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0127] The corresponding matching is performed by decoupling the video to be accompanied by music and the music to be matched into three parts: beat data, emotion data and content data. According to the matching comparison results, candidate matching music is selected from the music to be matched, thereby realizing fast and automatic music matching of the video to be accompanied by music. Without the need for users to manually select music, candidate matching music that is compatible with the beat, emotion and plot content of the video to be accompanied by music can be automatically matched according to the beat, emotion and plot content of the video to be accompanied by music, thereby improving the user's video background music matching efficiency and video editing efficiency.

[0128] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0130] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a terminal device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0131] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video music matching method, characterized in that: The video music matching method comprises: Acquire a video to be matched with music, and obtain video beat data, video emotion data, and video content data of the video to be matched with music according to the video to be matched with music; Acquire multiple pieces of music to be matched, and obtain music beat data, music style data, and music content data of each piece of music to be matched according to the multiple pieces of music to be matched; Compare the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video; According to the comparison result, candidate matching music is determined from the multiple pieces of music to be matched.

2. A video music matching method according to claim 1, characterized in that: The step of obtaining the video beat data, video emotion data and video content data of the video to be matched with music comprises: Acquire inertial sensor data of the video to be matched with music and viewing angle data of each video frame to obtain the video beat data of the video to be matched with music, wherein the video beat data represents the video rhythm of the video to be matched with music; Performing human body detection and scene detection on the video to be matched with music, and obtaining the video emotion data of the video to be matched with music according to the human body detection results and the scene detection results; The content of each video frame of the video to be matched with music is analyzed by a video understanding algorithm, and the content of each video frame is trained and analyzed according to a large language model to obtain the video content data of the video to be matched with music.

3. A video music matching method according to claim 2, characterized in that: The step of acquiring a plurality of music to be matched, and obtaining music beat data, music style data, and music content data of each of the music to be matched according to the plurality of music to be matched, comprises: Analyze the audio waveform of the music to be matched to obtain the music beat data of the music to be matched; Detecting the label data of the music to be matched, or analyzing the music to be matched based on an AI algorithm to obtain the music style data of the music to be matched; The title data, the tag data and the lyrics data of the music to be matched are analyzed to obtain the music content data of the music to be matched.

4. A video music matching method according to claim 3, characterized in that: The step of comparing the music beat data, the music style data and the music content data of each of the to-be-matched music with the video beat data, the video emotion data and the video content data corresponding to the to-be-matched video, respectively, comprises: Performing a beat comparison on the music beat data of the music to be matched and the video beat data of the video to be matched with the music, to obtain a beat comparison score; Matching the music style data of the music to be matched with the video emotion data of the video to be matched with music according to a preset mapping relationship to obtain an emotion comparison score; The title data, the tag data and the lyrics data in the music content data of the music to be matched are respectively compared with the video content data of the video to be matched with music, and a content comparison score is obtained according to the comparison weights of the title data, the tag data and the lyrics data.

5. A video music matching method according to claim 4, characterized in that: The comparison weight of the title data and the label data is greater than the comparison weight of the lyrics data. The lyrics data is trained and analyzed according to a large language model, and compared with the video content data of the video to be matched with music based on the training and analysis results.

6. The video music matching method according to claim 4 or 5, characterized in that: The step of determining candidate matching music from the plurality of music to be matched according to the comparison result includes: Calculate the comprehensive matching score of each of the to-be-matched music according to the weights of the beat comparison score, the emotion comparison score and the content comparison score; The comprehensive matching scores of the music to be matched are sorted to obtain N music to be matched with the highest comprehensive matching scores, and the N music to be matched are determined as candidate matching music.

7. A video music matching method according to claim 6, wherein the video music matching method further comprises: In response to the user's selection operation, target matching music is determined from the candidate matching music, and the target matching music is merged with the video to be matched with music to obtain a target video.

8. A video music matching device, characterized in that: The video music matching device comprises: A module for obtaining a video to be matched with music, wherein the module is used to obtain the video to be matched with music, and obtain video beat data, video emotion data and video content data of the video to be matched with music according to the video to be matched with music; A module for acquiring multiple music to be matched, wherein the module for acquiring multiple music to be matched is used to acquire multiple music to be matched, and obtain music beat data, music style data and music content data of each of the music to be matched according to the multiple music to be matched; A comparison module, the comparison module is used to compare the music beat data, the music style data and the music content data of each of the music to be matched with the video beat data, the video emotion data and the video content data corresponding to the video to be matched with music; A determination module, the determination module is used to determine candidate matching music from the plurality of music to be matched according to the comparison result; A merging module is used to respond to a user's selection operation, determine target matching music from the candidate matching music, merge the target matching music with the video to be matched with music, and obtain a target video.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the video soundtrack method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video soundtrack method according to any one of claims 1 to 7 are implemented.

Images