Method for synthesizing video MV by music based on self-supervised learning

Abstract

The invention discloses a method for synthesizing a video MV by music based on self-supervised learning, and the method comprises the following steps: 1, separating an audio stream and a video streamfrom an existing material library; 2, extracting people, actions, expressions and scene information from the video by using a deep learning technology based on video understanding; 3, automatically classifying the voiceprint information according to the rhythm of the music; 4, separating voices, musical instruments, accompaniments and lyrics from the music; 5, synchronizing audio and video relatedfeature information by a timestamp in the video file; 6, learning corresponding video information according to the music features to form a mapping relationship between music and videos; 7, inputtingany piece of music, and synthesizing a corresponding video MV; according to the invention, a proper video clip can be automatically matched and selected from massive existing video data, music is mapped to generate a corresponding short video MV, and more intuitive visual impact and more vivid auditory experience are provided for a user.

Description

technical field [0001] The invention relates to the technical field of media asset management, in particular to a method for synthesizing video MV from music based on self-supervised learning. Background technique [0002] Driven by the technology of mobile Internet, big data, and AI intelligence, short videos are breaking the traditional thinking of the content industry with their own advantages. The short video platform can realize precise matching and intelligent diversion based on the interests and preferences of users, and intelligently radiate multiple distribution channels through short video content to accurately reach multi-level users, enabling users to understand video topics at a low cost and resonate with them, and gain more Many endorsements and retweets. With the development of 5G technology, the operating cost of the platform is reduced, the speed of the mobile terminal is greatly improved, the traffic of the short video blowout and the outstanding marketing...

AI Technical Summary

Problems solved by technology

[0003] Both music and short video MV have entertainment properties, but due to their voice characteristics, it is impossible to provide users with intuitive and full picture information, and the traditional method of manual video conversion requires manual collection, browsing, and labeling of a large amount of video data, and then selects Some fragments that match the rhythm of the music and other information consume a lot of time and energy