Video translation method, system and device and storage medium

Abstract

The invention discloses a video translation method, a system and a device and a storage medium. The method comprises the following steps: acquiring video data; performing voice segmentation on the video data to obtain a voice segment and a video segment corresponding to the voice segment; performing voice recognition on the voice segment to obtain a first text, and translating the first text to obtain a second text; obtaining synthetic voice according to the second text, and enabling the synthetic voice to be matched with the video clip by adjusting the synthetic voice and the video clip corresponding to the synthetic voice; and detecting and adjusting the lip shape in the video clip so as to enable the lip shape to be synchronously matched with the synthetic voice. According to the method, the function of automatically translating the video is realized, the audio of the target language sound can be generated, the video with the sound matched with the lip shape is generated, the communication obstacle between different languages is solved, manual dubbing is not needed, the translation cost is reduced, and the method can be widely applied to the field of video processing.

Description

technical field [0001] The present invention relates to the field of video processing, in particular to a video translation method, system, device and storage medium. Background technique [0002] With the continuous development of the Internet, video has become a new carrier of information transmission. However, the objective existence of different languages ​​has caused great obstacles in the communication of information, and the current solutions are mostly to add subtitles in a specified language or use artificial dubbing. Based on the current method, there are still the following disadvantages: (1) subtitles will make the audience focus on the text information, and it is easy to ignore the overall picture; (2) dubbing requires the dubber to re-record the script according to the new language, requiring other Mastering professional skills and being good at the target language requires extremely high requirements for dubbers; (3) The voices of dubbing and video characters...

AI Technical Summary

Problems solved by technology

However, the objective existence of different languages ​​has caused great obstacles in the communication of information, and the current solutions are mostly to add subtitles in a specified language or use artificial dubbing

Based on the current method, there are still the following disadvantages: (1) subtitles will make the audience focus on the text information, and it is easy to ignore the overall picture; (2) dubbing requires the dubber to re-record the script according to the new language, requiring other Mastering professional skills and being good at the target language requires extremely high requirements for dubbers; (3) The voices of dubbing and video characters are not the same, lacking the charm of the original sound; (4) After dubbing is directly superimposed on the original video, the relative voice will be out of sync Lip movement, resulting in poor user perception

Images