Video processing method and device, equipment and storage medium

Abstract

The invention provides a video processing method and device, equipment and a storage medium, and the method comprises the steps: carrying out the voice recognition of an audio in a target video draft, obtaining an initial text segment of the target video draft, and carrying out the recognition of the initial text segment of the target video draft according to an initial language type and a target language type of the target video draft; and performing language conversion on the initial text fragment of the target video draft to obtain a target text fragment, generating a target audio fragment based on the target text fragment, and generating an editing result video draft corresponding to the target video draft based on the target audio fragment. According to the embodiment of the invention, the initial text fragment of the target video draft is converted into the target text fragment, and the target audio fragment belonging to the target language type is generated based on the target text fragment, so that the editing result video draft is generated; the target video draft of one language type is converted into the editing result video draft of the other language type, the diversified video processing requirements of a user are met, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a video processing method, device, equipment and storage medium. Background Art

[0002] With the continuous development of computer technology, people's demands for video processing functions supported by video editing applications are becoming more and more diverse. For example, most video editing software supports audio recognition into text, that is, subtitle generation function.

[0003] In order to meet users' growing and diversified video processing needs, how to further enrich video processing methods is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In order to solve the above technical problems, an embodiment of the present disclosure provides a video processing method.

[0005] In a first aspect, the present disclosure provides a video processing method, the method comprising:

[0006] Performing speech recognition on the audio in the target video draft to obtain an initial text segment of the target video draft;

[0007] According to the initial language type and the target language type of the target video draft, the initial text segment is converted into a target text segment; wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type;

[0008] Generating a target audio segment based on the target text segment; wherein the target audio segment belongs to the target language type;

[0009] An editing result video draft corresponding to the target video draft is generated based on the target audio segment; wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment corresponding to the target video draft, and the editing result video draft belongs to the target language type.

[0010] In an optional implementation manner, before converting the language of the initial text segment to obtain the target text segment according to the initial language type and the target language type of the target video draft, the method further includes:

[0011] Determining an initial language type of the target video draft according to an initial text segment of the target video draft;

[0012] And, receiving a target language type input for the target video draft.

[0013] In an optional implementation manner, determining the initial language type of the target video draft according to the initial text segment of the target video draft includes:

[0014] Converting the text in the initial text segment of the target video draft into a preset type code;

[0015] The initial language type of the target video draft is determined according to the coding range to which the preset type code corresponding to the text belongs.

[0016] In an optional implementation manner, the language conversion of the initial text segment to obtain the target text segment according to the initial language type and the target language type of the target video draft includes:

[0017] If it is determined that the initial language type and the target language type belong to different dialect types under the same language type, the initial text segment of the target video draft is determined as the target text segment.

[0018] In an optional implementation manner, before converting the language of the initial text segment to obtain the target text segment according to the initial language type and the target language type of the target video draft, the method further includes:

[0019] Performing speech recognition on the audio in the target video draft to obtain speech voiceprint information of the target video draft;

[0020] Different text tracks are generated for different voice print information; wherein the different text tracks are used to carry initial text segments corresponding to different voice print information.

[0021] In an optional implementation manner, generating a target audio segment based on the target text segment includes:

[0022] Determine text reading sound information corresponding to the first text track;

[0023] The text reading sound information is used to generate a target audio segment based on the target text segment on the first text track.

[0024] In an optional implementation manner, before generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes:

[0025] According to the duration of the initial audio segment corresponding to the target audio segment, the playback speed of the target audio segment is adjusted to obtain a target audio segment after speed change; wherein the initial audio segment belongs to the target video draft, and the difference between the duration of the target audio segment after speed change and the duration of the initial audio segment is not greater than a preset duration threshold;

[0026] Correspondingly, the step of generating an editing result video draft corresponding to the target video draft based on the target audio segment includes:

[0027] Based on the speed-changed target audio segment, an editing result video draft corresponding to the target video draft is generated.

[0028] In an optional implementation manner, after generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes:

[0029] In response to a selection operation on a first target audio segment in the editing result video draft, displaying a plurality of candidate text contents corresponding to the first target audio segment;

[0030] In response to a selection operation on a target candidate text content among the plurality of candidate text contents, generating an audio segment corresponding to the target candidate text content;

[0031] The first target audio segment in the editing result video draft is updated using the audio segment corresponding to the target candidate text content.

[0032] In an optional implementation manner, before generating the editing result video draft corresponding to the target video draft based on the target audio segment corresponding to the initial audio segment, the method further includes:

[0033] Determining the target text segment as a subtitle segment of a corresponding target audio segment;

[0034] Accordingly, the step of generating an editing result video draft corresponding to the target video draft based on the target audio segment corresponding to the initial audio segment includes:

[0035] Based on the target audio segment corresponding to the initial audio segment and the subtitle segment of the target audio segment, an editing result video draft corresponding to the target video draft is generated.

[0036] In an optional implementation manner, before performing speech recognition on the audio in the target video draft to obtain the initial text segment corresponding to the target video draft, the method further includes:

[0037] Determine a target video draft based on the original video;

[0038] Correspondingly, after generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes:

[0039] In response to the video export operation, a target video corresponding to the original video is generated based on the editing result video draft; wherein the target video belongs to the target language type.

[0040] In a second aspect, the present disclosure provides a video processing device, the device comprising:

[0041] A speech recognition module, used for performing speech recognition on the audio in the target video draft to obtain an initial text segment of the target video draft;

[0042] A conversion module, configured to perform language conversion on the initial text segment to obtain a target text segment according to the initial language type and the target language type of the target video draft; wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type;

[0043] A first generating module, configured to generate a target audio segment based on the target text segment; wherein the target audio segment belongs to the target language type;

[0044] The second generation module is used to generate an editing result video draft corresponding to the target video draft based on the target audio segment; wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment of the target video draft, and the editing result video draft belongs to the target language type.

[0045] In a third aspect, the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the above method.

[0046] In a fourth aspect, the present disclosure provides a video processing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0047] In a fifth aspect, the present disclosure provides a computer program product, wherein the computer program product comprises a computer program / instructions, and the computer program / instructions implement the above method when executed by a processor.

[0048] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has at least the following advantages:

[0049] The disclosed embodiment provides a video processing method, first, performing speech recognition on the audio in the target video draft to obtain the initial text segment of the target video draft, performing language conversion on the initial text segment according to the initial language type and the target language type of the target video draft to obtain the target text segment, wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type, then, generating a target audio segment based on the target text segment, wherein the target audio segment belongs to the target language type, and finally, generating an editing result video draft corresponding to the target video draft based on the target audio segment, wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial audio segment in the target video draft, and the editing result video draft belongs to the target language type. It can be seen that the disclosed embodiment realizes the video editing function of converting a target video draft of one language type into an editing result video draft of another language type by performing language conversion on the initial text segment of the target video draft to obtain a target text segment belonging to the target language type, and generating a target audio segment belonging to the target language type based on the target text segment, and then generating an editing result video draft, thereby meeting the user's growing diversified video processing needs and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A flowchart of a video processing method provided by an embodiment of the present disclosure;

[0053] Figure 2 A schematic diagram of a video editing page provided by an embodiment of the present disclosure;

[0054] Figure 3 A schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure;

[0055] Figure 4 A schematic diagram of the structure of a video processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0058] With the continuous development of computer technology, people's demands for video processing functions supported by video editing applications are becoming more and more diverse. For example, most video editing software supports audio recognition into text, that is, subtitle generation function.

[0059] In order to meet users' growing and diversified video processing needs, how to further enrich video processing methods is a technical problem that needs to be solved urgently.

[0060] To this end, an embodiment of the present disclosure provides a video processing method, first, performing speech recognition on the audio in a target video draft to obtain an initial text segment of the target video draft, and performing language conversion on the initial text segment of the target video draft according to the initial language type and the target language type of the target video draft to obtain a target text segment, wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type; then, generating a target audio segment based on the target text segment, wherein the target audio segment belongs to the target language type; finally, generating an editing result video draft corresponding to the target video draft based on the target audio segment, wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment in the target video draft, and the editing result video draft belongs to the target language type. It can be seen that the embodiment of the present disclosure realizes the video editing function of converting a target video draft of one language type into an editing result video draft of another language type by performing language conversion on an initial text segment of a target video draft to obtain a target text segment belonging to the target language type, and generating a target audio segment belonging to the target language type based on the target text segment, and then generating an editing result video draft, thereby meeting the users' growing diversified video processing needs and improving the user experience.

[0061] Based on this, the present disclosure provides a video processing method, referring to Figure 1 , is a flowchart of a video processing method provided by an embodiment of the present disclosure, the method comprising:

[0062] S101: Performing speech recognition on the audio in the target video draft to obtain an initial text segment of the target video draft;

[0063] The video processing method provided by the embodiments of the present disclosure can be applied to a client. For example, the client may include a mobile client (referred to as a mobile client) deployed on a smart phone, tablet computer, etc., and may also include a PC client (referred to as a PC client) deployed on a desktop computer, etc.

[0064] In the disclosed embodiment, the target video draft may be any video draft in the draft box of the current user, or a video draft being displayed on a video editing page, etc.

[0065] The initial text segment is the text content obtained after performing speech recognition on the audio in the target video draft.

[0066] S102: According to the initial language type and the target language type of the target video draft, language conversion is performed on the initial text segment to obtain a target text segment.

[0067] The initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type.

[0068] In an optional implementation, the target video draft is displayed on the video editing page, specifically, the video track, audio track, text track, etc. of the target video draft are displayed according to the timeline. The video track is used to carry the video screen of the target video draft based on the timeline, the audio track is used to carry the audio in the target video draft based on the same timeline, and the text track is used to carry the initial text segment based on the same timeline.

[0069] In the disclosed embodiment, the initial language type refers to the language type to which the target video draft belongs. Specifically, the language type of the target video draft is called the initial language type. The target language type can be any language type different from the initial language type. For example, if the initial language type is Chinese, the target language type can be English. It can also be any language type associated with the initial language type. For example, the target language type can also be a different dialect type of the same language type as the initial language type. For example, if the initial language type is Mandarin of the Chinese type, the target language type can be Shanghai dialect of the Chinese type, etc.

[0070] In the disclosed embodiment, before converting the initial text segment into the target text segment, the language type of the target video draft, ie, the initial language type, is first determined. Optionally, the initial language type of the target video draft is determined based on the initial text segment of the target video draft.

[0071] In an optional implementation, the text in the initial text segment of the target video draft is converted into a preset type code, and the initial language type of the target video draft is determined according to the code range to which the preset type code corresponding to the text belongs.

[0072] The preset type encoding may include a Unicode (also known as a unified code) encoding type. For example, assuming that the preset type encoding is Unicode encoding, assuming that the text in the initial text segment is in the Unicode encoding range of [0x4e00, 0x9fa5], then according to the Unicode encoding, it is determined that the text in the initial text segment belongs to Chinese characters, that is, the initial language type of the target video draft is Chinese.

[0073] In an optional implementation, before determining the target language type of the target video draft, target language type options may be displayed, and the user may select the target language type of the target video draft from the target language type options. In addition, the user may also input the target language type for the target video draft, thereby determining the target language type of the target video draft. The target language type options may include at least one language type different from the initial language type of the target video draft.

[0074] In the disclosed embodiment, after the target language type and the initial language type of the target video draft are determined, the initial text segment of the target video draft is language-converted to obtain the target text segment.

[0075] In an optional implementation, based on the target language type and initial language type of the target video draft, a corresponding translation service is called to perform language translation on the initial text segment to obtain a target text segment corresponding to the initial text segment, wherein the translation service may specifically be a pre-deployed translation tool, translation module, etc.

[0076] In actual applications, if the initial language type and the target language type are different dialect types under the same language type, there is no need to perform language conversion on the initial text segment, that is, the initial text segment can be directly used as the target text segment.

[0077] In the disclosed embodiment, the language type conversion of the text in the target video draft is completed by converting the initial text segment of the target video draft into a target text segment belonging to the target language type.

[0078] In an optional implementation, on the video editing page of the target video draft, the target text segment can be used to replace the initial text segment on the text track based on the timeline, that is, the target text segment can be displayed on the video editing page.

[0079] In actual applications, in order to support the ability to distinguish characters through audio in the target video draft to further enhance the user experience, the embodiment of the present disclosure also needs to perform speech recognition on the audio in the target video draft before performing language conversion on the initial text segment to obtain the target text segment, so as to obtain the voice print information of the target video draft.

[0080] Specifically, voice recognition is performed on the audio clip in the target video draft to obtain the voice and soundprint information of the target video draft, and different text tracks are generated for different voice and soundprint information, wherein different text tracks are used to carry the initial text clips corresponding to different voice and soundprint information. In other words, different text tracks are added to the initial text clips based on the voice and soundprint information.

[0081] like Figure 2 , which is a schematic diagram of a video editing page provided by an embodiment of the present disclosure. The video editing page displays a target video draft 201, a video track corresponding to the target video draft, and text track 1 and text track 2, which respectively carry initial text segments of corresponding voice print information based on the timeline.

[0082] For example, assuming that the target video draft contains sound A and sound B, through speech recognition, voiceprint information a (corresponding to sound A) and voiceprint information b (corresponding to sound B) are obtained, and text track 1 and text track 2 are generated based on the voiceprint information respectively. Text track 1 is used to carry the initial text segment of voiceprint information a based on the timeline, and text track 2 is used to carry the initial text segment of voiceprint information b based on the timeline.

[0083] S103: Generate a target audio segment based on the target text segment.

[0084] The target audio segment belongs to the target language type.

[0085] In the disclosed embodiment, after voice recognition is performed on the audio in the target video draft to obtain different text tracks, based on the initial text segments corresponding to the different voice print information carried by different text tracks, the initial text segments on the different text tracks are converted into corresponding target text segments, and then corresponding target audio segments are generated based on the target text segments on the different text tracks for different characters.

[0086] In an optional implementation, before generating a target audio segment based on a target text segment, first determine the text reading sound information corresponding to the text track where the target text segment is located. Specifically, determine the text reading sound information corresponding to the first text track, and use the text reading sound information to generate a target audio segment based on the target text segment on the first text track. The first text track is any text track among multiple text tracks on the video editing page.

[0087] In an optional implementation, a human voice reading method is used to generate a corresponding target audio segment based on the target text segment to complete the process of converting text to audio. It is worth noting that the embodiments of the present disclosure do not limit other methods that can achieve text conversion to audio.

[0088] In the disclosed embodiment, different text reading sound information is determined for different text tracks to generate target audio segments of different human voices, thereby presenting a variety of human characters, ensuring the video editing effect, and further improving the user experience.

[0089] S104: Generate an editing result video draft corresponding to the target video draft based on the target audio segment.

[0090] There is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment of the target video draft, and the editing result video draft belongs to the target language type.

[0091] In an optional implementation, after the target audio segment is generated, the target audio segment is used to replace the audio in the target video draft to obtain an edited video draft.

[0092] Specifically, the audio and background music in the target video draft are separated, and the corresponding audio is replaced with the target audio clip. After the replacement is completed, the background music and the target audio clip are synthesized to obtain the edited result video draft.

[0093] In addition, subtitles can also be added to the editing result video draft, that is, displayed in the video screen in the form of text. Specifically, before generating the editing result video draft corresponding to the target video draft based on the target audio segment, the target text segment is determined as the subtitle segment of the corresponding target audio segment, and then, based on the target audio segment and the subtitle segment of the target audio segment, the editing result video draft corresponding to the target video draft is generated. Among them, the target text segment is determined as the subtitle segment of the corresponding target audio segment, that is, the target text segment is converted into the subtitle segment of the target audio segment.

[0094] In an optional implementation, based on the target text segment, a subtitle display service can be used to convert the target text segment into a subtitle segment, and then the target audio segment and the subtitle segment of the target audio segment are synthesized to generate an editing result video draft displaying subtitles, wherein the subtitle display service can specifically be a pre-deployed subtitle display tool, subtitle display module, etc.

[0095] In addition, in order to ensure that the displayed subtitles are consistent with the punctuation and speed of the target audio segment, the subtitle segment must be adjusted to achieve synchronization with the target audio segment.

[0096] In the video processing method provided by the embodiment of the present disclosure, first, speech recognition is performed on the audio in the target video draft to obtain the initial text segment of the target video draft, and the initial text segment is converted into a target text segment according to the initial language type and the target language type of the target video draft, wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type, then, a target audio segment is generated based on the target text segment, wherein the target audio segment belongs to the target language type, and finally, an editing result video draft corresponding to the target video draft is generated based on the target audio segment, wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial audio segment in the target video draft, and the editing result video draft belongs to the target language type. It can be seen that the embodiment of the present disclosure realizes the video editing function of converting a target video draft of one language type into an editing result video draft of another language type by converting the initial text segment of the target video draft into a target text segment belonging to the target language type, and generating a target audio segment belonging to the target language type based on the target text segment, and then generating an editing result video draft, thereby meeting the user's growing diversified video processing needs and improving the user experience.

[0097] In actual applications, in order to ensure that the target audio clip is consistent with the character's lip movements in the target video draft and to further enhance the user experience, in the disclosed embodiment, before generating the editing result video draft corresponding to the target video draft based on the target audio clip, the playback speed of the target audio clip is adjusted to achieve synchronization between the target audio clip and the character's lip movements.

[0098] Specifically, the play speed of the target audio segment is adjusted according to the duration of the initial audio segment corresponding to the target audio segment to obtain a target audio segment after speed change.

[0099] Among them, the initial audio segment belongs to the target video draft, that is, the audio file of the target video draft includes multiple initial audio segments, and the difference between the duration of the target audio segment after speed change and the duration of the corresponding initial audio segment is not greater than the preset duration threshold. The preset duration threshold represents the maximum value of the difference between the duration of the target audio segment and the duration of the initial audio segment, that is, the difference between the durations of the two is less than or equal to the preset duration threshold.

[0100] For example, assuming that the duration of the initial audio segment is 8 seconds, if the duration of the target audio segment is 5 seconds, the playback speed of the target audio segment is slowed down so that the difference between the two durations is less than or equal to the preset duration threshold. If the duration of the target audio segment is 10 seconds, the playback speed of the target audio segment is accelerated so that the difference between the two durations is less than or equal to the preset duration threshold.

[0101] Based on the target audio segment after speed change, an editing result video draft corresponding to the target video draft is generated. The above content has detailed how the target audio segment generates the editing result video draft corresponding to the target video draft, which will not be repeated here.

[0102] It should be noted that based on the target audio clip after speed change, the playback speed of the background music also needs to be adjusted before it is synthesized with the background music in the target video draft to ensure that the background music and the target audio clip remain synchronized, so as to further enhance the user experience.

[0103] The disclosed embodiment adjusts the playback speed of the target audio segment to synchronize the target audio segment with the character's lip movements, thereby making the edited video draft more realistic after the language type is converted.

[0104] In actual applications, an editing result video draft corresponding to the target video draft is generated based on the target audio clip. During the preview of the editing result video draft, inappropriate target audio clips in the editing result video draft can also be replaced to meet the needs of users. In the embodiment of the present disclosure, alternative audio clips are provided, that is, the sound and text content are changed so that users can select suitable target audio clips.

[0105] Specifically, when a selection operation is received for the first target audio segment in the editing result video draft, multiple candidate text contents corresponding to the first target audio segment are displayed; when a selection operation is received for a target candidate text content among the multiple candidate text contents, an audio segment corresponding to the target candidate text content is generated, and the first target audio segment in the editing result video draft is updated using the audio segment corresponding to the target candidate text content.

[0106] Based on the updated first target audio segment, it is necessary to readjust the playback speed of the first target audio segment according to the playback duration of the initial audio segment, so that when the editing result video draft is played, the first target audio is synchronized with the character's lip movements.

[0107] The first target audio segment is any target audio segment in the editing result video draft, and the multiple candidate text contents are different translation results corresponding to the first target audio segment under the same target language type. For example, assuming that the target language type is Chinese, candidate text content 1 is "You are so beautiful", and candidate text content 2 is "You are so beautiful".

[0108] In addition, the selection operation of the first target audio segment and the selection operation of the target candidate text content may be a single-click or double-click operation on the first target audio segment and the target candidate text content, and the embodiments of the present disclosure do not impose any limitation on this.

[0109] In the disclosed embodiment, by providing candidate text content and corresponding audio clips, the target audio clip in the editing result video draft is changed, which meets the needs of users and improves the user experience.

[0110] In the disclosed embodiment, before performing speech recognition based on the audio in the target audio segment to obtain the initial text segment corresponding to the target video draft, the target video draft may be obtained in advance.

[0111] Specifically, the target video draft is determined based on the original video, wherein the original video may include video material in the user's photo album, or video material shot by the user based on a shooting page, and the like.

[0112] In an optional implementation, the user performs video editing operations on the original video based on the video editing page and saves it into a draft box. Then, the video draft corresponding to the original video in the draft box can be determined as the target video draft.

[0113] In another optional implementation, the original video is imported into a draft box to obtain a target video draft, and then the target video draft is subjected to language conversion editing and the like.

[0114] Based on the above embodiment, after determining the target video draft, the target video draft is subjected to speech recognition to obtain an initial text segment, and language conversion is performed based on the initial text segment to obtain a target text segment, and a target audio segment is generated based on the target text segment, and finally an editing result video draft corresponding to the target video draft is generated.

[0115] In actual applications, in order to support the release of the editing result video draft after language conversion, the embodiment of the present disclosure also supports video export operations. Specifically, in response to the video export operation, a target video corresponding to the original video is generated based on the editing result video draft, and the target video belongs to the target language type.

[0116] In an optional implementation, the video export operation may include a trigger operation of an export control on a video clip page, and when a trigger operation for video export is received, the edited video draft is generated into a target video, wherein the target video has a corresponding relationship with the original video and the target video belongs to the target language type. Then, based on the target video, when a trigger operation for a publishing control for the target video is received, the target video is published.

[0117] The disclosed embodiment supports the video export operation, thereby enabling the release of the target video after language conversion, and further improving the editing experience of the user.

[0118] Based on the above method embodiment, the present disclosure also provides a video processing device, referring to Figure 3 , is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure, the device comprising:

[0119] A speech recognition module 301 is used to perform speech recognition on the audio in the target video draft to obtain an initial text segment of the target video draft;

[0120] The conversion module 302 is used to perform language conversion on the initial text segment to obtain a target text segment according to the initial language type and the target language type of the target video draft; wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type;

[0121] A first generating module 303 is used to generate a target audio segment based on the target text segment; wherein the target audio segment belongs to the target language type;

[0122] The second generation module 304 is used to generate an editing result video draft corresponding to the target video draft based on the target audio segment; wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment of the target video draft, and the editing result video draft belongs to the target language type.

[0123] In an optional implementation, the device further includes:

[0124] A first determination module, configured to determine an initial language type of the target video draft according to an initial text segment of the target video draft;

[0125] The receiving module is used to receive a target language type input for the target video draft.

[0126] In an optional implementation manner, the first determining module includes:

[0127] A first conversion submodule, configured to convert text in the initial text segment of the target video draft into a preset type code;

[0128] The first determination submodule is used to determine the initial language type of the target video draft according to the coding range to which the preset type code corresponding to the text belongs.

[0129] In an optional implementation, the conversion module includes:

[0130] The second determination submodule is used to determine the initial text segment of the target video draft as the target text segment when it is determined that the initial language type and the target language type belong to different dialect types under the same language type.

[0131] In an optional implementation, the device further includes:

[0132] A first acquisition module is used to perform voice recognition on the audio clip in the target video draft to obtain the voice print information of the target video draft;

[0133] The third generating module is used to generate different text tracks for different voice print information; wherein the different text tracks are used to carry initial text segments corresponding to different voice print information.

[0134] In an optional implementation manner, the first generating module includes:

[0135] A third determination submodule is used to determine the text reading sound information corresponding to the first text track;

[0136] The first generating submodule is used to generate a target audio segment based on a target text segment on the first text track by using the text reading sound information.

[0137] In an optional implementation, the device further includes:

[0138] An adjustment module is used to adjust the playback speed of the target audio segment according to the duration of the initial audio segment corresponding to the target audio segment to obtain a target audio segment after speed change; wherein the initial audio segment belongs to the target video draft, and the difference between the duration of the target audio segment after speed change and the duration of the initial audio segment is not greater than a preset duration threshold;

[0139] Accordingly, the second generating module is specifically used for:

[0140] Based on the speed-changed target audio segment, an editing result video draft corresponding to the target video draft is generated.

[0141] In an optional implementation, the device further includes:

[0142] A display module, configured to display a plurality of candidate text contents corresponding to a first target audio segment in the editing result video draft in response to a selection operation on the first target audio segment;

[0143] A second acquisition module, configured to generate an audio segment corresponding to a target candidate text content in response to a selection operation on a target candidate text content among the plurality of candidate text contents;

[0144] An updating module is used to update the first target audio segment in the editing result video draft using the audio segment corresponding to the target candidate text content.

[0145] In an optional implementation, the device further includes:

[0146] A second determination module, configured to determine the target text segment as a subtitle segment of a corresponding target audio segment;

[0147] Accordingly, the second generating module is specifically used for:

[0148] Based on the target audio segment corresponding to the initial audio segment and the subtitle segment of the target audio segment, an editing result video draft corresponding to the target video draft is generated.

[0149] In an optional implementation, the device further includes:

[0150] A third determination module, used to determine a target video draft based on the original video;

[0151] Accordingly, the device further comprises:

[0152] The fourth generating module is used to generate a target video corresponding to the original video based on the editing result video draft in response to the video export operation; wherein the target video belongs to the target language type.

[0153] In the video processing device provided by the embodiment of the present disclosure, first, speech recognition is performed on the audio in the target video draft to obtain the initial text segment of the target video draft, and the initial text segment is converted into a target text segment according to the initial language type and the target language type of the target video draft, wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type, then, a target audio segment is generated based on the target text segment, wherein the target audio segment belongs to the target language type, and finally, an editing result video draft corresponding to the target video draft is generated based on the target audio segment, wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial audio segment in the target video draft, and the editing result video draft belongs to the target language type. It can be seen that the embodiment of the present disclosure realizes the video editing function of converting a target video draft of one language type into an editing result video draft of another language type by converting the initial text segment of the target video draft into a target text segment belonging to the target language type, and generating a target audio segment belonging to the target language type based on the target text segment, and then generating an editing result video draft, thereby meeting the user's growing diversified video processing needs and improving the user experience.

[0154] In addition to the above-mentioned method and apparatus, the embodiments of the present disclosure further provide a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device implements the video processing method described in the embodiments of the present disclosure.

[0155] The embodiment of the present disclosure further provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the video processing method described in the embodiment of the present disclosure is implemented.

[0156] In addition, the present disclosure also provides a video processing device, see Figure 4 As shown, it may include:

[0157] Processor 401, memory 402, input device 403 and output device 404. The number of processors 401 in the video processing device can be one or more. Figure 4 In some embodiments of the present disclosure, the processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means, wherein: Figure 4 The example of connecting through bus is taken in the following.

[0158] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing of the video processing device by running the software programs and modules stored in the memory 402. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. The input device 403 can be used to receive input digital or character information, and generate signal input related to user settings and function control of the video processing device.

[0159] Specifically in this embodiment, the processor 401 will load the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 will run the applications stored in the memory 402, thereby realizing various functions of the above-mentioned video processing device.

[0160] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0161] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video processing method, characterized in that: The method comprises: Performing speech recognition on the audio in the target video draft to obtain an initial text segment of the target video draft; According to the initial language type and the target language type of the target video draft, the initial text segment is converted into a target text segment; wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type; Generating a target audio segment based on the target text segment; wherein the target audio segment belongs to the target language type; An editing result video draft corresponding to the target video draft is generated based on the target audio segment; wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment of the target video draft, and the editing result video draft belongs to the target language type.

2. The method according to claim 1, characterized in that Before converting the language of the initial text segment to obtain the target text segment according to the initial language type and the target language type of the target video draft, the method further includes: Determining an initial language type of the target video draft according to an initial text segment of the target video draft; And, receiving a target language type input for the target video draft.

3. The method according to claim 2, characterized in that The step of determining the initial language type of the target video draft according to the initial text segment of the target video draft comprises: Converting the text in the initial text segment of the target video draft into a preset type code; The initial language type of the target video draft is determined according to the coding range to which the preset type code corresponding to the text belongs.

4. The method according to claim 1, characterized in that: The step of converting the language of the initial text segment to obtain the target text segment according to the initial language type and the target language type of the target video draft includes: If it is determined that the initial language type and the target language type belong to different dialect types under the same language type, the initial text segment of the target video draft is determined as the target text segment.

5. The method according to claim 1, characterized in that Before converting the language of the initial text segment to obtain the target text segment according to the initial language type and the target language type of the target video draft, the method further includes: Performing speech recognition on the audio in the target video draft to obtain speech voiceprint information of the target video draft; Different text tracks are generated for different voice print information; wherein the different text tracks are used to carry initial text segments corresponding to different voice print information.

6. The method according to claim 5, characterized in that The generating a target audio segment based on the target text segment comprises: Determine text reading sound information corresponding to the first text track; The text reading sound information is used to generate a target audio segment based on the target text segment on the first text track.

7. The method according to claim 1, characterized in that Before generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes: According to the duration of the initial audio segment corresponding to the target audio segment, the playback speed of the target audio segment is adjusted to obtain a target audio segment after speed change; wherein the initial audio segment belongs to the target video draft, and the difference between the duration of the target audio segment after speed change and the duration of the initial audio segment is not greater than a preset duration threshold; Correspondingly, the step of generating an editing result video draft corresponding to the target video draft based on the target audio segment includes: Based on the speed-changed target audio segment, an editing result video draft corresponding to the target video draft is generated.

8. The method according to claim 1, characterized in that After generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes: In response to a selection operation on a first target audio segment in the editing result video draft, displaying a plurality of candidate text contents corresponding to the first target audio segment; In response to a selection operation on a target candidate text content among the plurality of candidate text contents, generating an audio segment corresponding to the target candidate text content; The first target audio segment in the editing result video draft is updated using the audio segment corresponding to the target candidate text content.

9. The method according to claim 1, characterized in that: Before generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes: Determining the target text segment as a subtitle segment of a corresponding target audio segment; Correspondingly, the step of generating an editing result video draft corresponding to the target video draft based on the target audio segment includes: Based on the target audio segment and the subtitle segment of the target audio segment, an editing result video draft corresponding to the target video draft is generated.

10. The method according to claim 1, characterized in that Before performing speech recognition on the audio in the target video draft to obtain the initial text segment corresponding to the target video draft, the method further includes: Determine a target video draft based on the original video; Correspondingly, after generating the editing result video draft corresponding to the target video draft based on the target audio segment, the method further includes: In response to the video export operation, a target video corresponding to the original video is generated based on the editing result video draft; wherein the target video belongs to the target language type.

11. A video processing device, characterized in that: The device comprises: A speech recognition module, used for performing speech recognition on the audio in the target video draft to obtain an initial text segment of the target video draft; A conversion module, configured to perform language conversion on the initial text segment to obtain a target text segment according to the initial language type and the target language type of the target video draft; wherein the initial language type is used to characterize the language type to which the target video draft belongs, and the target text segment belongs to the target language type; A first generating module, configured to generate a target audio segment based on the target text segment; wherein the target audio segment belongs to the target language type; The second generation module is used to generate an editing result video draft corresponding to the target video draft based on the target audio segment; wherein there is a corresponding relationship between the target audio segment in the editing result video draft and the initial text segment of the target video draft, and the editing result video draft belongs to the target language type.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 10.

13. A video processing device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

Images