Content dubbing method and device, electronic equipment, medium and program product

Abstract

The invention provides a content dubbing method and device, electronic equipment, a medium and a program product, and relates to the technical field of video processing, the method comprises the following steps: in response to dubbing operation on content including an image and / or a video, acquiring a reference audio matched with the content, the reference audio being used for generating an ending background audio played before the content is ended, according to the method, the content is dubbed according to the content and the reference audio, the ending background music generated based on the reference audio is suitable for dubbing the ending part of the content, so that the ending part of the content is matched with the ending background audio, the style of the whole music is coherent in listening feeling and consistent with the emotional trend of the content, and the dubbing quality of the content is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video processing, and in particular, to a content dubbing method, apparatus, electronic device, medium, and program product. Background Art

[0002] With the development of social media and video technology, more and more users use dubbing to enrich the audio-visual effects of videos when making videos. In the related art, the rhythm of the dubbing is inconsistent with the rhythm of the video. For example, when the video is about to end, the dubbing is still at the climax. Therefore, there is a problem of low quality of the dubbed videos produced in the related art. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a content dubbing method, apparatus, electronic device, medium, and program product.

[0004] According to a first aspect of an embodiment of the present disclosure, a content dubbing method is provided. The method includes: In response to a dubbing operation on the content, obtaining a reference audio adapted to the content, where the content includes an image and / or a video, and the reference audio is used to generate an ending background audio played before the end of the content; Dubbing the content according to the content and the reference audio.

[0005] Optionally, the ending background audio is used to gradually approach the end of the content.

[0006] Optionally, the obtaining the reference audio adapted to the content includes: Performing scene recognition on the content to obtain a target scene corresponding to the content; Obtaining a reference audio applicable to the target scene from a preset audio library, where the preset audio library is used to store reference audios applicable to different scenes.

[0007] Optionally, the preset audio library stores multiple reference audios of different durations applicable to the same scene. The obtaining the reference audio applicable to the target scene from the preset audio library includes: Obtaining multiple reference audios applicable to the target scene from the preset audio library; Determining a reference audio with a duration equal to a target duration from the multiple reference audios.

[0008] Optionally, the target duration is obtained according to the total duration of the content.

[0009] Optionally, the dubbing the content according to the content and the reference audio includes: Perform zero-padding on the reference audio according to the duration difference between the total duration of the content and the duration of the reference audio to obtain an intermediate audio with a duration equal to the total duration of the content; Generate a recommended audio for dubbing the content based on the content and the intermediate audio, and synthesize the recommended audio with the content.

[0010] Optionally, the generating a recommended audio for dubbing the content based on the content and the intermediate audio includes: Perform content understanding on the content to obtain text information for describing the content; Obtain dubbing characteristics according to the text information, where the dubbing characteristics include at least one of the following: the emotion information of the recommended audio, the style information of the recommended audio, and the number of beats of the recommended audio; Generate the recommended audio for dubbing the content according to the dubbing characteristics and the intermediate audio.

[0011] According to a second aspect of the embodiments of the present disclosure, there is provided a content dubbing device, the device includes: A response module, configured to obtain a reference audio adapted to the content in response to a dubbing operation on the content, where the content includes an image and / or a video, and the reference audio is used to generate an ending background audio played before the end of the content; A dubbing module, configured to dub the content according to the content and the reference audio.

[0012] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: A processor; A memory for storing processor-executable instructions; Wherein, the processor is configured to execute the instructions to implement the steps of the method described in the first aspect.

[0013] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the content dubbing method provided in the first aspect of the present disclosure are implemented.

[0014] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the content dubbing method provided in the first aspect of the present disclosure are implemented.

[0015] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: In response to a dubbing operation on content including images and / or videos, a reference audio adapted to the content is obtained, and the reference audio is used to generate an ending background audio played before the end of the content. Then, according to the content and the reference audio, the content is dubbed. Since the ending background music generated based on the reference audio is suitable for dubbing the ending part of the content, the ending part of the content is adapted to the ending background audio, and the style and listening feeling of the whole piece of music are coherent, consistent with the emotional trend of the content, making the overall dubbing effect coordinated, thereby improving the dubbing quality of the content.

[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0018] Figure 1 is a flowchart of a content dubbing method shown according to an exemplary embodiment.

[0019] Figure 2 is Figure 1 a schematic flowchart of step S110 in

[0020] Figure 3 is Figure 1 a schematic flowchart of step S120 in

[0021] Figure 4 is a flowchart of a content dubbing method shown according to another exemplary embodiment.

[0022] Figure 5 is a block diagram of a content dubbing device shown according to an exemplary embodiment.

[0023] Figure 6 is a block diagram of an electronic device for a content dubbing method shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0025] With the development of social media and video technology, more and more users like to record by sharing photos or videos. However, it is often difficult to achieve the expected effect by simply expressing emotions through static photos. Therefore, more and more people begin to use album video production tools to edit photos into videos to better tell stories and express emotions. However, some album video production tools generate silent videos. Adding voiceovers to silent album videos can improve the quality and viewing experience of the videos, but traditional voiceover methods require manual recording, which is not only costly and time-consuming, but also requires professional recording equipment and personnel.

[0026] To this end, there has emerged a method of generating background music based on a diffusion model and then synthesizing it with the video to obtain the final voiceover video. However, in the obtained voiceover video, the rhythm of the voiceover does not match the rhythm of the video, and the inappropriate voiceover rhythm results in a low quality of the voiceover video. For example, in the first case, when the video is approaching the end of playback, the voiceover is still at the climax. Or, in the second case, the video is at a high-pitched part while the voiceover suddenly stops. To improve the first case, a fade-out function is usually added at the end of the video to achieve the consistency between the video and the voiceover by reducing the volume of the voiceover at the end of the video. To improve the second case, a fade-in function is usually added at the end of the video to increase the volume of the voiceover at the high-pitched part of the video to achieve the consistency between the video and the voiceover.

[0027] In video editing, the voiceover needs to follow the video content, gradually enter the mood from the beginning, reach the climax, and then fade out. Since in video editing, the length and number of sub-videos selected by the user are not fixed, it will lead to a mismatch between the mood trend of the generated music and the video. Since the music starts from scratch, it is usually easier to achieve the mood elevation at the beginning part, but the ending is difficult to handle because for videos of different lengths, the music may suddenly stop at the climax. And fade-in and fade-out mainly involve the control of amplitude, simply controlling the volume, and the user's experience in the music rhythm is not good.

[0028] To solve the above problems, embodiments of the present disclosure provide a content voiceover method. Please refer to Figure 1 The content voiceover method can be applied to Figure 5 the content voiceover device 300 shown in Figure 6 the electronic device 800 shown in Figure 1 computer program products, and computer-readable storage media. In this embodiment, taking the application to an electronic device as an example, the electronic device can be a mobile terminal, a tablet computer, a computer, a server, etc., and this embodiment does not limit this. The following will elaborate in detail on the Step S110: In response to a dubbing operation on content, obtain a reference audio adapted to the content, where the content includes an image and / or a video, and the reference audio is used to generate an ending background audio played before the end of the content.

[0029] An application supporting the dubbing function is installed on the electronic device. The application can include social applications, camera applications, video applications, etc. The application can provide a dubbing interface for the user. When the user needs to dub content, the user can perform a dubbing operation on this dubbing interface. In response to the dubbing operation on the content, obtain a reference audio adapted to the content. The reference audio in this embodiment is an audio suitable for ending. The reference audio can be a sound segment of the ending part in existing music. The mood, rhythm, or melody of the reference audio is suitable for ending. For example, the reference audio is the audio within the target duration before the end of the music in the music library, and the target duration can be 1s - 4s.

[0030] For example, when the content includes multiple images, or the content includes an image and a video, the user can select the above content based on the dubbing interface, automatically synthesize a video based on the user's selection operation, and obtain a reference audio adapted to the automatically synthesized video. When the content includes a video, the user selects the content based on the dubbing interface and obtains a reference audio adapted to the selected video.

[0031] Step S120: Dub the content according to the content and the reference audio.

[0032] Dub the content according to the content and the reference audio to improve the richness of the content. Since the mood, rhythm, or melody of the reference audio is suitable for ending, then combined with the reference audio, the mood, rhythm, melody, etc. of the generated ending background audio generated by referring to the mood, rhythm, or melody of the reference audio are suitable for dubbing the ending of the content. Use the ending audio to dub the ending part of the content, so that the ending audio matches the ending part of the content, and a natural, coherent, and smooth end of both the dubbing and the content can be achieved.

[0033] In one implementation, the duration of the reference audio is a preset duration. Through the reference audio, or through the reference audio and the content, generate an ending background audio for the ending part of the content, that is, use the ending background audio to dub the part within the preset duration before the end of the content. In other parts of the content, dub through the reference audio, thereby completing the dubbing of the content.

[0034] In another embodiment, a recommended audio is generated based on the content and a reference audio, and the content is dubbed with the recommended audio. The duration of the recommended audio may be the same as that of the content. The ending background audio within a preset duration before the end of the recommended audio is generated through the reference audio, or through the reference audio and the content, and the audio for the remaining part of the recommended audio is generated through the content.

[0035] The content dubbing method provided in this embodiment responds to a dubbing operation on content including an image and / or a video, obtains a reference audio adapted to the content, and the reference audio is used to generate an ending background audio played before the end of the content. Then, the content is dubbed according to the content and the reference audio. Since the ending background music generated based on the reference audio is suitable for dubbing the ending part of the content, the ending part of the content is adapted to the ending background audio, and the style and listening feeling of the entire piece of music are coherent and consistent with the rhythm of the content, making the overall dubbing effect coordinated. Thereby, the dubbing quality of the content is improved, and the user's viewing experience is more fluent and natural.

[0036] Optionally, since the reference audio ends in a gradual change, the ending background audio generated using the reference audio is used to gradually approach the end of the content. For example, the gradual change method of the ending background audio may be that the emotion can fade from strong to weak, the rhythm of the music gradually slows down, and the volume gradually decreases at a preset rate. The content is smoothly transitioned to the end of the content with the gradually changing ending background audio, avoiding the abrupt feeling of the sudden end of the dubbing. The style of the dubbing obtained in this example is consistent with the emotional trend of the video, achieving natural dubbing for the ending part of the content.

[0037] In one embodiment, referring to Figure 2 , step S110 may include the following steps: Step S111, perform scene recognition on the content to obtain the target scene corresponding to the content.

[0038] The scenes in this step are preset, and the scenes may include a sports scene, a pet scene, a travel scene, a food scene, etc. The target scene is the scene corresponding to the content among multiple scenes.

[0039] The content is understood through a content understanding large model to obtain text information for describing the content. The content understanding large model is trained through sample content and sample text information for describing the sample content. For example, if the content expresses the information of having a meal together at the table, the corresponding text information may include "steamed fish", "thick soup", etc. Then, according to the text information, the target scene corresponding to the content is determined. Exemplarily, the corresponding relationship between the text information and the target scene is preset in advance, and according to this corresponding relationship, the target scene corresponding to the text information is determined, and this scene is the target scene corresponding to the content. Continuing with the above example, according to the text information such as "steamed fish" and "thick soup", the target scene is determined to be the food scene.

[0040] It should be noted that the information contained in the content to be dubbed may not be single, but relatively rich. Therefore, the number of target scenes corresponding to the same content can be one or multiple. For example, the content expresses the information of having a meal together at the table, and one of the people is holding a puppy. Through the content understanding large model to understand this content, the obtained text information may include "steamed fish", "thick soup", "puppy". According to the text information such as "steamed fish" and "thick soup", the target scene is determined to include the food scene. And according to the text information such as "puppy", the target scene is also determined to include the pet scene.

[0041] Another example is that when the number of target scenes is one, among the multiple scenes determined according to the text information, the one with more text information corresponding to the scene is used as the target scene. Continuing with the above example, the text information of the food scene is "steamed fish" and "thick soup", and the text information corresponding to the pet scene is "puppy". The number of text information corresponding to the food scene is more than that of the pet scene. Therefore, the food scene can be determined as the target scene.

[0042] Step S112: Obtain a reference audio applicable to the target scene from a preset audio library, where the preset audio library is used to store reference audios applicable to different scenes.

[0043] As a way, a preset audio library is used to store reference audios applicable to different scenarios. The musical styles, rhythms, and emotions of the reference audios for different scenarios may be different. The preset audio library stores multiple reference audios of different durations applicable to the same scenario. For example, the number of reference audios for the same scenario can be 5. Obtain multiple reference audios applicable to the target scenario from the preset audio library. Then, determine the reference audio with a duration equal to the target duration from the multiple reference audios. The rhythm trend of the reference audio with the same duration as the target duration has a higher degree of fit with the ending part of the content to be dubbed, and can perform a more coherent and smooth dubbing for the ending part of the content, thereby improving the dubbing quality of the content. When there is no reference audio with a duration equal to the target duration among the multiple reference audios, determine the one with the smallest difference between the duration and the target duration as the reference audio from the multiple reference audios.

[0044] Optionally, the target duration is the duration of the reference audio, and the target duration can be a pre-set duration. For example, the target duration is 1s - 4s. Setting the duration in advance can quickly dub the content.

[0045] Optionally, the target duration can also be obtained according to the total duration of the content. For example, the target duration accounts for a preset proportion of the total duration of the content. The preset proportion can be 5%, 8%, 10%, etc. Taking the total duration of the content as 20s and the preset proportion as 5%, then the corresponding target duration is 1s. The total duration of the content can determine the target duration of the reference audio. Dubbing the content according to the reference audio can improve the consistency between the dubbing and the content, make the dubbing more adaptable to the content, and the dubbing effect is better, thus improving the dubbing quality of the content.

[0046] In this embodiment, select the target scenario corresponding to the content. Since the preset audio library stores reference audios applicable to different scenarios, obtain the reference audio applicable to the target scenario from the preset audio library. According to the reference audio determined by the target scenario, use this reference audio to dub the ending of the content, and a more suitable ending for the content can be obtained, so that the audio and the content can synchronously reach the end.

[0047] In one embodiment, please refer to Figure 3 , step S120 may include the following steps: Step S121, perform a zero-padding operation on the reference audio according to the duration difference between the total duration of the content and the duration of the reference audio to obtain an intermediate audio with a duration equal to the total duration of the content.

[0048] The intermediate audio consists of a zero-padding part and a reference audio in sequence. The amplitude of the zero-padding part is 0, and the sound intensity is also 0. For example, if the duration of the content is 30s and the reference audio is 2s, the first 28s of the content is the zero-padding part, and the last 2s is the reference audio. Through the zero-padding operation, the position of the reference audio for dubbing the content can be determined.

[0049] Step S122: Generate a recommended audio for dubbing the content according to the content and the intermediate audio, and synthesize the recommended audio with the content.

[0050] Generate a recommended audio for dubbing the content according to the content and the intermediate audio. The recommended audio can be pure music, a song, a voiceover, etc. The total duration of the recommended audio is the same as the total duration of the content. Synthesize the recommended audio with the content to obtain a synthesized audio, so as to realize dubbing the content with the recommended audio.

[0051] The ending part of the recommended audio is the ending background audio. When dubbing the content with the recommended audio, when the recommended audio plays to the ending background audio, the content also plays to the ending part, and the ending background audio and the content end synchronously, making the trend of the dubbing and the content consistent and improving the dubbing quality.

[0052] Optionally, the number of recommended audios is multiple. Multiple recommended audios are displayed on the dubbing interface of the application, and the user can select the recommended audios based on the dubbing interface. In response to the selection operation, synthesize the recommended audio selected by the selection operation with the content to obtain a synthesized audio.

[0053] Optionally, the number of recommended audios can be 1. After generating the recommended audio, directly synthesize it with the content to obtain a synthesized audio, thereby realizing dubbing.

[0054] As a way, perform content understanding on the content to obtain text information for describing the content. An audio generation large model generates the recommended audio for dubbing the content according to the text information and the intermediate audio, where the audio generation large model is trained by sample text information, sample intermediate audio, and sample recommended audio. The audio generation large model can generate a recommended audio for dubbing by learning the sample text information, sample intermediate audio, and sample recommended audio. And the audio generation large model learns the sample intermediate audio for dubbing the ending part. Therefore, the ending background audio in the recommended audio generated by the audio generation large model is more suitable for dubbing the ending part. When dubbing the content, the rhythm of the recommended audio is consistent with the rhythm of the content. When the dubbing reaches the ending part, the ending background audio in the recommended audio ends synchronously with the content, improving the consistency of the dubbing.

[0055] Optionally, the audio generation large model may include a T2A (Text-to-Action) large model.

[0056] As another approach, perform content understanding on the content to obtain text information for describing the content; based on the text information, obtain dubbing characteristics, where the dubbing characteristics include at least one of the following: the emotional information of the recommended audio, the style information of the recommended audio, the beats per minute (BPM) of the recommended audio; based on the dubbing characteristics and the intermediate audio, generate the recommended audio for dubbing the content. The emotional information can be information such as cheerful, sad, high-pitched, angry, etc., the style information can be information such as folk, rock, ancient style, nostalgic, etc., and the beats per minute is the unit of the number of beats per minute. Optionally, the generated recommended audio can be pure music.

[0057] As another approach, perform content understanding on the content to obtain text information for describing the content; based on the text information, obtain dubbing characteristics, where the dubbing characteristics include at least one of the following: the emotional information of the recommended audio, the style information of the recommended audio, the beats per minute of the recommended audio; based on the text information, the dubbing characteristics, and the intermediate audio, generate the recommended audio for dubbing the content.

[0058] The present disclosure also provides a content dubbing method. Please refer to Figure 4 , and the content dubbing method includes: Step S210: Input an image and / or a video into a content understanding large model to obtain text information.

[0059] Step S220: Input the text information and a reference audio into an audio generation model to obtain a recommended audio.

[0060] Among them, the reference audio is used to generate an ending background audio played before the end of the content. The recommended audio is used to dub the content.

[0061] Among them, steps S210 to S220 refer to the above embodiments and are not limited herein.

[0062] In the voice-over solutions in the related art, emotions are usually controlled to enter and exit through amplitude. However, according to the number and length of sub-videos input by the user, the length of the music generated each time is different. The entry of emotions starts from zero and can be well controlled, but the exit of emotions is a difficult problem. By only controlling through amplitude, only the volume can be reduced, but the actual emotion has not ended, affecting the overall video editing experience. The content voice-over method provided in this embodiment generates a recommended audio with reference to the rhythm, emotion, melody, etc. of the reference audio. Through the recommended audio, the ending part of the content can be naturally voice-overed, making the style of the entire voice-over coherent and smooth, thereby improving the voice-over quality and also enhancing the user's voice-over experience for the content.

[0063] Figure 5 FIG. is a block diagram of a content voice-over device shown according to an exemplary embodiment. Please refer to Figure 5 , the content voice-over device 300 includes: A response module 310, configured to obtain a reference audio adapted to the content in response to a voice-over operation on the content, where the content includes an image and / or a video, and the reference audio is used to generate an ending background audio played before the end of the content; A voice-over module 320, configured to perform voice-over on the content according to the content and the reference audio.

[0064] Optionally, the ending background audio is used to gradually approach the end of the content.

[0065] Optionally, the response module 310 includes: A target scene acquisition module, configured to perform scene recognition on the content to obtain a target scene corresponding to the content; A reference audio acquisition module, configured to obtain a reference audio applicable to the target scene from a preset audio library, where the preset audio library is used to store reference audios applicable to different scenes.

[0066] Optionally, the target scene acquisition module is specifically configured to perform content understanding on the content through a content understanding large model to obtain text information for describing the content, where the content understanding large model is trained through sample content and sample text information for describing the sample content; and determine the target scene corresponding to the content according to the text information.

[0067] Optionally, the preset audio library stores multiple reference audios with different durations applicable to the same scene. The reference audio acquisition module is specifically configured to obtain multiple reference audios applicable to the target scene from the preset audio library; and determine a reference audio with a duration equal to the target duration from the multiple reference audios.

[0068] Optionally, the target duration is obtained according to the total duration of the content.

[0069] Optionally, the dubbing module 320 includes: A zero-padding module configured to perform zero-padding on the reference audio according to the duration difference between the total duration of the content and the duration of the reference audio, to obtain an intermediate audio with a duration equal to the total duration of the content; A generation module configured to generate a recommended audio for dubbing the content according to the content and the intermediate audio, and synthesize the recommended audio with the content.

[0070] Optionally, the generation module includes: An understanding module configured to perform content understanding on the content to obtain text information for describing the content; A recommendation module configured to generate the recommended audio for dubbing the content through an audio generation large model according to the text information and the intermediate audio, where the audio generation model is obtained by training with sample text information, sample intermediate audio, and sample recommended audio.

[0071] Optionally, the recommendation module includes: A text information obtaining module configured to perform content understanding on the content to obtain text information for describing the content; A dubbing characteristic obtaining module configured to obtain dubbing characteristics according to the text information, where the dubbing characteristics include at least one of the following: the emotion information of the recommended audio, the style information of the recommended audio, and the number of beats of the recommended audio; A recommended audio generation module configured to generate the recommended audio for dubbing the content according to the dubbing characteristics and the intermediate audio.

[0072] Regarding the content dubbing device 300 in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0073] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the content dubbing method provided by the present disclosure are implemented.

[0074] Figure 6 It is a block diagram of an electronic device for a content dubbing method shown according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0075] Please refer to Figure 6 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output interface 812, a sensor component 814, and a communication component 816.

[0076] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0077] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0078] The power component 806 provides power to various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0079] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0080] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0081] The input / output interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0082] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0083] The communication component 816 is configured to facilitate communication, in a wired or wireless manner, between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0084] In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0085] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions may be executed by a processor 820 of the electronic device 800 to complete the above-described methods. For example, the non-transitory computer-readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0086] In another exemplary embodiment, a computer program product is also provided, and the computer program product includes a computer program capable of being executed by a programmable device, and the computer program has a code portion for performing the above-described content dubbing method when executed by the programmable device.

[0087] It should be understood that, unless otherwise specifically stated, the features of some embodiments of the various aspects of the present disclosure described herein may be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of any two or more of them; similarly, "at least one of..." includes any one of the related listed items and any combination of any two or more of them.

[0088] In addition, the word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Instead, the word exemplary is intended to present concepts in a concrete fashion. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified, or clear from the context, "X applies A or B" is intended to mean any of the natural inclusive permutations. That is, if X applies A; X applies B; or X applies both A and B, then "X applies A or B" is satisfied under any of the foregoing instances. Additionally, unless otherwise specified or clear from the context that it is referring to the singular form, the articles "a" and "an" as used in this application and the appended claims are generally understood to mean "one or more".

[0089] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. Specifically with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if not structurally equivalent to the disclosed structure. Additionally, although a particular feature of the present disclosure may have been disclosed with respect to only one of several implementations, such feature may, as may be desired and advantageous for any given or particular application, be combined with one or more other features of other implementations. Further, with respect to the terms "comprising", "possessing", "having", "include", or variants thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term "including".

[0090] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known or customary techniques in the art that are not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

[0091] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the figures, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A content dubbing method, characterized in that: The method comprises: In response to a dubbing operation on content, obtaining reference audio adapted to the content, wherein the content includes an image and / or a video, and the reference audio is used to generate an ending background audio played before the content ends; Dubbing the content is performed based on the content and the reference audio.

2. The method according to claim 1, characterized in that The ending background audio is used to fade toward the end of the content.

3. The method according to claim 1, characterized in that The obtaining of reference audio adapted to the content includes: Performing scene recognition on the content to obtain a target scene corresponding to the content; A reference audio suitable for the target scene is obtained from a preset audio library, wherein the preset audio library is used to store reference audio suitable for different scenes.

4. The method according to claim 3, characterized in that The preset audio library stores a plurality of reference audios of different durations applicable to the same scene, and obtaining the reference audio applicable to the target scene from the preset audio library includes: Acquire multiple reference audios suitable for the target scene from a preset audio library; A reference audio whose duration is equal to the target duration is determined from the multiple reference audios.

5. The method according to claim 4, characterized in that The target duration is obtained according to the total duration of the content.

6. The method according to any one of claims 1 to 5, characterized in that: The dubbing the content according to the content and the reference audio includes: According to the duration difference between the total duration of the content and the duration of the reference audio, the reference audio is padded with zeros to obtain an intermediate audio having a duration equal to the total duration of the content; Based on the content and the intermediate audio, a recommended audio for dubbing the content is generated, and the recommended audio is synthesized with the content.

7. The method according to claim 6, characterized in that The step of generating a recommended audio for dubbing the content according to the content and the intermediate audio comprises: Understanding the content to obtain text information describing the content; Obtaining dubbing characteristics according to the text information, wherein the dubbing characteristics include at least one of the following: emotional information of the recommended audio, style information of the recommended audio, and the number of beats of the recommended audio; The recommended audio for dubbing the content is generated based on the dubbing characteristics and the intermediate audio.

8. A content dubbing device, characterized in that: The device comprises: A response module, configured to obtain reference audio adapted to the content in response to a dubbing operation on the content, wherein the content includes an image and / or a video, and the reference audio is used to generate an ending background audio played before the content ends; The dubbing module is configured to dub the content according to the content and the reference audio.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the steps of any one of the methods described in claims 1 to 7 when executing the instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images