Video data processing method, electronic device, and readable medium

Abstract

The present application provides a video data processing method, an electronic device, and a readable medium. In the video data processing method, the electronic device displays a first interface, the first interface displays a cover thumbnail of a first video and a first button, and the first button can be understood as a trigger button that the electronic device generates a short video by using a one-click video editing function; and when a user clicks the first button on the first interface, the electronic device can play a second video, the second video comprises at least part of audio and background music of the first video, and at least part of audio in a raw video can be reserved in the short video generated by the user using the one-click video editing function. When the user clicks the first button, the electronic device can obtain the second video, the second video comprises at least part of audio and background music of the first video, so that by means of a one-click operation of the user, the electronic device can automatically mix the audio and the background music of the first video.

Description

Video data processing method, electronic device and readable medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on June 29, 2023, with application number 202310794342.4 and invention name “Video data processing method, electronic device and readable medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of data processing technology, and in particular to a method for processing video data, an electronic device, a computer program product, and a computer-readable storage medium. Background Art

[0003] The one-click movie-making function means that after the user selects photos or videos, he clicks the one-click movie-making button, and the electronic device automatically combines the photos or videos selected by the user with the special effects, background music, etc. configured by the electronic device to obtain a short video made with one click.

[0004] However, the short video generated by the electronic device using the one-click video function does not retain the original sound of the video selected by the user, resulting in the loss of the audio in the video selected by the user.

[0005] Summary of the Invention

[0006] The present application provides a method for processing video data, an electronic device, a computer program product, and a computer-readable storage medium, the purpose of which is to achieve that a short video generated by a one-click blockbuster function can retain at least part of the audio in the original video.

[0007] In order to achieve the above objectives, this application provides the following technical solutions:

[0008] In a first aspect, the present application provides a method for processing video data, including: an electronic device displays a first interface, the first interface displays a cover thumbnail and a first button of a first video; the electronic device displays a second interface in response to a user clicking the first button, the second interface being a playback interface of a second video, the second video including at least part of the audio and background music of the first video.

[0009] As can be seen from the above content: the electronic device displays a first interface, which displays a cover thumbnail of the first video and a first button. The first button can be understood as the trigger button for the electronic device to generate a short video using the one-click movie function. After the user clicks the first button on the first interface, the electronic device can play a second video, which includes at least part of the audio and background music of the first video. This allows the short video generated by the user using the one-click movie function to retain at least part of the audio of the original video. After the user clicks the first button, the electronic device can obtain a second video, which includes at least part of the audio and background music of the first video. It also allows the user to automatically mix the audio and background music of the first video with a single click.

[0010] In a possible implementation, before displaying the second interface, the method further includes: the electronic device displays a third interface, where the third interface is a buffer interface for generating the second video.

[0011] In one possible embodiment, before displaying the second interface, it also includes: the electronic device configures background music, and the background music matches the first video; the electronic device recognizes that the audio of the first video includes human voice, and then fuses the audio of the first video and the background music to obtain fused audio; the electronic device fuses the image of the first video and the fused audio to obtain the second video.

[0012] In this possible implementation, the electronic device configures background music that matches the style of the first video for the first video. This ensures that, when the electronic device fuses the audio and background music of the first video, the styles of the audio and background music are unified, avoiding any inconsistency. If the electronic device recognizes that the audio of the first video retains a human voice, it fuses the audio and background music of the first video, ensuring that the second video retains the valid audio of the first video and preventing invalid audio from being retained in the second video.

[0013] In one possible embodiment, the electronic device fuses the audio and background music of the first video to obtain the fused audio, and further includes: the electronic device performs noise reduction processing on the audio of the first video to obtain the noise-reduced audio; wherein, the electronic device fuses the audio and background music of the first video to obtain the fused audio, including: the electronic device fuses the noise-reduced audio of the first video with the background music to obtain the fused audio.

[0014] In this possible implementation, before the electronic device fuses the audio and background music of the first video, it first performs noise reduction processing on the music of the first video, so as to obtain a clean human voice in the first video and ensure the effect after the audio and background music of the first video are fused.

[0015] In one possible implementation, the electronic device recognizes that the audio of the first video includes human voice, including: the electronic device performs sound event recognition on the audio of the first video to obtain a sound event recognition result; and the electronic device determines that the sound event recognition result includes a human voice event.

[0016] In one possible implementation, the acoustic event recognition includes speech recognition.

[0017] In one possible embodiment, after the electronic device configures the background music, it also includes: the electronic device recognizes that the audio of the first video includes key audio, then fuses the audio of the first video and the background music to obtain fused audio; the electronic device fuses the image of the first video and the fused audio to obtain the second video.

[0018] In this possible implementation manner, the audio of the first video includes key audio, and the audio of the first video and the background music are integrated to ensure that the key audio of the first video is retained in the second video.

[0019] In one possible embodiment, the process of the electronic device recognizing that the audio of the first video includes key audio also includes: the electronic device extracts features from the audio of the first video to obtain long-time audio features, and performs scene recognition on the long-time audio features to obtain scene recognition results, and the scene recognition results indicate that the first video belongs to the first scene; the electronic device recognizes that the audio of the first video includes key audio, including: the electronic device recognizes that the audio of the first video includes key audio of the first scene.

[0020] In one possible implementation, the electronic device is configured with multiple scenes, and the electronic device recognizes that the audio of the first video includes key audio, including: the electronic device recognizes the key audio belonging to the first scene in the audio of the first video.

[0021] In one possible implementation, the electronic device recognizes that the audio of the first video includes key audio, including: the electronic device performs sound event recognition on the audio of the first video to obtain a sound event recognition result; and the electronic device determines that the sound event recognition result includes the sound event of the key audio.

[0022] In one possible implementation, the key audio at least includes human voice, or the key audio does not include human voice.

[0023] In one possible implementation, fusing the audio and background music of the first video to obtain fused audio includes: fusing the audio and background music of the first video in a corresponding time period according to a fusion ratio of audio amplitudes to obtain fused audio.

[0024] In one possible implementation, the audio amplitude fusion ratio is used to indicate that the amplitude of the audio of the first video in the fused audio is greater than the amplitude of the background music. In this possible implementation, the fusion ratio of the audio of the first video and the background music indicates that the amplitude of the audio of the first video is greater than the amplitude of the background music, thereby making the audio of the first video more prominent relative to the background music in the second video.

[0025] In one possible embodiment, the audio and background music of the first video are fused to obtain fused audio, including: fusing the audio and background music of the first video according to the principle that the energy of the audio of the first video is substantially the same as the energy of the background music, to obtain fused audio, so that in the second video, the volume of the background music and the audio of the first video are maintained at substantially the same level.

[0026] In one possible embodiment, the audio and background music of the first video are fused according to the principle that the energy of the audio of the first video is substantially the same as the energy of the background music to obtain fused audio, including: taking the product of the audio energy of the first video and a first coefficient as the energy of the audio of the first video in the fused audio, taking the product of the background music energy and a second coefficient as the energy of the background music in the fused audio, the first coefficient being the ratio of the background music energy to the sum of the audio energy of the first video and the energy of the background music, and the second coefficient being the ratio of the audio energy of the first video to the sum of the audio energy of the first video and the energy of the background music.

[0027] In one possible embodiment, the audio and background music of the first video are fused to obtain fused audio, including: the key audio includes at least human voice, and the audio and background music of the first video are fused according to a first ratio of audio amplitude in a corresponding time period to obtain fused audio; or, the key audio does not include human voice, and the audio and background music of the first video are fused according to a second ratio of audio amplitude in a corresponding time period to obtain fused audio; wherein the first ratio of audio amplitude and the second ratio of audio amplitude are both used to indicate the proportional relationship between the audio and background music of the first video; the first ratio of audio amplitude is greater than the second ratio of audio amplitude, so that the audio of the first video includes human voice and the human voice can be more prominent.

[0028] In a second aspect, the present application provides an electronic device comprising: one or more processors, a memory, and a display screen; the memory and the display screen are coupled to the one or more processors, the memory is used to store a computer program, the computer program includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the video data processing method as described in the first aspect and any one of its possible implementations.

[0029] In a third aspect, the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed, it is specifically used to implement the video data processing method as described in the first aspect and any one of its possible implementation methods.

[0030] In a fourth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method for processing video data as described in the first aspect and any one of its possible implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a diagram showing the process of generating a short video with one click provided by an embodiment of the present application;

[0032] FIG2 is a hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0033] FIG3 is a flow chart of a method for processing video data provided by an embodiment of the present application;

[0034] FIG4 is a diagram showing scene recognition using a neural network model according to an embodiment of the present application;

[0035] FIG5 is a diagram showing a scene and a corresponding target acoustic event provided by an embodiment of the present application;

[0036] FIG6 is a flowchart of a method for processing video data provided by another embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more; "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0038] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0039] The "multiple" involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first" and "second" are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance or order.

[0040] The one-click movie feature, or one-click blockbuster feature, allows users to select photos or videos and click the one-click movie button (or the one-click blockbuster button). The electronic device then automatically combines the selected photos or videos with the configured special effects and background music to create a short video. Special effects are those that can be added to video frames to create special effects supported by the source material, such as animation effects like snowflakes and fireworks, as well as filters, stickers, and frames. In some embodiments, special effects may also be referred to as styles or style themes.

[0041] However, the short videos generated by the electronic device using the one-click video function do not retain the original sound of the user-selected video, but only include background music, resulting in the loss of audio in the user-selected video.

[0042] Based on this, an embodiment of the present application provides a method for processing video data, which synthesizes data such as images, videos, background music and special effects, and can retain at least part of the audio in the video in the process of obtaining a one-click short video, and can also highlight specific audio in the video in the one-click short video.

[0043] 1 , taking a mobile phone as an example, the process of generating a short video using the one-click blockbuster function provided in an embodiment of the present application is introduced.

[0044] The user opens the gallery application, and the interface of the gallery application displays multiple videos and images saved locally. The user can select any image and / or video displayed on the interface of the gallery application to clearly generate the material for the one-click short video. Exemplarily, the interface of the gallery application is shown in Figure 1 (a), including the cover thumbnail and image of the video. The user long presses the cover thumbnail 101, image 102 and image 103 of the video in the interface shown in Figure 1 (a) to select video 101, image 102 and image 103 as the material for the one-click short video. Figure 1 (b) shows the interface of the user selecting video 101, image 102 and image 103. The interface also displays a one-click blockbuster button 104, which is used to trigger the mobile phone to generate a one-click short video using video 101, image 102, image 103 and special effects.

[0045] As shown in Figure 1(b), the user clicks the "One-Click Movie" button 104. The phone then matches the background music based on the style of video 101, image 102, and image 103. It can also match special effects, synthesizing video 101, image 102, and image 103, along with the phone's configured special effects and background music, to create a "One-Click Movie" short video. Figure 1(c) shows the waiting screen (or buffering interface) for the phone to generate the "One-Click Movie" short video. After the phone generates the "One-Click Movie" short video, it can automatically play the short video as shown in Figures 1(d) and (e).

[0046] In some embodiments, the mobile phone may not display the screen shown in (c) of Figure 1. After the user clicks the one-click blockbuster button 104, the mobile phone generates a one-click short video and automatically plays the short video as shown in (d) and (e) of Figure 1.

[0047] The one-click short video generated by the mobile phone retains part of the audio in video 101. In some embodiments, the one-click short video can also highlight the human voice in video 101.

[0048] In some embodiments, users can select at least one video in the gallery application interface, and the phone will combine the video, special effects, and background music to create a short video with one click. In other embodiments, users can also select at least one image in the gallery application interface, and the phone will combine the image, special effects, and background music to create a short video with one click.

[0049] The video data processing method provided in the embodiment of the present application, in the process of synthesizing the image, video selected by the user and the background music and special effects configured by the mobile phone, the mobile phone needs to fuse the audio and background music in the video. Therefore, in order to facilitate the explanation of the process of fusing the audio and background music in the video, the following content of the embodiment of the present application is introduced using the example of a user selecting an image and at least one video in the interface of the gallery application.

[0050] The video data processing method provided in the embodiments of the present application can also be applied to electronic devices such as tablet computers, personal digital assistants (PDAs), desktop computers, laptop computers, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices.

[0051] Taking a mobile phone as an example, Figure 2 shows an example of the composition of an electronic device provided in an embodiment of the present application. As shown in Figure 2, the electronic device 100 may include a processor 110, an internal memory 120, a camera 130, a display 140, a mobile communication module 150, a wireless communication module 160, an audio module 170, and a sensor module 180.

[0052] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a smart sensor hub, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0054] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0055] The internal memory 120 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 120, and / or the instructions stored in the memory provided in the processor.

[0056] In some embodiments, the internal memory 120 stores instructions for executing methods for processing video data. The processor 110 can execute the instructions stored in the internal memory 120 to synthesize data such as images, video, background music, and special effects to generate a one-click short video. Furthermore, during the generation of the one-click short video, at least part of the audio in the video can be retained, and specific audio in the video can be highlighted in the one-click short video.

[0057] The electronic device implements display functionality through a GPU, display screen 140, and an application processor. A GPU is a microprocessor for image processing that connects display screen 140 and the application processor. The GPU performs mathematical and geometric calculations for image rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0058] Display screen 140 is used to display images, video interfaces, and the like. Display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device can include one or N display screens 140, where N is a positive integer greater than one.

[0059] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0060] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0061] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 110 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0062] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 150 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0063] The electronic device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0064] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0065] The speaker 170A, also called a "speaker," is used to convert audio electrical signals into sound signals. The electronic device can listen to music or make hands-free calls through the speaker 170A.

[0066] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be heard by placing the receiver 170B close to the human ear.

[0067] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, and identification of sound sources, and realize directional recording function, etc.

[0068] The headphone jack 170D is used to connect a wired headphone and can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0069] In the sensor module 180, the pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 140. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a device including at least two parallel plates with conductive material. When force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 140, the electronic device detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device can also calculate the position of the touch based on the detection signal of the pressure sensor 180A.

[0070] Touch sensor 180B, also known as a "touch-sensitive device," can be disposed on display screen 140. Touch sensor 180B and display screen 140 form a touch screen, also known as a "touch screen." Touch sensor 180B is configured to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via display screen 140. In other embodiments, touch sensor 180B can also be disposed on the surface of the electronic device, at a location different from that of display screen 140.

[0071] The acceleration sensor 180C can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, it can detect the magnitude and direction of gravity and can also be used to identify the posture of the electronic device.

[0072] The gyro sensor 180D can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (ie, x, y, and z axes) can be determined by the gyro sensor 180D.

[0073] The technical solutions involved in the following embodiments can all be implemented in electronic devices with the above hardware architecture.

[0074] 3 , the following describes a method for processing video data by an electronic device to generate a short video in one click.

[0075] As shown in FIG3 , an embodiment of the present application provides a method for processing video data, including:

[0076] S301: The electronic device obtains the video and image selected by the user.

[0077] The user opens the gallery application, and the electronic device displays the interface of the gallery application as shown in Figure 1 (a), which includes multiple videos and multiple images. The user long presses video 101, image 102, and image 102 to select the video and image.

[0078] In response to the user's long press operation on the video and image, the electronic device obtains the video and image selected by the user. In some embodiments, the electronic device obtaining the video and image may refer to: obtaining video stream data of the video and image data of the image.

[0079] S302: The electronic device configures background music that matches the style of the image and video.

[0080] The style of an image can also refer to the subject matter of the image or the category to which the image belongs. The style of a video can also refer to the subject matter of the video or the category to which the video belongs. For example, images 102 and 103 shown in Figure 1 (a) are of a football game scene, and the styles of images 102 and 103 are sports styles. Similarly, the style of video 101 is also sports style. As another example, if the images and videos selected by the user include dietary elements such as dishes and food, the images and videos are of a dietary style.

[0081] In some embodiments, the background music configured for the electronic device may be one or more.

[0082] In some embodiments, the background music configured by the electronic device to match the style of the image and video may refer to the rhythm of the background music and the style matching of the image and video. In one possible implementation, if the style of the image and video belongs to the lively and bustling series, background music with a strong sense of movement is configured; if the style of the image and video belongs to the quiet series, relatively soothing background music is configured. For example, the video 101, image 102, and image 103 shown in (a) of Figure 1 are of sports style, belonging to the lively and bustling series, and the background music configured by the electronic device is a background style with a strong sense of movement. For another example, the image and video selected by the user are of food style, belonging to the quiet series, and the background music configured by the electronic device is relatively soothing background music.

[0083] In some embodiments, the electronic device identifies the style of the image and the style of the video separately and matches background music that matches both the image and video styles. In other embodiments, the electronic device may also match background music that matches the style of the image or the style of the video. The background music matched by the electronic device in this step can be used as the background music for the one-click short video.

[0084] In some embodiments, the electronic device can identify key features of an image and use them to derive the image's style. In other embodiments, the electronic device can capture multiple frames of a video, identify key features in each of the frames, and use the key features of one frame to derive the image's style. The electronic device can then comprehensively assess the style of the multiple frames to derive the video's style. In one possible implementation, the electronic device is configured with an image recognition model that can identify key features of an image and derive the image's style based on the key features.

[0085] In some embodiments, in addition to configuring background music that matches the style of the image and video, the electronic device may also configure special effects that match the style of the image and video.

[0086] In some embodiments, the electronic device has multiple special effects templates, each of which may include special effects content such as multiple audio effects, video effects, and other special effects, and also include descriptive information such as the time to add each special effect and the position of the special effect in the video frame. Each special effects template includes special effects content and description information indicated by the template information of the special effects template. The template information of each special effects template can be saved in a public resource path in json format. Each special effects template configured by the electronic device corresponds to an ID, which can uniquely indicate the special effects template, and the template information of the special effects template can be determined by the ID of the special effects template. Special effects that match the styles of images and videos configured by the electronic device can refer to matching special effects templates.

[0087] S303: The electronic device obtains a video.

[0088] The electronic device obtains a video from the videos obtained in step S301 and performs the following processing from step S304 to step S310. In some embodiments, if the electronic device obtains an image obtained in step S301, the electronic device skips the image and does not process the image.

[0089] In some embodiments, the electronic device obtaining a video may refer to obtaining video stream data of a video.

[0090] S304: The electronic device separates the image and audio of the video.

[0091] In some embodiments, the electronic device may utilize the multi-output mode of FFmpeg to separate the video from the audio and video, and then the electronic device may extract the object elements and timeline of the separated images and audio.

[0092] The video stream data of a video obtained by the electronic device in step S303 includes an image frame sequence and an audio stream, where the image frame sequence includes multiple images. The electronic device needs to perform steps S305 and S306 on the audio stream in the video. Therefore, the electronic device needs to first separate the image frame sequence and the audio stream from the video stream data.

[0093] S305: The electronic device performs scene recognition on the audio separated from the video to obtain a scene recognition result.

[0094] In some embodiments, the electronic device may be configured with a neural network model, referred to as a scene recognition network model, which is used to perform scene recognition on the audio stream. The scene recognition network model may be trained, as shown in FIG4(a) .

[0095] As shown in Figure 4(a), feature extraction is performed on the sample audio to obtain audio features. To improve the training effect of the scene recognition network model, the audio features can be subjected to data enhancement processing. The enhanced audio features are input into the neural network model, and the scene recognition network model is trained based on the loss function. The trained scene recognition network model can identify the scene to which the input audio belongs.

[0096] In some embodiments, as shown in (b) of FIG4 , after the electronic device executes step S304, it extracts features from the audio stream to obtain audio features, and calls a trained scene recognition network model to process the audio features to obtain the scene to which the audio belongs. The scene to which the audio belongs is the scene recognition result.

[0097] In some embodiments, the electronic device extracts features from the audio stream, including extracting long-term audio features and short-term audio features from the audio stream. The scene recognition network model processes the long-term audio features and short-term audio features to obtain the scene to which the audio stream belongs. Long-term audio features can be understood as extracting features of audio from a longer time period of the audio stream, such as a segment of audio of more than ten seconds in the audio stream; short-term audio features can be understood as extracting features of audio from a shorter time period or a certain moment in the audio stream, such as a segment of audio of a few seconds or one second of audio in the audio stream.

[0098] In one possible implementation, the electronic device may be configured with a scene recognition network model. This scene recognition network model is not limited to recognizing a single scene. For example, the scene recognition network model may identify whether an audio stream includes a kitchen, a park, a square, a conference room, and so on. Using a single scene recognition network model, the electronic device can identify different scenes, which has the advantage of a simple processing flow.

[0099] The electronic device may also use other methods to perform scene recognition on the audio to obtain a scene recognition result, and the embodiments of the present application are not limited to this.

[0100] S306: The electronic device performs acoustic event recognition on the audio separated from the video to obtain an acoustic event recognition result.

[0101] An acoustic event can be understood as a specific audio, that is, a specific sound, such as a human voice, chewing sounds, applause, and fireworks. In some embodiments, the scope of acoustic events recognized by the electronic device can be configured.

[0102] In some embodiments, electronic devices can be configured with a neural network model, referred to as an acoustic event recognition network model, for use in identifying acoustic events in audio streams. This acoustic event recognition network model can also be trained using the same training methods as the scene recognition network model, and will not be further described here.

[0103] In some embodiments, after executing step S304, the electronic device extracts features from the audio stream to obtain audio features, and calls a trained acoustic event recognition network model to process the audio features to obtain acoustic events included in the audio stream, that is, acoustic event recognition results.

[0104] In some embodiments, the electronic device extracts features from an audio stream, including extracting short-term audio features from the audio stream. The scene recognition network model processes the short-term audio features to obtain acoustic events included in the audio stream. Short-term audio features can be understood as extracting features of audio from a short period of time or a specific moment in the audio stream, such as a few seconds of audio or one second of audio from the audio stream.

[0105] In one possible implementation, an electronic device may be configured with an acoustic event recognition network model. This acoustic event recognition network model is not limited to recognizing a single acoustic event. For example, the acoustic event recognition network model may identify whether an audio stream contains human voices, chewing sounds, applause, or fireworks. Using a single acoustic event recognition network model to identify different acoustic events simplifies the processing flow.

[0106] The electronic device may also use other methods to perform acoustic event recognition on audio to obtain acoustic event recognition results, and the embodiments of the present application are not limited to this.

[0107] In other embodiments, the electronic device can perform acoustic event recognition on the audio separated from the video to obtain an acoustic event recognition result, and then determine the scene to which the audio belongs based on the acoustic event recognition result. A scene is configured with key audio, i.e., specific audio. The electronic device can identify the key audio in the audio and determine the scene to which the audio belongs based on the scene corresponding to the key audio.

[0108] S307: The electronic device identifies whether the acoustic event recognition result includes the target acoustic event based on the scene recognition result.

[0109] In some embodiments, the electronic device may configure a target sound event corresponding to a scene. After the electronic device executes steps S305 and S306, it may use the scene recognition results to clarify the scene to which the audio belongs and determine whether the sound event recognition results include the target sound event corresponding to the scene to which the audio belongs.

[0110] In some embodiments, the target sound event can be understood as a key sound or an indicative sound in a scene. The target sound event can indicate the scene to which the audio belongs.

[0111] In some embodiments, human voice events are target sound events in all scenarios. Different target sound events can be set for specific scenarios, such as chewing in a kitchen, applause in a conference room, and fireworks in a square.

[0112] If the electronic device recognizes that the acoustic event recognition result includes the target acoustic event, step S308 is executed. If the electronic device recognizes that the acoustic event recognition result does not include the target acoustic event, the process returns to step S303 and the electronic device obtains the next video.

[0113] In some embodiments, after the electronic device identifies that the acoustic event recognition result includes a target acoustic event, it may also record the target acoustic event included in the acoustic event recognition result. In one possible implementation, the electronic device records the target acoustic event included in the acoustic event recognition result by setting a tag for the target acoustic event and recording the tag.

[0114] For example, as shown in Figure 5(a), the scene recognition network model identifies scenes including: kitchen, conference room, square, and park. The acoustic events identified by the acoustic event recognition network model include: human voice events, chewing sound events, applause events, fireworks sound events, and bird song sound events. The target acoustic events corresponding to the kitchen scene include: human voice events and chewing sound events; the target acoustic events corresponding to the conference room include: human voice events and applause events; the target acoustic events corresponding to the square scene include: human voice events and fireworks sound events; and the target acoustic events corresponding to the park scene include: human voice events and bird song sound events.

[0115] As shown in Figure 5(b), the electronic device performs scene classification and acoustic event recognition on the input audio. Scene classification involves identifying whether the audio belongs to a kitchen, conference room, square, or park scene. Acoustic event recognition involves identifying whether the audio contains human voice events, chewing sounds, applause events, fireworks sounds, and bird sounds. The electronic device then combines the scene recognition results to determine whether the acoustic event recognition results include the target acoustic event.

[0116] S308: The electronic device determines whether the target sound event includes only human voice.

[0117] After obtaining the target acoustic event included in the acoustic event recognition result, the electronic device may further identify whether the target acoustic event includes only human voices and does not include other acoustic events.

[0118] In some embodiments, the electronic device identifies the label of the recorded target sound event. If only the label of the human voice event is identified, it is determined that the target sound event only includes the human voice. If the label of the human voice event and the labels of other sound events are identified, it is determined that the target sound event does not only include the human voice.

[0119] If the electronic device determines that the target sound event includes only human voice, the process goes to step S309 . If the electronic device determines that the target sound event does not include only human voice, the process goes back to step S310 .

[0120] S309: The electronic device performs noise reduction processing on the audio separated from the video.

[0121] In some embodiments, the electronic device may use Wiener filtering or a deep neural network model to perform noise reduction on the audio. The specific implementation process of the noise reduction process is not described here. Of course, the electronic device may also use other noise reduction methods to perform noise reduction on the audio.

[0122] Audio that only includes human voice events means the audio consists solely of human speech. To preserve the clean human voice, the electronic device performs noise reduction on the audio. Typically, noise reduction on audio refers to removing noise other than human voices. In scenarios where the target sound events consist solely of human voices, noise reduction on the audio by the electronic device can preserve other target sounds more completely in the one-click short video.

[0123] In some embodiments, step S308 may not be performed, and the electronic device may perform noise reduction processing on the audio when the event recognition result includes only the human voice event, or when there are other scenarios where the human voice event is also included.

[0124] In some other embodiments, step S308 and step S309 may not be performed.

[0125] S310: The electronic device marks the audio as audio to be fused.

[0126] The electronic device determines in step S307 that the audio separated from the video includes the target sound event, thereby determining that the audio needs to be retained in the one-click short video, and the electronic device can mark the audio as audio to be fused.

[0127] In some embodiments, the electronic device can mark the audio as audio to be fused by configuring a parameter for the audio. The parameter can be the start and end time period of the audio. The parameter can be used with the audio to perform the fusion action in step S311 below. For example, if the parameter configured for a segment of audio is 00:01:10 to 00:01:30, the parameter indicates that the audio has sound in the time period of 00:01:30 to 00:01:30 and can be fused with background music.

[0128] S311. The electronic device fuses the audio to be fused and the background music in the acquired video in a corresponding time period to obtain fused audio.

[0129] The electronic device, through steps S303 to S310, selects the audio to be merged from the audio separated from the user-selected video. The electronic device then merges the audio to be merged with the background music configured in step S302. In some embodiments, the audio to be merged and the background music are audio streams configured with timestamps. Therefore, the audio to be merged and the background music are merged in corresponding time periods.

[0130] In some embodiments, the fusion of the audio to be fused and the background music in the corresponding time period can be understood as the fusion of the background music in the time period indicated by the parameters of the audio to be fused with the audio to be fused.

[0131] In some embodiments, the audio to be fused and the background music may be fused at a certain ratio. It is understood that merging the audio to be fused and the background music at a certain ratio means that the amplitudes of the audio to be fused and the background music are fused at a certain ratio during corresponding time periods. This ratio can be any value.

[0132] In one possible implementation, the fusion ratio of the audio to be fused is greater than the fusion ratio of the background music, thereby highlighting the audio to be fused in the one-click short video. In some embodiments, the target sound events included in the audio to be fused include human voice events, and the degree to which the fusion ratio of the audio to be fused is greater than the fusion ratio of the background music is increased, thereby significantly highlighting the human voice in the one-click short video.

[0133] For example, if the target sound events in the audio to be fused include human voice events, the audio to be fused and the background music can be mixed in a 7:3 ratio; if the target sound events in the audio to be fused include human voice events and other target sound events, the audio to be fused and the background music can be mixed in a 6:4 ratio. The above fusion ratios are not specifically limited and can be other ratios.

[0134] In other embodiments, the audio to be fused and the background music are dynamically fused according to the amplitude (or energy) of the audio to be fused and the background music.

[0135] In one possible implementation, the principle of fusing the audio to be fused and the background music is that the energy of the audio to be fused and the energy of the background music are kept at a comparable level, that is, in the fused audio, the energy of the audio to be fused and the energy of the background music are substantially the same. The average energy of the audio to be fused is denoted as Pow(original sound), and the average energy of the background music is denoted as Pow(background). The fusion coefficient of the audio to be fused can be denoted as Pow(background) / (Pow(background)+Pow(original sound)), and the fusion coefficient of the background music can be denoted as Pow(original sound) / (Pow(background)+Pow(original sound)). The product of the amplitude of the audio to be fused and the fusion coefficient of the audio to be fused is taken as the amplitude of the audio to be fused in the fused audio, and the product of the amplitude of the background music and the fusion coefficient of the background music is taken as the amplitude of the background music in the fused audio.

[0136] It should be noted that, for scenes with multiple audios to be fused, the fusion ratio of each audio to be fused and the background music in the corresponding time period can refer to the method provided in the above content.

[0137] In some embodiments, if there are multiple audio tracks to be fused, the electronic device may combine the multiple audio tracks into one audio track in chronological order and then combine it with the background music. If there are multiple background music tracks, the background music tracks may also be combined into one background music track first.

[0138] In other embodiments, there are multiple audios to be fused, and the electronic device may also fuse each audio to be fused with the background music, and then piece the fused audios together in chronological order to form a complete fused audio segment.

[0139] S312. The electronic device fuses the image separated from the fused audio and video and the acquired image to obtain a one-click short video.

[0140] In some embodiments, the electronic device may first fuse the images separated from the fused audio and video to obtain a video, and then further fuse the video with the acquired images to obtain a one-click short video. The electronic device may edit the acquired images after the video, and the electronic device may edit the images after the video in descending order of their storage time.

[0141] In a possible implementation, when the electronic device fuses the video and the acquired image, the video may be edited to retain some video clips.

[0142] In some embodiments, the duration of the background music is greater than the duration of the video selected by the user in step S301, and if the user has selected multiple videos, the duration of the background music is greater than the total duration of the multiple videos selected by the user. In this way, the one-click short video generated by the electronic device also includes background music for the image selected by the user.

[0143] In some embodiments, the electronic device configures special effects that match the style of the image and video selected by the user. The special effects configured by the electronic device can also be added to the image separated from the video and the acquired image to present special effects.

[0144] Another embodiment of the present application also provides a method for generating a short video by one-click. The following, in conjunction with Figure 6, introduces the process of generating a short video by one-click using a method for processing video data by an electronic device.

[0145] As shown in FIG6 , an embodiment of the present application provides a method for processing video data, including:

[0146] S601: The electronic device obtains the video and image selected by the user.

[0147] The content of step S601 can refer to the content of the aforementioned step S301 and will not be repeated here.

[0148] S602: The electronic device matches the image and the background music with the style of the video.

[0149] The content of step S602 can refer to the content of the aforementioned step S302 and will not be repeated here.

[0150] S603: The electronic device obtains a video.

[0151] The content of step S603 can refer to the content of the aforementioned step S303 and will not be repeated here.

[0152] S604: The electronic device separates the image and audio of the video.

[0153] The content of step S604 can refer to the content of the aforementioned step S304 and will not be repeated here.

[0154] S605: The electronic device performs speech recognition on the audio separated from the video to obtain a speech recognition result.

[0155] As a special sound in audio, human voice usually needs to be retained in a one-click short video. Therefore, for a video, the electronic device separates the audio in the video and then performs voice recognition on the audio to determine whether the audio includes a voice signal, which is usually understood to be a human voice.

[0156] In some embodiments, the electronic device uses a cepstrum method to calculate the fundamental frequency. If the electronic device determines that the fundamental frequency exists, it indicates that the audio includes a speech signal, that is, the speech recognition result is that the audio includes a speech signal.

[0157] In the field of speech recognition, audio features can be extracted from a segment of audio. These features can then be used to identify speech or non-speech signals to determine whether the segment contains a speech signal. Determining the fundamental frequency using the cepstrum method can be considered a method for extracting audio features. Of course, electronic devices can also use other methods for feature extraction.

[0158] In other embodiments, the electronic device uses a neural network model to perform speech recognition and obtain speech recognition results. In one possible implementation, the electronic device uses a recurrent neural network model to perform speech endpoint detection (or voice activity detection) on audio. The recurrent neural network model can use short-term zero-crossing rate and short-term energy as endpoint detection features, and then use the endpoint detection features to identify whether the audio contains a speech signal, i.e., a human voice.

[0159] S606: The electronic device determines whether the speech recognition result includes a human voice.

[0160] The electronic device determines whether the voice recognition result includes a human voice. This can be understood as the electronic device determining whether the voice recognition result indicates that the audio includes a human voice.

[0161] If the electronic device determines that the voice recognition result includes a human voice, step S607 is executed; if the electronic device determines that the voice recognition result does not include a human voice, the process returns to step S603 and the electronic device obtains the next video.

[0162] The electronic device determines that the voice recognition result indicates that the audio does not include human voice, which means that the audio can be retained in the one-click short video. Therefore, the electronic device obtains the next video selected by the user through step S603.

[0163] S607: The electronic device performs noise reduction processing on the audio separated from the video.

[0164] In some embodiments, the electronic device may use Wiener filtering or a deep neural network model to perform noise reduction on the audio. The specific implementation process of the noise reduction process is not described here. Of course, the electronic device may also use other noise reduction methods to perform noise reduction on the audio.

[0165] The speech recognition result indicates that the audio contains human voices. To obtain a clean human voice, the electronic device performs noise reduction on the audio. Generally, noise reduction on audio refers to removing noise other than human voices.

[0166] S608: The electronic device marks the audio as audio to be fused.

[0167] The content of step S608 can refer to the content of the aforementioned step S310 and will not be repeated here.

[0168] S609: The electronic device fuses the video to be fused and the background music in the acquired video in a corresponding time period to obtain fused audio.

[0169] The content of step S609 can refer to the content of the aforementioned step S311 and will not be repeated here.

[0170] S610: The electronic device fuses the image separated from the fused audio and video and the acquired image to obtain a one-click short video.

[0171] The content of step S610 can refer to the content of the aforementioned step S312 and will not be repeated here.

[0172] Figure 1 shows a scenario where a user opens a gallery application and uses the videos and images stored in the gallery application to generate a one-click short video. In some embodiments, the one-click short video function can also be applied to video shooting scenarios.

[0173] When a user opens a camera application and uses the one-click short video function to shoot a short video, the electronic device can also realize the one-click short video, which can at least retain part of the audio collected by the microphone during the video shooting process and can also highlight specific audio.

[0174] In some embodiments, a method for generating a short video in a video shooting scenario includes:

[0175] The electronic device starts a camera application to shoot a video and obtain video data, wherein the electronic device collects an image of the video data through a camera and collects an audio of the video data through a microphone.

[0176] The electronic device configures background music that matches the style of the video. In some embodiments, the electronic device may also configure special effects that match the video. In some embodiments, the electronic device may configure background music and / or special effects for the video in response to a user specifying background music and / or special effects. The implementation of the electronic device configuring background music that matches the style of the video can be found in step S302 of the aforementioned embodiment and will not be further described here.

[0177] When an electronic device is shooting a video, the image and audio in the video can be understood as being collected and stored separately. The image and audio are only merged into the video after the user operates the electronic device screen to input a save video operation. Therefore, the electronic device can perform scene recognition on the audio captured by the microphone to obtain a scene recognition result, and also perform acoustic event recognition to obtain an acoustic event recognition result. The implementation methods of scene recognition and acoustic event recognition can be found in the contents of steps S305 and S306 in the aforementioned embodiment, and will not be repeated here.

[0178] The electronic device combines the scene recognition results and identifies that the sound event recognition results include the target sound event, then fuses the audio collected by the microphone and the background music in the corresponding time period to obtain the fused audio. The specific implementation of this step can be found in the content of step S311 in the above embodiment and will not be repeated here.

[0179] The electronic device then fuses the fused audio with the image captured by the camera to produce a short video that can be produced with one click.

[0180] In some embodiments, after the electronic device identifies that the sound event recognition result includes the target sound event, it may further determine whether the target sound event only includes human voices. If so, noise reduction processing is performed on the audio. The specific implementation process of noise reduction processing can be found in the content of step S309 in the above embodiment and will not be repeated here.

[0181] In other embodiments, a method for generating a short video in a video shooting scenario using one click includes:

[0182] The electronic device starts a camera application to shoot a video and obtain video data, wherein the electronic device collects an image of the video data through a camera and collects an audio of the video data through a microphone.

[0183] The electronic device configures background music that matches the style of the video. In some embodiments, the electronic device may also configure special effects that match the video. In some embodiments, the electronic device may configure background music and / or special effects for the video in response to a user specifying background music and / or special effects. The implementation of the electronic device configuring background music that matches the style of the video can be found in step S302 of the aforementioned embodiment and will not be further described here.

[0184] The electronic device performs speech recognition on the audio collected by the microphone to obtain a speech recognition result. The implementation of speech recognition can be found in step S605 in the aforementioned embodiment and will not be repeated here. In some embodiments, if the electronic device recognizes that the speech recognition result includes a human voice, it performs noise reduction processing on the audio. The specific implementation process of the noise reduction processing can be found in step S309 in the aforementioned embodiment and will not be repeated here.

[0185] When the electronic device determines that the speech recognition result includes a human voice, it fuses the audio collected by the microphone and the background music in the corresponding time period to obtain a fused audio. The specific implementation of this step can be found in the content of step S311 in the above embodiment and will not be repeated here.

[0186] The electronic device then fuses the fused audio with the image captured by the camera to produce a short video that can be produced with one click.

[0187] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a computer or a processor, the computer or processor executes one or more steps in any of the above methods.

[0188] The computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0189] Another embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer or processor, causes the computer or processor to execute one or more steps in any of the above methods.

Claims

1. A method for processing video data, characterized in that: include: The electronic device displays a first interface, wherein the first interface displays a cover thumbnail of the first video and a first button; In response to the user clicking the first button, the electronic device displays a second interface, where the second interface is a playback interface of a second video, and the second video includes at least part of the audio and background music of the first video.

2. The method for processing video data according to claim 1, characterized in that: Before displaying the second interface, the method further includes: The electronic device displays a third interface, where the third interface is a buffer interface for generating the second video.

3. The method for processing video data according to claim 1 or 2, characterized in that: Before displaying the second interface, the method further includes: The electronic device configures the background music, and the background music matches the first video; The electronic device recognizes that the audio of the first video includes human voice, and then fuses the audio of the first video and the background music to obtain fused audio; The electronic device fuses the image of the first video and the fused audio to obtain the second video.

4. The method for processing video data according to claim 3, characterized in that: Before the electronic device fuses the audio of the first video and the background music to obtain the fused audio, the electronic device further includes: The electronic device performs noise reduction processing on the audio of the first video to obtain noise-reduced audio; The electronic device fuses the audio of the first video and the background music to obtain fused audio, including: the electronic device fuses the noise-reduced audio of the first video and the background music to obtain fused audio.

5. The method for processing video data according to claim 3 or 4, characterized in that: The electronic device recognizes that the audio of the first video includes human voice, including: The electronic device performs acoustic event recognition on the audio of the first video to obtain an acoustic event recognition result; The electronic device determines that the acoustic event recognition result includes a human voice event.

6. The method for processing video data according to claim 5, characterized in that: The acoustic event recognition includes speech recognition.

7. The method for processing video data according to claim 3, characterized in that: After the electronic device configures the background music, the electronic device further includes: The electronic device recognizes that the audio of the first video includes key audio, and then fuses the audio of the first video and the background music to obtain fused audio; The electronic device fuses the image of the first video and the fused audio to obtain the second video.

8. The method for processing video data according to claim 7, characterized in that: The process in which the electronic device recognizes that the audio of the first video includes key audio further includes: The electronic device extracts features from the audio of the first video to obtain long-duration audio features, and performs scene recognition on the long-duration audio features to obtain a scene recognition result, wherein the scene recognition result indicates that the first video belongs to a first scene; The electronic device recognizes that the audio of the first video includes key audio, including: The electronic device recognizes that the audio of the first video includes key audio of the first scene.

9. The method for processing video data according to claim 7, characterized in that: The electronic device is configured with multiple scenarios, and the electronic device recognizes that the audio of the first video includes key audio, including: The electronic device recognizes key audio belonging to a first scene in the audio of the first video.

10. The method for processing video data according to claim 7, characterized in that: The electronic device recognizes that the audio of the first video includes key audio, including: The electronic device performs acoustic event recognition on the audio of the first video to obtain an acoustic event recognition result; The electronic device determines that the acoustic event recognition result includes an acoustic event of a key audio.

11. The method for processing video data according to claim 7, characterized in that: The key audio at least includes human voice, or the key audio does not include human voice.

12. The method for processing video data according to claim 3 or 7, characterized in that: The step of fusing the audio of the first video and the background music to obtain fused audio includes: The audio of the first video and the background music are fused in a corresponding time period according to a fusion ratio of the audio amplitude to obtain the fused audio.

13. The method for processing video data according to claim 12, characterized in that: The fusion ratio of the audio amplitude is used to indicate that the amplitude of the audio of the first video in the fused audio is greater than the amplitude of the background music.

14. The method for processing video data according to claim 3 or 7, characterized in that: The step of fusing the audio of the first video and the background music to obtain fused audio includes: The audio of the first video and the background music are fused according to the principle that the energy of the audio of the first video is substantially the same as the energy of the background music to obtain the fused audio.

15. The method for processing video data according to claim 14, characterized in that: The step of fusing the audio of the first video and the background music according to the principle that the energy of the audio of the first video is substantially the same as the energy of the background music to obtain the fused audio includes: The product of the audio energy of the first video and the first coefficient is used as the energy of the audio of the first video in the fused audio, and the product of the background music energy and the second coefficient is used as the energy of the background music in the fused audio, the first coefficient being the proportion of the background music energy in the sum of the audio energy of the first video and the energy of the background music, and the second coefficient being the proportion of the audio energy of the first video in the sum of the audio energy of the first video and the energy of the background music.

16. The method for processing video data according to claim 11, characterized in that: The step of fusing the audio of the first video and the background music to obtain fused audio includes: The key audio includes at least human voice, and the audio of the first video and the background music are fused according to a first ratio of audio amplitudes in a corresponding time period to obtain the fused audio; Alternatively, the key audio does not include human voice, and the audio of the first video and the background music are fused in a corresponding time period according to a second ratio of audio amplitude to obtain the fused audio; Among them, the first ratio of the audio amplitude and the second ratio of the audio amplitude are both used to indicate the proportional relationship between the audio of the first video and the background music; the first ratio of the audio amplitude is greater than the second ratio of the audio amplitude.

17. An electronic device, characterized in that: include: one or more processors, memory, and display screens; The memory and the display screen are coupled to the one or more processors, and the memory is used to store the computer. A computer program comprising computer instructions, wherein when the one or more processors execute the computer instructions, the electronic device executes the method for processing video data as described in any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed, is specifically used to implement the video data processing method according to any one of claims 1 to 16.

Images