Video reconstruction method and device

Abstract

The invention discloses a video reconstruction method and device, and relates to the field of computers. According to elements such as figures, animals, articles and sceneries contained in an image of a video, an audio mixed with various sounds in the video is separated into a plurality of sub-audios containing one kind of sound, namely the audio is reconstructed. Therefore, the image and the reconstructed audio can be recombined to obtain the reconstructed video, and the reconstructed video can only reserve the sound specified by the user or peel off the sound specified by the user. The specified sound can be the sound of a certain person, animal, article or scenery. The audio in the video is separated into the independent sub-audios corresponding to the image elements, so that the sound made by any sound-making element in the image can be reserved or removed, and a user can conveniently obtain the required video.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a video reconstruction method and device. Background Art

[0002] In some scenarios, users want to retain a certain or several sounds from the complex sounds in a video. For example, a user uses a camera to record a concert. At the shooting scene, there may be interference sounds such as the shouts of the audience and the roar of the equipment, and the user hopes to retain only the voice of the singer and the accompaniment music in the video. At present, conventional processing solutions are not effective. For example, some noise reduction methods will weaken the sounds that users want to retain, such as the concert accompaniment music, and some will retain the surrounding interference sounds, such as the shouts of the audience.

[0003] In some scenarios, users want to remove (strip) certain sounds from a video. For example, users record scenes such as classrooms and performances, and there are whispers, footsteps, and environmental noise on the road. Users want to strip whispers, footsteps, and environmental noise on the road. For example, the classroom recording scene is a physics experiment, and users want to retain the electromagnetic sound during the experiment and eliminate human voices. Currently, conventional processing solutions cannot accurately strip specified sounds. For example, some methods can only suppress environmental sounds, and some methods cannot distinguish between different types of sounds and can only remove the audio as a whole. Summary of the invention

[0004] The embodiments of the present application provide a video reconstruction method and device, which can accurately retain or remove specified sounds in a video containing multiple sounds. The specified sounds can be human voices, sounds made by animals, sounds made by objects, or environmental sounds, etc.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a video reconstruction method is provided, the method comprising: obtaining a first video, the first video comprising an image and audio; obtaining one or more image elements in the image of the first video, wherein the one or more image elements comprise at least one of a person, an animal, an object, and a scene; obtaining one or more sub-audios from the audio of the first video according to the one or more image elements, wherein each sub-audio corresponds to an image element; after receiving a first operation from a user, reconstructing the first video according to the sub-audio selected in the first operation.

[0007] In this method, the audio mixed with multiple sounds in the video is separated into multiple sub-audios containing one sound according to the elements such as people, animals, objects, and scenery contained in the image of the video, that is, the audio is reconstructed. In this way, the image and the reconstructed audio can be recombined to obtain a reconstructed video, and the reconstructed video can only retain the specified sound or strip off the specified sound (the sub-audio selected in the first operation). The specified sound can be the sound of a certain person, animal, object or scenery. Compared with general processing schemes, the video reconstruction method provided in the embodiment of the present application can retain or remove the sound emitted by any audible element in the image by separating the audio in the video into independent sub-audios corresponding to each image element, so that users can easily obtain the required video.

[0008] Reconstructing the first video according to the sub-audio selected in the first operation includes: removing the sub-audio selected in the first operation from the first video; or combining the sub-audio selected in the first operation with the image of the first video into a reconstructed video. This facilitates user operation.

[0009] In combination with the first aspect, in a possible implementation, obtaining one or more sub-audios from the audio of the first video based on one or more image elements includes: obtaining voiceprint features of each of the one or more image elements; and obtaining the sub-audio corresponding to each image element from the audio of the first video based on the voiceprint features of each image element.

[0010] In this method, the corresponding sub-audio is separated from the audio of the video according to the voiceprint features of each image element that may make a sound in the image, so as to separate the sound made by the image element from the video, so that the sound made by the image element can be easily extracted or removed later.

[0011] In one possible implementation, the image feature information of each of the one or more image elements is input into a preset sound source model, and the sound source model outputs the voiceprint feature of the image element. The sound source model includes a voiceprint library, which includes a large number of voiceprint features of people, animals, objects and scenery.

[0012] Among them, in a possible implementation, obtaining the sub-audio corresponding to each image element from the audio of the first video according to the voiceprint features of each image element includes: suppressing audio other than the sub-audio in the audio of the first video that is closest to the voiceprint features of the first image element, and obtaining the sub-audio corresponding to the first image element; wherein the first image element is any one of the one or more image elements.

[0013] In combination with the first aspect, in a possible implementation, the one or more image elements include a first person image and a second person image, that is, the video includes multiple image elements of the same type. The voiceprint features of multiple image elements (persons) of this type can be first obtained according to a preset sound source model, for example, the first voiceprint feature and the second voiceprint feature can be obtained; then the first sub-audio can be obtained from the audio of the first video according to the first voiceprint feature; the second sub-audio can be obtained from the audio of the first video according to the second voiceprint feature; the corresponding relationship between the first person image, the second person image, the first sub-audio and the second sub-audio can be obtained according to the time when the first person image appears in the first video, the time when the second person image appears in the first video, the time domain information of the first sub-audio and the time domain information of the second sub-audio. Among them, the image elements and sub-audio that match the time when they appear in the video are corresponding.

[0014] Among them, in a possible implementation, when the time domains of the first person image and the second person image overlap, AI facial recognition can be further performed to determine the person who is making the sound in the first person image and the second person image.

[0015] This method can improve the accuracy of the correspondence between image elements and sub-audio, and optimize the effect of video reconstruction.

[0016] In combination with the first aspect, in a possible implementation, the one or more image elements include a second image element, and after obtaining one or more sub-audios from the audio of the first video based on the one or more image elements, an operation of the user selecting one from the multiple sub-audios is received; and the selected sub-audio is determined to be the sub-audio corresponding to the second image element.

[0017] In this method, the video reconstruction effect can be corrected according to the user operation to optimize the video reconstruction effect.

[0018] A sub-audio may be continuous or disconnected in the time domain.

[0019] In a second aspect, the present application further provides a video reconstruction device, the video reconstruction device comprising a memory and one or more processors. The memory and the processor are coupled. The memory is used to store computer program code, the computer program code comprising computer instructions, when the computer instructions are executed by the processor, the video reconstruction device performs the method in the first aspect and any possible implementation thereof.

[0020] In a third aspect, the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect and any possible implementation method thereof.

[0021] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on a computer, the computer executes the method of the first aspect and any possible implementation manner thereof.

[0022] It can be understood that the beneficial effects that can be achieved by the video reconstruction device of the second aspect, the computer-readable storage medium of the third aspect, and the computer program product of the fourth aspect provided above can be referred to the beneficial effects of the first aspect and any possible implementation manner thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of a scenario for extracting and processing audio in the video reconstruction method provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a scenario in which audio is stripped in a video reconstruction method provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a flow chart of a video reconstruction method provided in an embodiment of the present application;

[0026] Figure 4 A hardware structure diagram of a video reconstruction device provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a process flow of a video reconstruction method provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of a flow chart for determining the association between image elements and audio in the video reconstruction method provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of a scenario example of a video reconstruction method provided in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of an example of obtaining individual audios from mixed audios provided in an embodiment of the present application;

[0031] Fig. 9 A schematic diagram of an example of a selection interface provided in an embodiment of the present application;

[0032] Fig.10 A schematic diagram of an example of an audio selection interface provided in an embodiment of the present application;

[0033] Fig.11 A schematic diagram of the structural composition of a video reconstruction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0035] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0036] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0037] In the technical solutions disclosed in this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information (such as personal images) involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0038] A video may include images and audio. When the recording scene of a video is more complex, the types of sounds included in the video will also be more. For example, a video recorded at a concert may include the voice of the singer, the accompanying music, the shouts of the audience, the roar of the machine, etc. For example, an interview video may include the voice of the interviewee, the barking of cats and dogs, the ticking of clocks, the roar of cars, etc. For example, a video recorded in an experimental classroom may include electromagnetic sounds, chemical reaction sounds, the sound of the teacher's explanation, the sound of students' discussion, the bell ringing, etc.

[0039] Users may want to keep one or more specific sounds in a video, or may want to remove one or more specific sounds. For example, keep the singer's voice and accompaniment music in a concert video; keep the interviewee's voice in an interview video; remove the roar of the machine in a concert video, remove the sound of students' discussion in an experimental classroom video, etc.

[0040] The embodiment of the present application provides a video reconstruction method, which separates the audio mixed with multiple sounds in the video into multiple sub-audios containing one sound according to the elements such as people, animals, objects, and scenery contained in the image of the video, that is, reconstructs the audio. In this way, the image and the reconstructed audio can be recombined to obtain a reconstructed video, and the reconstructed video can only retain the specified sound or strip the specified sound. The specified sound can be the sound of a certain person, animal, object, or scenery.

[0041] Figure 1 A schematic diagram of a scenario for extracting and processing audio in the video reconstruction method provided in an embodiment of the present application is shown. Figure 1 As shown, the audio of the video is mixed with multiple sounds, wherein each sound is represented by a graphic. Figure 1 The circle, ellipse, triangle and quadrilateral respectively correspond to a sound. The sound corresponding to one or more people, animals, objects or scenes contained in the video image can be specified, and the audio mixed with multiple sounds is extracted and processed according to the specified sound to obtain the audio containing the specified sound.

[0042] Figure 2 A schematic diagram of a scenario in which audio is stripped in the video reconstruction method provided in an embodiment of the present application is shown. Figure 2 As shown, the audio of the video is mixed with multiple sounds, wherein each sound is represented by a graphic. Figure 2 The circle, ellipse, triangle and quadrilateral respectively correspond to a sound. The sound corresponding to one or more people, animals, objects or scenes contained in the video image can be specified, and the audio mixed with multiple sounds is stripped according to the specified sound to obtain the audio stripped of the specified sound.

[0043] The video reconstruction method provided in the embodiment of the present application can be applied to a device with strong computing power, which can be a server, such as a cloud server; or an electronic device, such as a personal computer (PC), an ultra-mobile personal computer (UMPC), a handheld computer, a notebook computer, a tablet computer, a mobile phone, etc. The embodiment of the present application does not impose any limitation on this.

[0044] Exemplary, reference Figure 3 , an embodiment of the present application provides a video reconstruction method, a video reconstruction device obtains a video, and the video includes an image and an audio. The audio in the video is a mixture of multiple sounds, which is referred to as mixed audio in the embodiment of the present application. In one implementation, the video reconstruction device can separate the video into an image and mixed audio. The video reconstruction device can also obtain various image elements included in the video based on the image of the video, such as images of people, animals, objects, and scenery. For example, Figure 3 As shown, image element 1, image element 2, ... image element n are obtained according to the image of the video.

[0045] The video reconstruction device also matches the sub-audio corresponding to each image element in the mixed audio according to each image element in the video. Figure 3 As shown, audio 1, audio 2, ... audio m are obtained according to the mixed audio. For example, image element 1 is an image of person 1, and audio 1 is a sub-audio corresponding to the sound emitted by the person 1; image element 2 is an image of person 2, and audio 2 is a sub-audio corresponding to the sound emitted by the person 2; image element 3 is an image of animal 1, and audio 3 is a sub-audio corresponding to the sound emitted by the animal 1; image element 4 is an image of object 1, and audio 4 is a sub-audio corresponding to the sound emitted by the object 1; ...

[0046] It is understandable that some image elements generally do not make any sound, that is, some image elements cannot match the corresponding sub-audio in the mixed audio. That is, the value of m is less than or equal to the value of n.

[0047] The images of the producers of certain sounds in the mixed audio may not be included in the video, that is, the mixed audio may also include audios other than audio 1, audio 2, ..., audio m.

[0048] After obtaining independent audio 1, audio 2, ... audio m from a mixed audio of multiple audios, one or more of audio 1, audio 2, ... audio m can be extracted according to user operations and recombined with the video image, or one or more of audio 1, audio 2, ... audio m can be stripped and recombined with the video image.

[0049] It is possible to retain or remove a specified sound in a video containing multiple sounds. For example, the specified sound can be a sound made by a person, an animal, an object, or a scene included in the image of the video.

[0050] Optionally, the video reconstruction device may be a server. Optionally, the video reconstruction device may also be an electronic device with strong computing capabilities, such as a PC, a tablet computer, etc.

[0051] The above video files may be generated by electronic devices equipped with microphones and cameras, such as mobile phones, tablet computers, cameras, etc.

[0052] In some embodiments, after the electronic device with a microphone and a camera shoots and generates a video file, the video file is transmitted to the video reconstruction device for processing. In other embodiments, the video reconstruction device has a microphone and a camera, and the video reconstruction device can shoot and generate a video file and process the video file.

[0053] Figure 4 This is a hardware structure diagram of a video reconstruction device provided in an embodiment of the present application. Figure 4 As shown, the video reconstruction device 100 may include a processor 110 , a memory 120 , a power module 130 , a communication module 140 , and the like.

[0054] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0055] Among them, the controller can be the nerve center and command center of the video reconstruction device 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The video reconstruction device 100 can support one or more video codecs. In this way, the video reconstruction device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc. The NPU is a neural network (NN) computing processor. By drawing on the biological neural network structure, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the video reconstruction device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0056] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0057] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0058] The memory 120 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the video reconstruction device 100 by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function, etc. The data storage area may store data created during the use of the video reconstruction device 100, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0059] The power module 130 may be used to supply power to various components included in the video reconstruction device 100. In some embodiments, the power module 130 may be a battery, such as a rechargeable battery.

[0060] The communication module 140 can provide wireless communication solutions including cellular, Wi-Fi, Bluetooth (BT), wireless data transmission modules (e.g., 433MHz, 868MHz, 915MHz) applied to the video reconstruction device 100. The communication module 140 can be one or more devices integrating at least one communication processing module. The communication module 140 receives electromagnetic waves via an antenna, filters and modulates electromagnetic wave signals, and sends the processed signals to the processor 110. The communication module 140 can also receive signals to be sent from the processor 110, modulate the frequency of the signals, amplify them, and convert them into electromagnetic waves for radiation through the antenna.

[0061] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the video reconstruction device 100. In other embodiments of the present application, the video reconstruction device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0062] Optionally, the video reconstruction device 100 may further include an audio module 150 , a speaker 150A, a microphone 150B, a camera 160 , a display screen 170 , and the like.

[0063] The video reconstruction device 100 implements the display function through a GPU, a display screen 170, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 170 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0064] The display screen 170 is used to display images, videos, etc. The display screen 170 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc.

[0065] The video reconstruction device 100 can realize the shooting function through the ISP, the camera 160, the video codec, the GPU, the display screen 170 and the application processor.

[0066] The video reconstruction device 100 can implement audio functions such as music playing and recording through the audio module 150, the speaker 150A, the microphone 150B, and the application processor.

[0067] The audio module 150 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be arranged in the processor 110, or some functional modules of the audio module 150 can be arranged in the processor 110.

[0068] The speaker 150A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0069] The microphone 150B, also called a "microphone" or "microphone", is used to convert sound signals into electrical signals. The video reconstruction device 100 may be provided with at least one microphone 150B. In other embodiments, the video reconstruction device 100 may be provided with two microphones 150B, which can not only collect sound signals but also realize noise reduction. In other embodiments, the video reconstruction device 100 may also be provided with three, four or more microphones 150B to realize the collection of sound signals, noise reduction, identification of sound sources, and realization of directional recording functions.

[0070] The video reconstruction method provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0071] Figure 5A flow chart of a video reconstruction method provided in an embodiment of the present application is shown.

[0072] The video reconstruction device may be installed with a video processing application (application, App). The video processing App is used to reconstruct the video. For example, the video is edited according to one or more image elements in the video specified by the user, so that only the audio corresponding to the image element specified by the user is retained in the video, or the audio corresponding to the image element specified by the user is stripped from the video.

[0073] It should be noted that in the embodiment of the present application, the user may specify the sound made by a certain person, and the video processing App will perform AI image recognition on the image of the video to obtain the image of a certain person. The character images used in the technical solution of the present application are limited to information that obtains the user's separate consent, including but not limited to notifying and reminding the user to read the relevant user agreement (notification) before the user processes the video App, and signing the agreement (authorization) including authorization of relevant user information (character image). In one example, after the video processing App is started, an authorization prompt message is displayed to prompt the user to authorize the video processing App to perform image recognition on the character images in the video input to the video processing App. After receiving the user's operation of signing the authorization agreement, the video processing App displays a user interface for uploading video files. Users can upload video files that need to be reconstructed to the video reconstruction device in this user interface.

[0074] After the video processing App obtains the video file uploaded by the user, it can obtain the image in the video file and obtain the sub-audio corresponding to each image element in the video image.

[0075] For example, Figure 6 A flow chart of determining the association between image elements and audio in the video reconstruction method provided in an embodiment of the present application is shown.

[0076] like Figure 6 As shown, after the video processing App obtains the video file, it can separate the image and audio of the video to obtain image data and mixed audio data.

[0077] The video processing app performs artificial intelligence (AI) image recognition on the video image based on the image data to obtain various types of images included in the video image. For example, various types of images may include images of people, animals, objects, scenery, and other images. It is understandable that the image classification rules are not limited and the image classification can be adjusted according to actual needs.

[0078] Furthermore, image features may be extracted for each type of image to obtain specific image elements.

[0079] For example, Figure 7 A schematic diagram of a scenario example of a video reconstruction method provided in an embodiment of the present application is shown.

[0080] like Figure 7 As shown, the video image includes various image elements, where each shape represents a type of image. After the video processing app performs AI image recognition on the image, it obtains images of people, animals, objects, scenery, and other images.

[0081] After the video processing app further performs image feature extraction on the person image obtained through AI image recognition, it obtains multiple image elements belonging to the person image, including the image of person 1 (image element 1), the image of person 2 (image element 2), the image of person 3 (image element 3), ... After the video processing app further performs image feature extraction on the animal image obtained through AI image recognition, it obtains multiple image elements belonging to the animal image, including the image of animal 1 (image element 4), the image of animal 2 (image element 5), the image of animal 3 (image element 6), ... After the video processing app further performs image feature extraction on the object image obtained through AI image recognition, it obtains multiple image elements belonging to the object image, including the image of object 1 (image element 7), the image of object 2 (image element 8), ... After the video processing app further performs image feature extraction on the scene image obtained through AI image recognition, it obtains multiple image elements belonging to the scene image, including the image of scene 1 (image element 9), the image of scene 2 (image element 10), the image of scene 3 (image element 11), ... The video processing app further performs image feature extraction on other images acquired through AI image recognition to obtain images of other object 1 (image element 12), images of other object 2 (image element 13), and so on.

[0082] It is understandable that in other embodiments, image recognition can be directly performed on the video images to obtain various image elements. For example, the image of person 1, the image of person 2, the image of person 3, ...; the image of animal 1, the image of animal 2, the image of animal 3, ...; the image of object 1, the image of object 2, ...; the image of scene 1, the image of scene 2, the image of scene 3, ...; the image of other object 1, the image of other object 2, .... That is, there is no need to first obtain a certain type of image and then obtain specific image elements based on the certain type of image; instead, the specific image elements can be directly determined.

[0083] Among them, the method of obtaining specific image elements from the image of the video can be implemented using the image recognition (AI image recognition) algorithm in conventional technology, and the embodiments of the present application are not limited to this.

[0084] Continue to refer Figure 6 ,Furthermore, the video processing App obtains the sub-audio corresponding to each image element from the mixed audio according to each image element.

[0085] In some embodiments, each image element in the video image is marked, and marking information of each image element is obtained and saved.

[0086] In one example, the marking information includes voiceprint information, and the voiceprint information is used to identify the characteristics of the sound emitted by the image element.

[0087] In one implementation, when each image element is obtained by image feature extraction, image feature information of each image element can also be determined, and the image feature information is used to describe the image features of the image element. For example, the image feature information of person 1 (image element 1) is person, the image feature information of person 2 (image element 2) is person, the image feature information of person 3 (image element 3) is person, the image feature information of animal 1 (image element 4) is cat, the image feature information of animal 2 (image element 5) is dog, the image feature information of animal 3 (image element 6) is tiger, the image feature information of object 1 (image element 7) is trumpet, the image feature information of object 2 (image element 8) is violin, etc.

[0088] The image feature information of the image element is input into the preset sound source model to obtain the voiceprint feature corresponding to the image element. The sound source model includes a voiceprint library, for example, the voiceprint library may include voiceprint features of different human voices, voiceprint features of sounds made by animals (such as dog barking, cat meowing, tiger roaring, etc.), voiceprint features of sounds made by objects (such as violin sound, piano sound, alarm clock sound, etc.), and voiceprint features of sounds made by scenery (such as the sound of tree shaking, thunder associated with lightning, etc.). It should be noted that the voiceprint library in the embodiment of the present application is obtained by learning and analyzing the public sound source, which is derived from the public data set and is not collected for the voiceprint of a specific user. The sound source model can output the corresponding voiceprint feature signal according to the input image feature information. Exemplarily, after inputting "person" into the sound source model, a voiceprint feature signal of human voice is output; after inputting "cat" into the sound source model, a voiceprint feature signal of "cat meowing" is output; after inputting "violin" into the sound source model, a voiceprint feature signal of "violin" is output.

[0089] The voiceprint information can be determined based on the voiceprint features. For example, the image feature information of image element 4 is a cat. After "cat" is input into the sound source model, the sound source model outputs a voiceprint feature signal of a cat's cry, and the voiceprint information of image element 4 is determined to be "cat" accordingly; for example, the image feature information of image element 8 is a violin. After "violin" is input into the sound source model, the sound source model outputs a voiceprint feature signal of a "violin", and the voiceprint information of image element 8 is determined to be "violin" accordingly; ...

[0090] It is understandable that the image features of multiple image elements may be the same, for example, the image features of person 1 (image element 1), person 2 (image element 2), and person 3 (image element 3) are all "person". The voiceprint information of person 1, person 2, and person 3 can be distinguished, for example, the voiceprint information of person 1 (image element 1) is "person 1", the voiceprint information of person 2 (image element 2) is "person 2", and the voiceprint information of person 3 (image element 3) is "person 3".

[0091] It is understandable that some image elements may not be matched to corresponding voiceprint features in the voiceprint library. For example, some image elements generally do not make any sound. For another example, the voiceprint features included in the voiceprint library are not comprehensive enough, or the sound source model is not optimized enough, which may also cause the image element to not match the corresponding voiceprint feature. In one implementation, the voiceprint information of the image element that does not match the voiceprint feature can be marked as silent, or the voiceprint information is not marked for such image elements.

[0092] The problems of the voiceprint library being not comprehensive enough and the sound source model not being optimized enough can be improved through continuous self-learning and updating of the sound source model.

[0093] Optionally, the marking information may further include time domain information, where the time domain information is used to indicate a time period in which the image element appears in the video image.

[0094] For example, the time when image element 1 appears is from moment t1 to moment t2, and the time domain information of image element 1 is [t1, t2]; the time when image element 2 appears is from moment t3 to moment t4, and the time domain information of image element 2 is [t3, t4]; ... It can be understood that the time periods when various image elements appear in the video image can overlap. For example, t3 can be a moment earlier than t2.

[0095] Illustratively, Table 1 shows an example of tag information.

[0096] It should be noted that the embodiment of the present application uses the method of saving the marking information in a table as an example. In actual implementation, the marking information can be recorded and saved in other forms.

[0097] Table 1

[0098]

[0099] Furthermore, the video processing App obtains the sub-audio corresponding to each image element from the mixed audio according to the voiceprint feature of each image element, that is, determines the audio generated by the sound emitted by each image element. Figure 7 , after the images of person 1, person 2 and person 3 are input into the sound source model, the voiceprint features corresponding to the characters are obtained, and the sub-audios of person 1, person 2 and person 3 are separated from the mixed audio according to the voiceprint features of person 1, person 2 and person 3. After the images of animal 1, animal 2 and animal 3 are input into the sound source model, the voiceprint features corresponding to animal 1, animal 2 and animal 3 are obtained, and the sub-audios of animal 1, animal 2 and animal 3 are separated from the mixed audio according to the voiceprint features of animal 1, animal 2 and animal 3. After the images of object 1 and object 2 are input into the sound source model, the voiceprint features of object 1 and object 2 are obtained, and the sub-audios of object 1 and object 2 are separated from the mixed audio according to the voiceprint features of object 1 and object 2. After the images of scene 1, scene 2 and scene 3 are input into the sound source model, the voiceprint features corresponding to scene 1, scene 2 and scene 3 are obtained, and the sub-audios of scene 1, scene 2 and scene 3 are separated from the mixed audio according to the voiceprint features of scene 1, scene 2 and scene 3. Exemplarily, other object 1 and other object 2 do not make any sound, and the corresponding voiceprint features are not obtained, and the sub-audios of other object 1 and other object 2 are not separated from the mixed audio. That is, each independent sub-audio is separated from the audio mixed with multiple sounds.

[0100] Continue to refer Figure 6 In one implementation, the voiceprint features corresponding to each image element are used to perform voiceprint denoising on the mixed audio, and the sub-audio corresponding to the image element is obtained from the mixed audio. The voiceprint denoising using the voiceprint features means retaining the sub-audio closest to the voiceprint features and suppressing other sub-audio. Exemplarily, the voiceprint features of "person 3" are used to perform voiceprint denoising on the mixed audio, so that the audio except "person 3" in the mixed audio is suppressed, and the audio of "person 3" is retained, so that the audio of person 3 (image element 3) is obtained from the mixed audio. Exemplarily, the voiceprint features of "violin" are used to perform voiceprint denoising on the mixed audio, so that the audio except "violin" in the mixed audio is suppressed, and the audio of "violin" is retained, so that the audio of object 2 (image element 8) is obtained from the mixed audio. When the video image includes N image elements marked with voiceprint information, N independent audios can be obtained through N rounds of voiceprint denoising. Exemplarily, as Figure 8 As shown in the figure, after 8 rounds of voiceprint denoising on the mixed audio, 8 sub-audios are obtained, namely audio 1, audio 2, audio 3, audio 4, audio 5, audio 6, audio 7 and audio 8. It should be noted that Figure 8In the example shown, each independent audio is continuous in the time domain. In practice, the mixed audio and each sub-audio may be disconnected in the time domain.

[0101] In one implementation, each sub-audio is marked with voiceprint information. For example, if the voiceprint feature corresponding to image element 3 (the voiceprint feature of "person 3") is used to obtain audio 3, then the voiceprint information of audio 3 is marked as "person 3"; for example, if the voiceprint feature corresponding to image element 8 (the voiceprint feature of "violin") is used to obtain audio 8, then the voiceprint information of audio 8 is marked as "violin".

[0102] In some scenarios, sub-audio can be obtained from the mixed audio in combination with other information.

[0103] In one example, the image features of multiple image elements are the same, for example, the image features of person 1 (image element 1), person 2 (image element 2), and person 3 (image element 3) are all "person". In one scenario, person 1 (image element 1), person 2 (image element 2), and person 3 (image element 3) appear in the video image at different times, and the audio can be marked according to the time domain information of the image elements and the time domain of the audio. Exemplarily, the image feature "person" is input into a preset sound source model to obtain a voiceprint feature corresponding to the human voice. The voiceprint features of the human voice are used to separate multiple sub-audios from the mixed audio. Exemplarily, refer to Figure 8 , the start time of audio 1 is t1 and the end time is t2; the start time of audio 2 is t3 and the end time is t4; the start time of audio 3 is t5 and the end time is t6; according to the time domain information [t1, t2] of image element 1, audio 1 is determined to be the sub-audio corresponding to image element 1; according to the time domain information [t3, t4] of image element 2, audio 2 is determined to be the sub-audio corresponding to image element 2; according to the time domain information [t5, t6] of image element 3, audio 3 is determined to be the sub-audio corresponding to image element 3. Correspondingly, the voiceprint information of audio 1 is marked as "person 1", the voiceprint information of audio 2 is marked as "person 2", and the voiceprint information of audio 3 is marked as "person 3". In another scenario, person 1 (image element 1) and person 2 (image element 2) appear in the video image at the same time, and AI recognition can be further performed on the face to determine the time when image element 1 makes a sound and the time when image element 2 makes a sound according to the facial action. The time when the sound is emitted by image element 1 and image element 2 is matched with the time domain of the audio, and the audio is marked with voiceprint information according to the matching result.

[0104] In one example, there is audio in the mixed audio that does not capture the main body of the sound, that is, some audio does not match the image elements. In one implementation, after the audio with the marked voiceprint information is stripped from the mixed audio, the remaining audio signal is subjected to voiceprint recognition based on the voiceprint library to determine the voiceprint features of the remaining audio signal. Optionally, the voiceprint features can be used to mark the voiceprint information of the remaining audio signal. For example, voiceprint recognition is performed on the remaining audio signal based on the voiceprint library to determine that the remaining audio signal is closest to the voiceprint feature of "wind sound", and the voiceprint information of the remaining audio signal is marked as "wind sound".

[0105] Illustratively, Table 2 shows an example of voiceprint information of marked audio.

[0106] Table 2

[0107]

[0108] After the audio is marked with voiceprint information, the image element and the audio are associated through the voiceprint information. Exemplarily, the association relationship between the image element and the audio includes: image element 1-person 1-audio 1, image element 2-person 2-audio 2, image element 3-person 3-audio 3, image element 4-cat-audio 4, image element 5-dog-audio 5, image element 6-tiger-audio 6, image element 7-speaker-audio 7, image element 8-violin-audio 8, ...

[0109] This makes it easy to extract or strip the sound emitted by one or more image elements.

[0110] Continue to refer Figure 5 In some embodiments, the video may be reconstructed based on user selection, for example, the sound emitted by one or more image elements may be extracted, or the sound emitted by one or more image elements may be stripped.

[0111] In one example, after the video processing App uses the voiceprint information to establish associations between each image element and the sub-audio, a selection interface can be displayed to the user, where the user can choose to extract one or more audios or strip one or more audios. Fig. 9 FIG. 1 is a schematic diagram showing an example of a selection interface provided by an embodiment of the present application. Fig. 9 As shown, the video processing App displays a selection interface 900, which includes multiple options, each of which corresponds to an image element that can make a sound, such as "person 1", "person 2", "person 3", "cat", "dog", "tiger", "speaker", "violin", etc. Optionally, the options may also include audio that does not capture the sound subject, such as "wind sound".

[0112] The selection interface 900 may also include prompt information for prompting the user to select the audio to be extracted or stripped. Exemplarily, the selection interface 900 includes a prompt information "Please select the sound to be retained or the sound to be removed as needed".

[0113] The selection interface 900 may also include an “extract” control 901 and a “remove” control 902. The user may click the “extract” control 901 to retain the sub-audio corresponding to the selected option in the video, and the user may also click the “remove” control 902 to remove the sub-audio corresponding to the selected option in the video. Fig. 9 As shown, after the “Person 3” option and the “Violin” option are selected, if the user clicks on the “Extract” control 901, the sub-audio corresponding to “Person 3” and the sub-audio corresponding to “Violin” in the video can be retained, and the audio other than the sub-audio corresponding to “Person 3” and “Violin” can be removed. After the “Person 3” option and the “Violin” option are selected, if the user clicks on the “Remove” control 902, the sub-audio corresponding to “Person 3” and the sub-audio corresponding to “Violin” in the video can be removed.

[0114] In some embodiments, after receiving the user's audio extraction operation (for example, receiving a click operation on the "Extract" control 901), the selected image element is determined according to the one or more selected options, and the corresponding sub-audio is determined according to the selected image element. In one implementation, the audio associated with the image element can be determined according to the voiceprint information. Optionally, the audio associated with the selected image element can also be processed by sound quality restoration, noise denoising, etc. Afterwards, the video image and the audio associated with the selected image element are aligned in the time domain to generate a reconstructed video.

[0115] In other embodiments, after receiving the user's audio stripping operation (for example, receiving a click operation on the "remove" control 902), the selected image element is determined according to one or more options, and the corresponding sub-audio is determined according to the selected image element. In one implementation, the sub-audio corresponding to the selected image element is removed from all the audio included in the video to obtain the reconstructed audio. In another implementation, the audio of all the acquired sub-audios except the sub-audio corresponding to the selected image element is synthesized into one audio to obtain the reconstructed audio. Optionally, the reconstructed audio can be processed by sound quality restoration, noise denoising, etc. Afterwards, the video image and the reconstructed audio are aligned in the time domain to generate a reconstructed video.

[0116] In one example, Figure 7As shown, when the user specifies to keep the voice of person 3, the image of the video is merged with the audio of person 3, so that only the voice of person 3 in the video is retained. When the user specifies to strip the voice of person 3, the audio of person 3 is removed from the audio mixed with multiple sounds and then merged with the image of the video, so that the voice of person 3 in the video is stripped.

[0117] Optionally, in some embodiments, after the reconstructed video is generated, a function of previewing the reconstructed video may be provided to the user. After the user confirms to save the reconstructed video, the reconstructed video is saved.

[0118] In some embodiments, corrections can be made during the video reconstruction process in combination with user operations. In one example, after the video processing App obtains multiple sub-audios from the mixed audio, it can provide the user with an audio selection interface. The user can select the corresponding sub-audio for the image element in the audio selection interface. Exemplarily, in response to the user's Fig. 9 In the selection interface 900, the "Character 1" option is clicked, and the video processing App displays the following Fig.10 The audio selection interface 1000 shown includes an image control 1001, which is used to display an image element in the video image. For example, the image control 1001 is used to display the image of "person 1" in the above example. The audio selection interface 1000 also includes a "voice 1" option and a "voice 2" option, wherein the "voice 1" option is associated with audio 1, and the "voice 2" option is associated with audio 2, and both audio 1 and audio 2 are sub-audio obtained from the mixed audio of the video. The audio selection interface 1000 also includes an "OK" control 1002 and a "Cancel" control 1003. After the "voice 1" option or the "voice 2" option is selected, if the user clicks the "OK" control 1002, the audio associated with the selected option is associated with the image element associated with the image control 1001; in an implementation method, the audio associated with the selected option and the image element associated with the image control 1001 are marked with the same voiceprint information. If a user click operation on the “Cancel” control 1003 is received, the association between the audio associated with the selected option and the image element associated with the image control 1001 is canceled.

[0119] Optionally, in some embodiments, relevant configurations of the video reconstruction process (such as the correspondence between image elements and voiceprint features, options selected by the user, etc.) can be saved for subsequent video recording, video reconstruction, and other processes.

[0120] It should be noted that the embodiments of the present application are introduced by taking the reconstruction of a video as an example. In other embodiments, the video reconstruction method provided in the embodiments of the present application can also be applied to other scenarios. For example, the video reconstruction method provided in the embodiments of the present application can also be used in the fields of AI intelligent recognition, audio and video recording, video editing, etc. Using the video reconstruction method provided in the embodiments of the present application, based on an element in an image specified by a user, the sub-audio corresponding to the element is identified from audio mixed with multiple sounds, and a correspondence between the image element and the audio is established; wherein the image and the audio may not belong to the same video. Exemplary, Figure 7 The image and audio may not be separated from the same video.

[0121] It is understandable that the video reconstruction device provided in the embodiment of the present application includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0122] The embodiment of the present application can divide the functional modules of the above-mentioned video reconstruction device according to the above-mentioned method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0123] In one example, see Fig.11 , which shows a possible structural diagram of the video reconstruction device involved in the above embodiment. The video reconstruction device 1100 includes: a processing unit 1101 and a storage unit 1102.

[0124] The processing unit 1101 is used to control and manage the actions of the video reconstruction device 1100. The storage unit 1102 is used to store the program code and data of the video reconstruction device 1100. The processing unit 1101 calls the program code stored in the storage unit 1102 to execute each step in the above method embodiment.

[0125] Of course, the unit modules in the above video reconstruction device 1100 include but are not limited to the above processing unit 1101 and storage unit 1102. For example, the video reconstruction device 1100 may also include a display unit 1103 for displaying the user interface of the video reconstruction device 1100, such as a selection interface, an audio selection interface, etc.

[0126] The processing unit 1101 may be a processor or a controller, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The storage unit 1102 may be a memory. The display unit 1103 may be a display screen, etc.

[0127] For example, the processing unit 1101 is a processor (such as Figure 4 The processor 110 shown in FIG. 10A ) may be a storage unit 1102, which may be a memory (eg, Figure 4 The memory 120 shown in FIG. 100 ), the display unit 1103 may be a display screen (eg, Figure 4 The video reconstruction device 1100 provided in the embodiment of the present application can be Figure 4 The video reconstruction device 100 shown in the figure. The above processor, memory, display screen, etc. can be connected together, for example, through a bus. The processor calls the program code stored in the memory to execute each step in the above method embodiment.

[0128] The embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through lines. For example, the interface circuit can be used to receive signals from other devices (such as the memory of the electronic device). For another example, the interface circuit can be used to send signals to other devices (such as processors). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

[0129] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a video reconstruction device, the video reconstruction device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0130] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0131] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.

[0132] Among them, the video reconstruction device, chip system, computer-readable storage medium, computer program product or chip system provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0133] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A video reconstruction method, characterized in that: include: Get the first video; Obtain one or more image elements in the first video; The one or more image elements include at least one of a person, an animal, an object, and a scene; Acquire one or more sub-audios from the audio of the first video according to the one or more image elements; wherein each of the sub-audios corresponds to one of the image elements; After receiving a first operation from the user, the first video is reconstructed according to the sub-audio selected in the first operation.

2. The method according to claim 1, characterized in that The acquiring one or more sub-audios from the audio of the first video according to the one or more image elements comprises: Acquire a voiceprint feature of each of the one or more image elements; The sub-audio corresponding to each image element is obtained from the audio of the first video according to the voiceprint feature of each image element.

3. The method according to claim 2, characterized in that The acquiring of the voiceprint feature of each of the one or more image elements comprises: The image feature information of each of the one or more image elements is input into a preset sound source model to obtain the voiceprint feature of each of the image elements; wherein the sound source model includes a voiceprint library, and the voiceprint library includes voiceprint features of people, animals, objects and scenery.

4. The method according to claim 2, characterized in that: The acquiring the sub-audio corresponding to each image element from the audio of the first video according to the voiceprint feature of each image element comprises: Suppress the audio of the first video except the sub-audio closest to the voiceprint feature of the first image element, and obtain the sub-audio corresponding to the first image element; the first image element is any one of the one or more image elements.

5. The method according to claim 2, characterized in that: The one or more image elements include a first character image and a second character image, The acquiring of the voiceprint feature of each of the one or more image elements comprises: Acquire a first voiceprint feature and a second voiceprint feature; the first voiceprint feature and the second voiceprint feature are voiceprint features of a person; The acquiring the sub-audio corresponding to each image element from the audio of the first video according to the voiceprint feature of each image element comprises: Acquire a first sub-audio from the audio of the first video according to the first voiceprint feature; Acquire a second sub-audio from the audio of the first video according to the second voiceprint feature; According to the time when the first character image appears in the first video, the time when the second character image appears in the first video, the time domain information of the first sub-audio and the time domain information of the second sub-audio, the correspondence between the first character image, the second character image, the first sub-audio and the second sub-audio is obtained.

6. The method according to any one of claims 1 to 5, characterized in that: The one or more image elements include a second image element, and after obtaining one or more sub-audios from the audio of the first video according to the one or more image elements, the method further includes: receiving a second operation of the user; the second operation is used to select one of the plurality of sub-audios; Determine that the sub-audio selected in the second operation is the sub-audio corresponding to the second image element.

7. The method according to any one of claims 1 to 5, characterized in that: The reconstructing the first video according to the sub-audio selected in the first operation comprises: removing the sub-audio selected in the first operation from the first video; or, The sub-audio selected in the first operation and the image of the first video are combined into a reconstructed video.

8. A video reconstruction device, characterized in that: include: Processor and memory; The processor is coupled to the memory; the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the video reconstruction device executes the method according to any one of claims 1-7.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer program product runs on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images