Video photographing method and terminal device

Abstract

Embodiments of the present application provide a video photographing method and a terminal device. The method comprises: first, a photographing device receives a first photographing operation; in response to the first photographing operation, the photographing device records first target audio data and first other audio data; then, the photographing device receives a second photographing operation; in response to the second photographing operation, the photographing device records second target audio data and second other audio data, a signal-to-noise ratio of the second target audio data being greater than a signal-to-noise ratio of the first target audio data. A higher signal-to-noise ratio of audio data indicates better audio quality (i.e., degree of reproduction) and definition (i.e., degree of intelligibility). That is, the quality and definition of the audio of the second target audio data are better than those of the first target audio data.

Description

Video shooting method and terminal device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 28, 2023, with application number 202311283846.6 and application name “Video Shooting Method and Terminal Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of terminal technology, and in particular to a video shooting method and a terminal device. Background Art

[0003] With the popularization of terminal products (such as mobile phones, tablets, watches, etc.) and the advantages of terminal products such as portability and simple operation, more and more users will use terminal products to shoot videos, for example, shooting vlogs, recording lecture videos, recording live performance videos, live broadcasts, etc.

[0004] When a video is shot in a noisy environment or the sound source of the target object is not prominent, the audio quality and clarity of the video are affected. Subsequent use of the video requires specialized software to extract, edit, and remove noise from the audio. This results in inefficient video production, long production cycles, and a poor user experience.

[0005] Summary of the Invention

[0006] To solve the above technical problems, the present application provides a video shooting method and terminal device. The method can improve the signal-to-noise ratio of the audio data of the target object in the captured video, thereby improving the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio data of the target object in the captured video.

[0007] In a first aspect, an embodiment of the present application provides a video shooting method, which includes: first, a shooting device receives a first shooting operation; in response to the first shooting operation, the shooting device records first target audio data and first other audio data, the first target audio data being the audio data of the target object, and the first other audio data being the audio data of other objects; then, the shooting device receives a second shooting operation; in response to the second shooting operation, the shooting device records second target audio data and second other audio data, the second target audio data being the audio data of the target object, and the second other audio data being the audio data of other objects; wherein the signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data.

[0008] The higher the signal-to-noise ratio of the audio data, the better the audio quality (i.e., restoration) and clarity (i.e., recognizability). In other words, after receiving the second capture operation, the second target audio data recorded has better audio quality (i.e., restoration) and clarity (i.e., recognizability) than the first target audio data. This allows the video to be used directly later (for example, forwarded or shared on social platforms) without the need for specialized software to process the second target audio data before use, simplifying user operations and improving the user experience.

[0009] Exemplarily, the first shooting operation may refer to an operation of clicking a recording start option in a recording interface of a shooting device.

[0010] Exemplarily, there may be multiple possibilities for the second shooting operation, which are specifically determined according to the user interface (UI) design of the shooting application in the shooting device. This application does not limit what the second shooting operation is.

[0011] For example, the second shooting operation may be an operation of clicking on a confirmation option in a shooting mode prompt option that pops up on the recording interface after entering the recording interface.

[0012] For example, the second shooting operation may include, after entering the recording interface, clicking on a confirmation option in a shooting mode prompt option that pops up on the recording interface and clicking on a recording start option on the recording interface.

[0013] For example, the second shooting operation may include clicking a target recording option in an application interface of a camera application, and clicking a recording start option in a recording interface.

[0014] For example, the second shooting operation may be an operation of clicking a target object shooting option in the recording interface.

[0015] For example, the second shooting operation may include an operation of clicking a target object shooting option in the recording interface, and an operation of clicking a recording start option in the recording interface.

[0016] Exemplarily, in response to the first shooting operation, the shooting device may further record first target video data; and in response to the second shooting operation, the shooting device may further record second target video data.

[0017] Exemplarily, in response to the first shooting operation, the shooting device may call the front camera / rear camera to record the first target video data.

[0018] Exemplarily, in response to the second shooting operation, the shooting device may call the front camera / rear camera to record the second target video data.

[0019] Exemplarily, the first target video data, the first target audio data and the first other audio data may constitute the first audio-video data.

[0020] Exemplarily, the second target video data, the second target audio data and the second other audio data may constitute the second audio-video data.

[0021] Exemplarily, the shooting device may include but is not limited to: a mobile phone, a tablet computer, a smart watch, a laptop computer, a home camera, etc., and this application does not impose any restrictions on this.

[0022] According to the first aspect, the method further includes: the photographing device receiving a selection operation for a target object; and in response to the selection operation for the target object, the photographing device determining the target object. In this way, the user can customize the target object to meet the user's personalized needs.

[0023] According to the first aspect, or any implementation of the first aspect above, the method further includes: the camera device determining, as the target object, any one of the following: the object closest to the center of the captured image, the object closest to the camera device, or the object with the largest area in the captured image. In this way, the camera device can automatically select the target object without the user having to select the target object, simplifying user operations.

[0024] It should be understood that the shooting device can also use other methods to determine the target object, and this application does not limit this.

[0025] According to the first aspect, or any implementation of the first aspect above, the method further includes: the camera receiving a target object switching operation; and in response to the target object switching operation, the camera recording third target audio data and third other audio data, where the third target audio data is the audio data of the switched target object. In this way, during video capture, the target object can be changed as desired by the user without having to stop the current video recording and reshoot another video, thereby simplifying user operations.

[0026] The third other audio data is audio data of other objects, and the other objects in the third other audio data are the same as or different from the other objects in the first other audio data and the second other audio data.

[0027] According to the first aspect, or any implementation of the first aspect above, the target object is one or more.

[0028] According to the first aspect, or any implementation of the first aspect above, the second target audio data is audio data collected by the camera by performing directed sound pickup on the target object. In this way, the signal-to-noise ratio of the second target audio data can be improved.

[0029] Exemplarily, the first target audio data may be audio data collected by a shooting device performing omnidirectional sound pickup.

[0030] According to the first aspect, or any implementation of the first aspect above, the method further includes: in response to the second shooting operation, the shooting device records target video data, where the target video data is video data of a target object; and the shooting device collects audio data by performing directional sound pickup of the target object, specifically including: the shooting device determining the target object based on the target video data; the shooting device determining a first distance based on the target object, where the first distance is the distance between the target object and the shooting device; and the shooting device performing directional sound pickup of the target object based on the first distance to collect second target audio data. In this way, directional sound pickup of the target object can be achieved in the early stages of video shooting.

[0031] Exemplarily, the target video data here may refer to the second target video data mentioned above.

[0032] According to the first aspect, or any implementation of the first aspect above, the method further includes: the camera performing real-time sound source tracking of the target object; if the position of the target object changes, the camera re-determines the first distance and performs a step of pointing and picking up sound from the target object based on the first distance to collect second target audio data. In this way, even if the position of the target object changes, the audio quality and clarity of the target object in the captured video can be maintained.

[0033] According to the first aspect, or any implementation of the first aspect above, the method can be applied to at least one scenario of live broadcast, shooting singing videos, shooting sports videos, shooting meeting videos, shooting lecture videos, shooting speech videos or shooting interview videos.

[0034] It should be understood that the present application can be applied to any scenario requiring video shooting, and the present application does not limit this.

[0035] In a second aspect, an embodiment of the present application provides a video shooting method, which includes: first, the shooting device receives a second shooting operation; then, in response to the second shooting operation, the shooting device shoots first video data for a target object and points and picks up sound for the target object to collect first audio data; the shooting device merges the first video data and the first audio data to obtain audio and video data.

[0036] That is to say, in the process of video shooting, the present application collects audio data (i.e., first audio data) by performing directional sound pickup on the target object. Compared with the prior art of collecting audio data by performing omnidirectional sound pickup on the target object during video shooting, performing directional sound pickup on the target object can suppress the sounds of other objects, thereby improving the signal-to-noise ratio of the audio data of the target object, thereby improving the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio of the target object in the shot video. In this way, the video can be used directly later (for example, forwarded, shared to a social platform, etc.), without the need to use professional software to process the audio in the video before use, which can simplify user operations and improve user experience.

[0037] Exemplarily, the front camera / rear camera may be called to shoot first video data for the target object.

[0038] For example, the beam state of the microphone can be adjusted to achieve directional sound pickup of the target object to collect the first audio data. The beam state may include a beam pointing angle and a beam shape.

[0039] For example, the first audio data and the first video data can be stored in the same multimedia file, which is a video shot for the target object and can also be called an audio-video file (or audio-video data). Each frame of audio in the first audio data has a timestamp, and each frame of image in the first video data has a timestamp, and the images and audio captured at the same time have the same timestamp; thus, when playing the video shot for the target object, the playback module can play a frame of image and a frame of audio with the same timestamp at the same time, thereby achieving audio-visual synchronization.

[0040] According to a second aspect, a camera performs directed sound pickup at a target object to collect first audio data, including: the camera determines the target object based on first video data; the camera determines a first distance based on the target object; the first distance is the distance between the target object and the camera; and the camera performs directed sound pickup at the target object based on the first distance to collect the first audio data. In this way, directed sound pickup at the target object can be achieved in the early stages of video capture.

[0041] Exemplarily, the photographing device may determine the first distance based on the first M (M is a positive integer) frames of the first video data.

[0042] Exemplarily, the camera may also determine the first distance based on initial video data (or the first target video data in the first aspect above). The initial video data may refer to video data captured by the camera after entering the recording interface (at this time, the camera is activated) and before performing the second capture operation.

[0043] According to the second aspect, or any implementation of the second aspect above, a camera device collects first audio data by pointing and picking up sound from a target object based on a first distance, specifically including: the camera device determines a first angle of the target object relative to the camera device based on the first distance; the camera device adjusts the beam pointing angle of the microphone to the first angle; according to a preset rule and the first distance, adjusts the beam shape of the microphone to the first beam shape; and the camera device's microphone collects the first audio data using the first beam shape and the first angle. In this way, the microphone's target sound pickup area can include the target object and minimize other objects, thereby improving the signal-to-noise ratio of the target object's audio data.

[0044] Exemplarily, the first angle of the target object relative to the shooting device may refer to the angle between the line between the center of the shooting device and the target object and the X-axis; where X is a line passing through the center of the shooting device and perpendicular to the back of the shooting device.

[0045] Exemplarily, the preset rule may be a rule learned in advance through training, with the goal of increasing the sound pickup effect of the target object. For example, the preset rule is: when the distance is X1 meters, the short axis of the left beam is a1 mm and the short axis of the right beam is a2 mm, and / or, the long axis of the left beam is b1 mm and the long axis of the right beam is b2 mm; when the distance is X2 meters, the short axis of the left beam is a3 mm and the short axis of the right beam is a4 mm, and / or, the long axis of the left beam is b3 mm and the long axis of the right beam is b4 mm. Wherein, X1, X2, a1-a4, and b1-b4 are all positive integers.

[0046] According to the second aspect, or any implementation of the second aspect above, the camera performs real-time sound source tracking of a target object; if the position of the target object changes, the camera re-determines the first distance and performs the step of pointing and picking up sound from the target object based on the first distance to collect the first audio data. In this way, even if the position of the target object changes, the audio quality and clarity of the audio data of the target object in the captured video can be maintained.

[0047] For example, the second position information (that is, the changed position information) of the target object at the current moment (or current period) can be determined by sound source positioning. The second position information may include a second pitch angle (i.e., the angle between the line connecting the target object and the origin and the Z axis), a second horizontal angle (i.e., the angle between the line connecting the target object and the origin and the X axis), and a second distance (i.e., the distance between the target object and the terminal device); the second distance can then be used as the first distance.

[0048] According to the second aspect, or any implementation of the second aspect above, the method also includes: the shooting device receives a focus adjustment operation; in response to the focus adjustment operation, the shooting device adjusts the focus of the camera, and the camera uses the adjusted focus to shoot second video data for the target object, and the shooting device adjusts the beam state of the microphone, and the microphone uses the beam after the beam state is adjusted to point to the target object to pick up sound to collect second audio data, and the beam state includes a beam shape and a beam pointing angle.

[0049] In this way, when the distance between the target object and the terminal device becomes farther, the microphone's pickup distance can be increased to achieve long-distance sound pickup, thereby improving the sound pickup effect for the target object; and increasing the volume of the target object's audio data in the first audio data. And when the distance between the target object and the terminal device becomes closer, the microphone's pickup distance can be reduced to further improve the sound pickup effect for the target object; and reducing the volume of the target object's audio data in the first audio data. In other words, this application can perform corresponding "audio zoom" as the video zooms.

[0050] In addition, it is also possible to achieve visual and auditory synchronization of the distance between the target object and the shooting device, that is, the distance between the target object and the shooting device perceived by watching the video data is equal to or slightly different from the distance between the target object and the shooting device perceived by listening to the audio data. For example, when the actual distance between the target object and the shooting device is 5m, the user increases the focal length, and the proportion of the target object in the picture of the video data becomes larger; at this time, the distance between the target object and the shooting device perceived by watching the video data is 2m; however, since the existing technology cannot achieve "audio zoom", the distance between the target object and the shooting device perceived by listening to the audio data collected using the existing technology is 5m, while the distance between the target object and the shooting device perceived by listening to the audio data collected using the present application (i.e., the second audio data) is 2.5m.

[0051] According to the second aspect, or any implementation of the second aspect above, the camera is a rear camera.

[0052] According to the second aspect, or any implementation of the second aspect above, the method further includes: when a change in the distance between the target object and the camera device is detected, the camera device adjusts the beam state of the microphone, and the microphone uses the beam after the adjusted beam state to point at the target object to pick up sound to collect third audio data, where the beam state includes a beam shape and a beam pointing angle. In this way, even if the user does not perform a focus adjustment operation (or the camera device does not support the user performing a focus adjustment operation), audio zoom can still be achieved.

[0053] For example, the volume of the target object's audio data can be used to detect whether the distance between the target object and the camera has changed. Alternatively, if the camera's focal length remains unchanged, the proportion of the target object in the video data can be used to detect whether the distance between the target object and the camera has changed. This application is not limited to this. In addition, in one embodiment, during real-time sound source tracking, it can be determined whether the distance between the target object and the camera has changed.

[0054] According to the second aspect, or any implementation of the second aspect above, the camera is a front camera.

[0055] According to the second aspect, or any implementation of the second aspect above, the method further includes: the camera receiving a selection operation for a target object; and in response to the selection operation for the target object, the camera determining the target object. In this way, the user can customize the target object selection to meet the user's personalized needs.

[0056] According to the second aspect, or any implementation of the second aspect above, the method also includes: the shooting device determines any one of the object closest to the center of the shooting screen, the object closest to the shooting device, or the object with the largest proportion in the shooting screen as the target object.

[0057] According to the second aspect, or any implementation of the second aspect above, the target object is one or more.

[0058] Exemplarily, when there are multiple target objects, the beam state of the microphone can be adjusted according to the second position information of the multiple target objects so that the target pickup area of ​​the microphone is the intersection or union of the positions of the multiple target objects, or in other words, the target pickup area of ​​the microphone includes multiple target objects and as few other objects as possible.

[0059] According to the second aspect, or any implementation of the second aspect above, the method further includes: the camera receiving a target object switching operation; and in response to the target object switching operation, the camera capturing fourth video data for the switched target object and performing point-based sound pickup on the switched target object to collect fourth audio data. In this way, during video capture, the target object can be changed as desired by the user without having to stop the current video recording and reshoot another video, thereby simplifying user operations.

[0060] It should be noted that the first audio data in the second aspect and any implementation of the second aspect may include the second target audio data and the second other audio data in the first aspect and any implementation of the first aspect. And the fourth audio data in the second aspect and any implementation of the second aspect may include the third target audio data and the third other audio data in the first aspect and any implementation of the first aspect. In addition, the second audio data and the third audio data may both include audio data of the target object and audio data of other objects.

[0061] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0062] In a third aspect, an embodiment of the present application provides a terminal device, which can be used to execute the method executed by the shooting device in the above-mentioned first aspect and any one of the implementations of the first aspect; or, execute the method executed by the shooting device in the second aspect and any one of the implementations of the second aspect.

[0063] Exemplarily, terminal devices include but are not limited to: mobile phones, tablet computers, smart watches, laptop computers, home cameras, etc., and this application does not impose any restrictions on this.

[0064] According to a third aspect, the terminal device may include at least three groups of microphones.

[0065] According to the third aspect, or any implementation of the third aspect, a distance between any two groups of microphones in the at least three groups of microphones is greater than a preset distance.

[0066] Exemplarily, the preset distance is greater than 0 (that is, there is a certain distance between any two groups of microphones). The preset distance can be set according to specific needs, and this application does not impose any restrictions on this.

[0067] Illustratively, the distribution of at least three groups of microphones in different terminal devices may be different, and this application does not impose any limitation on this.

[0068] According to the third aspect, or any implementation of the third aspect, a group of microphones includes at least one microphone.

[0069] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0070] The third aspect and any implementation of the third aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0071] In a fourth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the video shooting method in the first aspect or any possible implementation of the first aspect.

[0072] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0073] In a fifth aspect, an embodiment of the present application provides a chip, which includes at least one processing circuit and an interface, wherein the processing circuit can execute the video processing method of the first aspect and any one of the first aspects.

[0074] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0075] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the video shooting method in the first aspect or any possible implementation of the first aspect.

[0076] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] FIG1 is a schematic diagram showing a hardware structure of an electronic device;

[0078] FIG2 is a schematic diagram illustrating a software structure of an electronic device;

[0079] FIG3 is a schematic diagram showing an exemplary distribution of microphones;

[0080] FIG4A is a schematic diagram illustrating an exemplary application scenario;

[0081] FIG4B is a schematic diagram illustrating an exemplary application scenario;

[0082] FIG4C is a schematic diagram illustrating an exemplary application scenario;

[0083] FIG4D is a schematic diagram illustrating an exemplary application scenario;

[0084] FIG5A is a schematic diagram illustrating an exemplary video shooting process;

[0085] FIG5B is a schematic diagram of an exemplary coordinate system;

[0086] FIG5C is a schematic diagram of a beam of a microphone shown as an example;

[0087] FIG5D is a schematic diagram of a beam of a microphone shown as an example;

[0088] FIG5E is a schematic diagram of a beam of a microphone shown as an example;

[0089] FIG5F is a schematic diagram illustrating exemplary initial audio data;

[0090] FIG5G is a schematic diagram illustrating exemplary initial audio data;

[0091] FIG5H is a schematic diagram showing an exemplary first audio data;

[0092] FIG5I is a schematic diagram showing an exemplary first audio data;

[0093] FIG6A is a schematic diagram of an exemplary mobile phone interface;

[0094] FIG6B is a schematic diagram of an exemplary mobile phone interface;

[0095] FIG6C is a schematic diagram of an exemplary mobile phone interface;

[0096] FIG6D is a schematic diagram of an exemplary mobile phone interface;

[0097] FIG6E is a schematic diagram of an exemplary mobile phone interface;

[0098] FIG6F is a schematic diagram of an exemplary mobile phone interface;

[0099] FIG6G is a schematic diagram of an exemplary mobile phone interface;

[0100] FIG7 is a schematic diagram illustrating an exemplary video shooting process;

[0101] FIG8A is a schematic diagram of an exemplary mobile phone interface;

[0102] FIG8B is a schematic diagram of an exemplary mobile phone interface;

[0103] FIG8C is a schematic diagram of an exemplary mobile phone interface;

[0104] FIG8D is a schematic diagram of an exemplary mobile phone interface;

[0105] FIG8E is a schematic diagram of an exemplary mobile phone interface;

[0106] FIG8F is a schematic diagram of a beam of a microphone shown as an example;

[0107] FIG8G is a schematic diagram of a beam of a microphone shown as an example;

[0108] FIG9 is a schematic diagram illustrating an exemplary video shooting process;

[0109] FIG10 is a schematic diagram illustrating an exemplary video shooting process;

[0110] FIG11A is a schematic diagram of an exemplary mobile phone interface;

[0111] FIG11B is a schematic diagram of an exemplary mobile phone interface;

[0112] FIG11C is a schematic diagram of an exemplary mobile phone interface;

[0113] FIG11D is a schematic diagram of an exemplary mobile phone interface;

[0114] FIG11E is a schematic diagram of an exemplary mobile phone interface;

[0115] FIG11F is a schematic diagram of an exemplary mobile phone interface;

[0116] FIG11G is a schematic diagram of an exemplary mobile phone interface;

[0117] FIG11H is a schematic diagram of an exemplary mobile phone interface;

[0118] FIG12 is a schematic diagram illustrating an exemplary video shooting process;

[0119] FIG13A is a schematic diagram of an exemplary mobile phone interface;

[0120] FIG13B is a schematic diagram of an exemplary mobile phone interface. DETAILED DESCRIPTION

[0121] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0122] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0123] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0124] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0125] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0126] FIG1 shows a schematic diagram of the structure of an electronic device 100. It should be understood that the electronic device 100 shown in FIG1 is merely an example of an electronic device, and that the electronic device 100 may have more or fewer components than shown, may combine two or more components, or may have a different component configuration. The various components shown in FIG1 may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0127] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0128] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0129] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0130] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0131] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0132] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0133] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0134] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 and to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as AR devices.

[0135] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0136] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0137] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0138] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0139] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0140] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0141] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0142] The wireless communication module 160 can provide wireless communication solutions including WLAN (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0143] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with a network and other devices via wireless communication technology. Wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0144] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0145] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0146] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0147] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0148] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. Among them, the N cameras 193 may include N1 front cameras and N2 rear cameras, N1 and N2 are integers, and N1+N2=N.

[0149] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0150] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0151] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0152] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0153] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0154] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0155] Exemplarily, the electronic device may include three or more groups of microphones 170C, wherein each group of microphones 170C may include at least one microphone.

[0156] It should be noted that the distance between any two groups of microphones 170C in the electronic device is greater than the preset distance; wherein, the preset distance is greater than 0 (that is, there is a certain distance between any two groups of microphones), and the preset distance can be set according to specific needs, and this application does not impose any restrictions on this.

[0157] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0158] FIG2 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0159] The layered architecture of electronic device 100 divides the software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0160] The application layer can include a series of application packages.

[0161] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0162] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0163] As shown in FIG2 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, a media service, and the like.

[0164] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0165] Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0166] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0167] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0168] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0169] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0170] Media services can be used to process multimedia data (such as audio data, video data, etc.).

[0171] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0172] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0173] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0174] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0175] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0176] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0177] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0178] A 2D graphics engine is a drawing engine for 2D drawings.

[0179] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0180] It is understood that the components included in the system framework layer, system library, and runtime layer shown in FIG2 do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or split certain components, or arrange the components differently.

[0181] Figure 3 is a schematic diagram showing an exemplary distribution of microphones. In Figure 3 , the electronic device is a mobile phone, and the electronic device includes three groups of microphones, each group of microphones includes one microphone.

[0182] 3 , illustratively, a mobile phone may include three microphones: microphone 1, microphone 2, and microphone 3. In some embodiments, microphone 1 may be referred to as a top microphone, microphone 2 may be referred to as a bottom microphone, and microphone 3 may be referred to as a back microphone.

[0183] It should be noted that the positions of the three microphones in Figure 3 are only an example of this application. The position of microphone 1 of different mobile phones may be slightly different, or the position of microphone 2 of different mobile phones may be slightly different, or the position of microphone 3 of different mobile phones may be slightly different.

[0184] In addition, when the electronic device is other terminal devices (such as tablets, smart watches, home cameras, etc.), the distribution of microphone 1, microphone 2 and microphone 3 may be different from the distribution of microphone 1, microphone 2 and microphone 3 in a mobile phone, and this application does not impose any restrictions on this.

[0185] Figure 4A is a schematic diagram of an exemplary application scenario, wherein the application scenario shown in Figure 4A is shooting a singing video.

[0186] 4A , in a possible scenario, while singers A and B are performing on stage in a studio, audience member 1 sitting in the audience can shoot a video of the chorus of singers A and B.

[0187] 4A , in a possible scenario, while singers A and B are performing on stage in a studio, audience member 1 sitting in the audience can shoot a solo video of singer A, or shoot a solo video of singer B.

[0188] Figure 4B is a schematic diagram of an exemplary application scenario, wherein Figure 4B illustrates shooting a sports video.

[0189] For example, when a family is doing outdoor activities, parents usually shoot videos of their children playing / entertaining / competing to record their daily life. For example, referring to FIG4B , for example, a parent is shooting a video of a child playing table tennis.

[0190] Figure 4C is a schematic diagram of an exemplary application scenario, wherein Figure 4C illustrates shooting a conference video.

[0191] 4C , illustratively, after the meeting begins, a video of the meeting speaker may be captured for subsequent meeting summary.

[0192] Figure 4D is a schematic diagram of an exemplary application scenario. Figure 4D illustrates shooting a video of a text recitation.

[0193] 4D , for example, in order to complete the school's recitation check-in task, parents can shoot a video of their child reciting a text.

[0194] It should be understood that the terminal devices used for shooting in Figures 4A to 4D include but are not limited to: mobile phones, tablets, smart watches, laptops, home cameras, etc., and this application does not impose any restrictions on this.

[0195] It should be understood that this application may also include other video shooting scenarios, such as shooting lecture videos, live broadcasts, shooting speech videos, shooting interview videos, etc., and this application does not limit this.

[0196] In the scene shown in Figure 4A, there are many people in the studio; in the scene shown in Figure 4B, there are many people outdoors, and there are also other noises such as wind blowing through trees, car horns, and birdsong. This will cause a lot of ambient noise in the shooting environment, which in turn affects the audio quality of the captured video. In Figure 4C, some participants in the meeting are closer to their terminal devices than the speaker is to their terminal device; in Figure 4D, the distance between the parent and the terminal device is closer than the distance between the child and the terminal device. This results in the target subject's voice not being prominent in the captured video (or the target subject is not the main sound source).

[0197] In order to solve the above problems, the present application proposes a video shooting method, which can suppress the sounds of other objects and thereby improve the signal-to-noise ratio of the audio data of the target object, thereby improving the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio of the target object in the shot video; in this way, the video can be used directly later (for example, forwarded, shared to social platforms, etc.), without the need to use professional software to process the audio in the video before use, which can improve the user experience.

[0198] FIG5A is a schematic diagram illustrating an exemplary video shooting process.

[0199] S501: Receive a first shooting operation.

[0200] S502 : Receive a response to a first shooting operation, and record first target audio data and first other audio data, where the first target audio data is audio data of a target object, and the first other audio data is audio data of other objects.

[0201] S503: Receive a second shooting operation.

[0202] 6A to 6G are schematic diagrams of exemplary mobile phone interfaces.

[0203] 6A , exemplarily, 601 is the main interface of the mobile phone, which includes one or more controls, including but not limited to: application icons (for example, the application icon of Huawei video application, the application icon of browser application, the application icon 602 of camera application), network logo, power logo, etc.

[0204] Continuing with FIG6A , in one possible scenario, a user clicks on the camera application icon 602, and the phone displays the camera application interface 603 in response to the user's operation, as shown in FIG6B . Exemplarily, the camera application interface 603 includes one or more controls, including but not limited to: a camera switching option, a flash option, an AI shooting option, a settings option, a night scene option, a portrait option, a photo option, and a video recording option 604. Furthermore, the camera application interface 603 may also include a preview box that displays an image captured by the camera.

[0205] 6B , in one possible embodiment, the user clicks on the recording option 604, and the mobile phone displays the recording interface 605 in response to the user's operation, as shown in FIG6C . The recording interface 605 includes one or more controls, including but not limited to: a focus control option, a flash option, a setting option, a recording start option 607, a camera switching option, an image viewing option, and a shooting mode prompt option 606. Exemplarily, the shooting mode prompt option 606 includes one or more controls, including but not limited to: a confirmation option, a cancel option, and a prompt message (such as "whether to shoot a video for the target object"). In addition, the recording interface 605 may also include a preview box, which displays the image captured by the camera.

[0206] 6C , for example, when a user needs to shoot a video of a target object (e.g., the application scenarios of FIG. 4A to FIG. 4D ), the user can click the confirmation option in the shooting mode prompt option 606. The mobile phone responds to the user's operation behavior and uses the video shooting method involved in this application to perform video recording (or referred to as video shooting) (i.e., execute S504) and display the video shooting interface 608, as shown in FIG. 6D . Alternatively, after the user clicks the confirmation option in the shooting mode prompt option 606, the mobile phone responds to the user's operation behavior and cancels the display of the shooting mode prompt option 606; then, the user can click the recording start option 607. The mobile phone responds to the user's operation behavior and executes S504 and displays the video shooting interface 608, as shown in FIG. 6D .

[0207] 6C , for example, when the user does not need to shoot a video of the target object, the user can directly click the recording start option 607. In response to the user's operation, the mobile phone uses the existing video shooting method to shoot the video (i.e., execute S502) and displays the video shooting interface 608, as shown in FIG6D . Alternatively, when the user does not need to shoot a video of the target object, the user clicks the cancel option in the shooting mode prompt option 606. In response to the user's operation, the mobile phone cancels the display of the shooting mode prompt option 606; then, the user can click the recording start option 607. In response to the user's operation, the mobile phone uses the existing video shooting method to shoot the video (i.e., execute S502) and displays the video shooting interface 608, as shown in FIG6D .

[0208] The operation of clicking the record start option 607 can be referred to as the first shooting operation. Alternatively, the operation of clicking the shooting mode prompt option 606 and the operation of clicking the record start option 607 can be referred to as the first shooting operation. Furthermore, the conventional video shooting method can record the first target audio data, the first other audio data, and the first target video data.

[0209] Referring again to FIG. 6B , in one possible embodiment, a user clicks on the record option 604. In response to the user's operation, the mobile phone displays a record interface 609, as shown in FIG. 6E . The record interface 609 includes one or more controls, including but not limited to: a focus control option, a flash option, a settings option, a recording start option 611, a camera switch option, an image view option, and a target object capture option 610.

[0210] 6E , illustratively, when the user does not need to shoot a video for the target object, the user can directly click the recording start option 611. The mobile phone responds to the user's operation behavior and uses the video shooting method of the prior art (i.e., executes S502) to shoot the video and displays the video shooting interface 608, as shown in FIG6D . When the user needs to shoot a video for the target object, the user clicks the target object shooting option 610. The mobile phone responds to the user's operation behavior and executes S504 and displays the video shooting interface 608, as shown in FIG6D . Alternatively, when the user needs to shoot a video for the target object, the user clicks the target object shooting option 610 and the recording start option 611 in sequence. The mobile phone responds to the user's operation behavior and executes S504 and displays the video shooting interface 608, as shown in FIG6D .

[0211] Referring again to FIG. 6A , in one possible approach, a user taps the camera application icon 602. In response to the user's operation, the phone displays the camera application interface 612, as shown in FIG. Exemplarily, the camera application interface 612 includes one or more controls, including but not limited to: a camera switch option, a flash option, an AI shooting option, a settings option, a night scene option, a portrait option, a photo option, a normal video recording option, a target video recording option 613, and the like. Furthermore, the camera application interface 612 may also include a preview box that displays images captured by the camera.

[0212] Continuing with FIG6F , illustratively, when a user wishes to capture a video of a target object, the user clicks on the target recording option 613. In response to the user's operation, the mobile phone displays a recording interface 614, as shown in FIG6G . Recording interface 614 includes one or more controls, including but not limited to: a focus control option, a flash option, a settings option, a recording start option 615, a camera switching option, and an image viewing option. Next, the user clicks on the recording start option 615. In response to the user's operation, the mobile phone executes S504 and displays the video capture interface.

[0213] 6F , illustratively, when the user does not need to shoot a video for the target object (i.e., does not distinguish between the target object and other objects, or is called normal shooting), the normal video recording option can be clicked. In this case, the first shooting operation can be the operation of clicking the normal video recording option.

[0214] It should be noted that the second shooting operation in S503 may be as follows:

[0215] In one possible manner, the second shooting operation of the user in S503 may be an operation of clicking the confirmation option in the shooting mode prompt option 606 in FIG6C .

[0216] In one possible manner, the second shooting operation of the user in S503 may include an operation of clicking the confirmation option in the shooting mode prompt option 606 in FIG6C and an operation of clicking the recording start option 607 in FIG6C .

[0217] In one possible manner, the second shooting operation of the user in S503 may be an operation of clicking the target object shooting option 610 in FIG6E .

[0218] In one possible manner, the second shooting operation of the user in S503 may include an operation of clicking the target object shooting option 610 in FIG. 6E and an operation of clicking the recording start option 611 in FIG. 6E .

[0219] In one possible manner, the second shooting operation of the user in S503 may include an operation of clicking the target video recording option 613 in FIG. 6F and an operation of clicking the recording start option 615 in FIG. 6G .

[0220] In one possible manner, the second shooting operation of the user in S503 may be an operation of clicking the recording start option 615 in FIG6G .

[0221] S504 , in response to the second shooting operation, recording second target audio data and second other audio data, where the second target audio data is audio data for the target object, and the second other audio data is audio data for other objects.

[0222] For example, after receiving the second shooting operation from the user, the mobile phone may call the camera to capture the first video data (ie, the target video data or the second target video data) in response to the second shooting operation. The image in the first video data may include the target object.

[0223] Exemplarily, in response to the second shooting operation, sound pickup is performed on the target object to collect first audio data, wherein the first audio data includes the second target audio data and the second other audio data.

[0224] It should be noted that, referring to FIG6B , when a user clicks on the recording option 604, the mobile phone responds to the user operation by displaying the recording interface 605 in FIG6C (or the recording interface 609 in FIG6E ), and the camera and microphone of the mobile phone are also activated; at this time, the camera of the mobile phone can collect video data (hereinafter referred to as initial video data for the sake of distinction), and the microphone can also collect audio data (hereinafter referred to as initial audio data for the sake of distinction). Also, referring to FIG6F , when a user clicks on the target recording option 613, the mobile phone responds to the user operation by displaying the recording interface 614 in FIG6G , and the camera and microphone of the mobile phone are also activated; at this time, the camera of the mobile phone can collect initial video data, and the microphone of the mobile phone collects initial audio data.

[0225] It should also be noted that, in the process of the mobile phone's microphone collecting initial audio data or first target audio data (taking the collection of initial audio data as an example), the beam of the mobile phone's microphone is a fixed beam (it can also be said that the mobile phone's microphone is omnidirectionally picking up sound). Wherein, when the camera used to collect initial video data is a rear camera, the beam shape of the fixed beam can be a first preset shape, and the beam pointing angle of the fixed beam can be a first preset angle. Wherein, the first preset shape and the first preset angle can be obtained by training in advance (before the terminal device leaves the factory), and the first preset shape and the first preset angle are the beam shape and beam pointing angle with the best sound pickup effect when shooting with the rear camera (wherein, during the training process, the rear camera is used to shoot video data and the microphone is used to collect audio data; then, the signal-to-noise ratio of the audio data is determined; when the signal-to-noise ratio is lower than the signal-to-noise ratio threshold, the beam shape and beam pointing angle of the microphone are adjusted until the optimal beam shape and beam pointing angle are obtained). When the camera used to capture the initial video data is a front-facing camera, the beam shape of the fixed beam may be a second preset shape, and the beam pointing angle of the fixed beam may be a second preset angle. The second preset shape and the second preset angle may also be pre-trained, and the second preset shape and the second preset angle are the beam shape and beam pointing angle that achieve optimal sound pickup when shooting with the front-facing camera.

[0226] For ease of description, a three-dimensional coordinate system based on a mobile phone is described below. Referring to FIG5B , for example, the center of the mobile phone can be used as the origin O, a line perpendicular to the back of the mobile phone and passing through the origin O can be used as the X-axis, a line perpendicular to the top of the mobile phone and passing through the origin O can be used as the Z-axis, and a line perpendicular to the side of the mobile phone and passing through the origin O can be used as the Y-axis.

[0227] FIG. 5C is a schematic diagram illustrating an exemplary beam of a microphone.

[0228] For example, assume that the shooting direction of the rear camera (the X-axis direction in Figure 5B) is 0 degrees. In Figure 5C, the solid line represents the left beam, and the dashed line represents the right beam. The microphone beam can be synthesized by combining the left and right beams. The beam pointing angle of the left beam is 45 degrees, and the beam pointing angle of the right beam is 345 degrees. At this time, the beam pointing angle of the microphone beam synthesized by the left and right beams is 0 degrees; the shapes of the left and right beams are shown in Figure 5C.

[0229] Exemplarily, after the mobile phone receives the user's second shooting operation, the terminal device can respond to the second shooting operation and instruct the image module (or image processing module, which can be a module in the processor) to determine the video focal length information based on the first video data, wherein the video focal length information may include the distance between the target object and the terminal device (hereinafter referred to as the first distance). Specifically, the image module can call a pre-trained image processing model (wherein, the image processing model can be a model in the prior art; this can also be described as calling the image processing model to load pre-trained model parameters) based on the first video data to determine the first distance. Specifically, the image processing model can perform target detection based on the first video data to determine the first distance between the target object and the terminal device.

[0230] Subsequently, the image module can output the video focal length information to the audio module (or audio processing module or audio module), and the audio module can perform directional sound pickup for the target object based on the first distance in the video focal length information. Specifically, the audio module can call a pre-trained beam adjustment algorithm (which can also be described as calling a beam adjustment algorithm to load pre-trained algorithm parameters) to determine the first angle of the target object relative to the terminal device based on the first distance; then, according to the first angle, adjust the beam state of the microphone so that the target pickup area of ​​the microphone contains the target object and contains as few other objects as possible, so as to achieve directional sound pickup of the target object.

[0231] Illustratively, the first angle of the target object relative to the terminal device can be as shown in FIG5B , where the first angle is the angle between the line connecting the origin O and the target object and the X-axis; the first angle can be referred to as a first horizontal angle. Illustratively, at least three microphones in the terminal device can determine the distance, horizontal angle, and elevation angle of any point in three-dimensional space relative to the terminal device; furthermore, the terminal device can derive the first angle of the target object relative to the terminal device based on the first distance of the target object relative to the terminal device.

[0232] Exemplarily, the beam state may include a beam pointing angle and a beam shape. Specifically, the beam state of the microphone may be adjusted based on a first angle, a preset rule, and a first distance. For example, the beam pointing angle of the microphone may be adjusted to the first angle, and the beam shape of the microphone may be adjusted to the first beam shape based on the preset rule and the first distance.

[0233] Exemplarily, the preset rule is a rule that is pre-trained and learned by the beam adjustment algorithm, with the goal of increasing the sound pickup effect of the target object. For example, the preset rule is: when the distance is X1 meters, the short axis of the left beam is a1 mm and the short axis of the right beam is a2 mm, and / or, the long axis of the left beam is b1 mm and the long axis of the right beam is b2 mm; when the distance is X2 meters, the short axis of the left beam is a3 mm and the short axis of the right beam is a4 mm, and / or, the long axis of the left beam is b3 mm and the long axis of the right beam is b4 mm. Wherein, X1, X2, a1-a4, and b1-b4 are all positive integers.

[0234] FIG. 5D is a schematic diagram illustrating an exemplary beam pattern of a microphone.

[0235] Referring to Figure 5D, for example, assuming that the first angle is 30 degrees, after the beam state of the microphone is adjusted according to the first angle, the preset rule and the first distance, the beam pointing angle of the left beam is 60 degrees, the beam pointing angle of the right beam is 0 degrees, and the beam pointing angle of the beam of the microphone synthesized by the left beam and the right beam is 30 degrees; the shapes of the left beam and the right beam are shown in Figure 5D.

[0236] Compared to FIG5C , the major axis of the left beam (or right beam) in FIG5D remains unchanged, and the minor axis of the left beam (or right beam) in FIG5D is shorter.

[0237] It should be noted that the directional sound pickup performed by the audio module according to the first angle in the video focal length information output by the image module is the first directional sound pickup.

[0238] During the process of shooting a video of a target object, the target object may be in motion. Therefore, after the first pointing pickup, the audio module can determine the audio data of the target object from the first audio data collected after the first pointing pickup (wherein the first audio data may include the audio data of the target object (i.e., the first target audio data) and the audio data of other objects (i.e., the first other audio data)) according to the first distance in the video focal length information. Specifically, the audio module can first call a pre-trained first audio processing model (wherein the first audio processing model can be a model in the prior art; this can also be described as calling the first audio processing model to load pre-trained model parameters), perform sound source positioning based on the first audio data, and determine the first position information of multiple objects (including the target object and other objects) contained in the first audio data. The first position information of a single object can include a third pitch angle (i.e., the angle between the line between the single object and the origin and the Z axis), a third horizontal angle (i.e., the angle between the line between the single object and the origin and the X axis), and a third distance (i.e., the distance between the single object and the terminal device). Next, a pre-trained second audio processing model is called (the second audio processing model can be a model from the prior art; this can also be described as calling the second audio processing model to load pre-trained model parameters) to determine the audio data of the target object based on the first distance and the third distance of the multiple objects. For example, the audio data of the object whose third distance is closest to the first distance is used as the audio data of the target object. In addition, the second audio processing model can also extract feature information from the audio data of the target object to obtain the feature information of the target object, which is used to determine the audio data of the target object from the subsequently collected first audio data.

[0239] Afterwards, the audio module can track the sound source of the target object. Exemplarily, the audio module can call the second audio processing model in real time (or periodically) to extract features from the audio data of multiple objects in the subsequently collected first audio data to obtain feature information of the multiple objects. Then, the feature information of the multiple objects is compared with the feature information of the target object to determine the audio data of the target object from the subsequently collected first audio data; for example, the audio data of the object whose feature information is closest to the feature information of the target object is used as the audio data of the target object. Afterwards, the first audio processing model can localize the sound source based on the audio data of the target object and determine the second position information of the target object (that is, the result of sound source tracking). The second position information of the target object may include a second pitch angle (i.e., the angle between the line connecting the target object and the origin and the Z axis), a second horizontal angle (i.e., the angle between the line connecting the target object and the origin and the X axis), and a second distance (i.e., the distance between the target object and the terminal device). Then, the audio module calls the beam adjustment algorithm to determine whether the position of the target object has changed based on the second position information of the target object at the current moment (or current cycle) and the second position information of the target object at the previous moment (or previous cycle).

[0240] Exemplarily, when the second position information of the target object at the current moment (or current cycle) is greater than the difference information between the second position information of the target object at the previous moment (or previous cycle) and the second position information of the target object at the previous moment (or previous cycle) is greater than the difference threshold, it is determined that the position of the target object has changed; at this time, the audio module can call the beam adjustment algorithm to readjust the beam state of the microphone according to the second position information of the target object at the current moment (or current cycle) (that is, the changed position information) (or understand it as redetermining the first distance and the first angle, that is, taking the second distance as the first distance, and taking the second horizontal angle as the first angle) to achieve directional pickup of the target object after the position change. By analogy, in the process of shooting a video of the target object, the audio module can capture the position of the target object, and continuously readjust the beam state of the microphone according to the current position information of the target object (that is, the second position information of the target object at the current moment (or current cycle)) to achieve dynamic directional pickup of the target object.

[0241] Specifically, the beam state of the microphone can be adjusted according to the second angle, the second distance and the preset rule; for example, the beam pointing angle of the microphone can be adjusted to the second angle, and the beam shape of the microphone can be adjusted according to the preset rule and the second distance.

[0242] FIG. 5E is a schematic diagram illustrating an exemplary beam of a microphone.

[0243] Referring to Figure 5E, for example, assuming that the second angle is 330 degrees, after the beam state of the microphone is adjusted according to the second angle, the second distance and the preset rule, the beam pointing angle of the left beam is 0 degrees, the beam pointing angle of the right beam is 300 degrees, and the beam pointing angle of the beam of the microphone synthesized by the left beam and the right beam is 330 degrees; the shapes of the left beam and the right beam are shown in Figure 5E.

[0244] Relative to Figure 5D, the short axes of the left beam (right beam) in Figure 5E and Figure 5D are equal, and the long axes of the left beam (right beam) in Figure 5E and Figure 5D are equal; the beam pointing angle of the beam of the microphone in Figure 5D is 30 degrees, and the beam pointing angle of the beam of the microphone in Figure 5E is 330 degrees.

[0245] Exemplarily, when the difference between the second position information of the target object at the current moment (or current period) and the second position information of the target object at the previous moment (or previous period) is less than or equal to the difference threshold, the first audio processing model can continue to be called to perform real-time (or periodic) sound source positioning based on the audio data of the target object to determine the second position information of the target object at the next moment (or next period).

[0246] 5F and 5G are schematic diagrams exemplarily illustrating the first target audio data and the first other audio data, wherein user A is the target object.

[0247] For example, the audio data of user A (as shown in FIG5F , i.e., the first target audio data) and the audio data of user B (as shown in FIG5G , i.e., the first other audio data) are similar in amplitude to the audio data of user A and the audio data of user B. Referring to FIG5F and FIG5G , the amplitude of the audio data of user A and the amplitude of the audio data of user B are similar in amplitude to each other.

[0248] 5H and 5I are schematic diagrams exemplarily showing the second target audio data and the second other audio data.

[0249] For example, after point-to-point audio pickup of user A, the collected first audio data includes user A's audio data (as shown in FIG5H , i.e., the second target audio data) and user B's audio data (as shown in FIG5I , i.e., the second other audio data). Referring to FIG5H and FIG5I , the amplitude of user A's audio data is greater than the amplitude of user B's audio data.

[0250] It can be seen that after the targeted sound pickup is performed, the signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data; therefore, the sound pickup effect of the target object is better.

[0251] In this way, during the video shooting process, directional sound pickup of the target object can suppress the sounds of other objects, thereby improving the signal-to-noise ratio of the target object's audio data. Therefore, the sound pickup effect of the target object is better, which can improve the quality (i.e., restoration) and clarity (i.e., recognizability) of the target object's audio in the captured video. In this way, the video can be used directly later (for example, forwarded, shared to social platforms, etc.), without using professional software to process the second target audio data in the video before use, which can simplify user operations and improve user experience.

[0252] For example, the first audio data and the first video data can be stored in the same multimedia file, which is a video shot for the target object and can also be called an audio-video file (or audio-video data). Each frame of audio in the first audio data has a timestamp, and each frame of image in the first video data has a timestamp, and the images and audio captured at the same time have the same timestamp; thus, when playing the video shot for the target object, the playback module can play a frame of image and a frame of audio with the same timestamp at the same time, thereby achieving audio-visual synchronization.

[0253] FIG. 7 is a schematic diagram illustrating an exemplary video shooting process.

[0254] S701: Receive a second shooting operation.

[0255] S702 : In response to a second shooting operation, first video data is acquired.

[0256] S703 : In response to the second shooting operation, performing directed sound pickup on the target object to collect first audio data.

[0257] For example, S701 to S703 may refer to the description of S503 to S504 above, which will not be repeated here.

[0258] S704: Receive a focus adjustment operation from the user.

[0259] 8A to 8E are schematic diagrams of exemplary mobile phone interfaces.

[0260] For example, when capturing a moving target object, the distance between the target object and the terminal device may change; for example, the distance between the target object and the terminal device may change from far to near, or vice versa. Therefore, to ensure that the size of the target object in the captured first video data remains unchanged (or substantially unchanged), the user may perform a focus adjustment operation to adjust the shooting distance.

[0261] For example, when the distance between the target object and the terminal device changes from far to near, the target object occupies a larger proportion in the shooting interface 608 as shown in Figure 8A; the user can click the focus reduction option 801 in the video shooting interface 608 to reduce the target object (that is, reduce the proportion of the target object in the shooting interface 608, as shown in Figure 8B).

[0262] For example, when the distance between the target object and the terminal device changes from near to far, the target object occupies a smaller proportion in the shooting interface 608, as shown in FIG8C ; the user can click the focus increase option 802 in the video shooting interface 608 to zoom in on the target object (i.e., increase the proportion of the target object in the shooting interface 608, as shown in FIG8B ).

[0263] For example, when the distance between the target object and the terminal device changes from far to near, the target object occupies a larger proportion in the shooting interface 608 as shown in Figure 8D; the user can perform a pinch operation (i.e., shortening the distance between two fingers) in the video shooting interface 608 to reduce the target object (i.e., reducing the proportion of the target object in the shooting interface 608, as shown in Figure 8B).

[0264] For example, when the distance between the target object and the terminal device changes from near to far, the target object occupies a smaller proportion in the shooting interface 608, as shown in FIG8E ; the user can perform a stretching operation (i.e., an operation of expanding the distance between two fingers) in the video shooting interface 608 to enlarge the target object (i.e., increase the proportion of the target object in the shooting interface 608, as shown in FIG8B ).

[0265] That is, the focus adjustment operation in S704 may include any one of clicking the focus reduction option 801 , clicking the focus increase option 802 , a pinch operation, or a stretch operation.

[0266] S705 , in response to the focus adjustment operation, adjust the beam state of the microphone.

[0267] Exemplarily, when the focus adjustment operation is clicking on the focus reduction option 801 or performing a pinch operation, the terminal device can, in response to the focus adjustment operation, instruct the audio module to call a beam adjustment algorithm to adjust the beam state of the microphone to reduce the microphone's pickup distance, thereby improving the sound pickup effect for the target object; and reduce the volume of the audio data of the target object in the first audio data. Specifically, the beam pointing angle of the microphone can be ensured to remain unchanged, the short axis of the left beam and the short axis of the right beam can be shortened, and the long axis of the left beam and the long axis of the right beam can be increased.

[0268] FIG8G is a schematic diagram showing an exemplary beam of a microphone.

[0269] 5D and 8G , when the focus adjustment operation is clicking the focus reduction option 801 or a pinching operation, after the beam state of the microphone is adjusted, the beam pointing angle of the microphone remains 30 degrees, the short axes of the left and right beams become shorter, and the long axes of the left and right beams become longer.

[0270] For example, when the focus adjustment operation is clicking the focus increase option 802 or performing a zoom operation, the terminal device can, in response to the focus adjustment operation, instruct the audio module to call a beam adjustment algorithm to adjust the microphone beam state, thereby increasing the microphone's sound pickup distance, achieving long-distance sound pickup, and improving the sound pickup effect for the target object; and increasing the volume of the audio data of the target object in the first audio data. Specifically, the microphone beam direction can be ensured to remain unchanged, shortening the short axis of the left beam and the short axis of the right beam.

[0271] FIG8F is a schematic diagram showing an exemplary beam.

[0272] 5D and 8F , when the focus adjustment operation is clicking the focus increase option 802 or the stretching operation, after the beam state of the microphone is adjusted, the beam pointing angle of the microphone remains 30 degrees, the short axes of the left beam and the right beam become shorter, and the long axes of the left beam and the right beam do not change.

[0273] Exemplarily, the image module can output the video focal length information (wherein, the video focal length information can include focal length information, which can be the target focal length or the focal length change corresponding to the focal length adjustment operation; for example, the current focal length of the camera is 20mm, and the focal length change corresponding to the focal length adjustment operation is +10mm, then the target focal length is 30mm) to the beam adjustment algorithm, and the beam adjustment algorithm adjusts the beam state of the microphone according to the focal length information.

[0274] S706: Perform audio collection based on the adjusted beam to obtain second audio data.

[0275] Exemplarily, after the beam state of the microphone is adjusted, the microphone can collect audio according to the adjusted beam to obtain the second video data.

[0276] S707 , in response to the focus adjustment operation, adjust the focus of the camera.

[0277] Exemplarily, the image module may determine a target focal length according to the focal length adjustment operation and the current focal length of the camera; and then adjust the focal length of the camera according to the target focal length.

[0278] S708 , performing video capture based on the adjusted focal length to obtain second video data.

[0279] Afterwards, the camera can capture video according to the adjusted focal length to obtain second audio data, wherein the second audio data includes audio data of the target object and audio data of other objects.

[0280] In the embodiment of FIG. 7 , the first video data and the first audio data, the second video data and the second audio data may be stored as one multimedia file.

[0281] It should be noted that for a terminal device that includes a front camera and a rear camera, the user can usually manually adjust the focal length during video shooting using the rear camera; therefore, the embodiment of Figure 7 can be applied to scenarios where the terminal device uses the rear camera for video shooting.

[0282] FIG9 is a schematic diagram illustrating an exemplary video shooting process.

[0283] S901: Receive a second shooting operation.

[0284] S902 : In response to a second shooting operation, collect first video data.

[0285] S903 : In response to the second shooting operation, performing directed sound pickup on the target object to collect first audio data.

[0286] For example, S901 to S903 may refer to the description of S503 to S504 above, which will not be repeated here.

[0287] S904: When it is detected that the distance between the target object and the terminal device changes, adjust the beam state of the microphone.

[0288] Exemplarily, the terminal device may detect in real time (or periodically) whether the distance between the target object and the terminal device changes, to determine whether to adjust the beam state of the microphone.

[0289] In one possible approach, the audio module may call a beam adjustment algorithm to detect whether the distance between the target object and the terminal device has changed based on the first audio data. Specifically, the beam adjustment algorithm may determine the volume of the audio data of the target object in the first audio data; when the difference between the volume of the audio data of the target object in the first audio data determined this time and the volume of the audio data of the target object in the first audio data determined last time is greater than a volume threshold, it may be determined that the distance between the target object and the terminal device has changed.

[0290] Among them, if the difference between the volume of the audio data of the target object in the first audio data determined this time and the volume of the audio data of the target object in the first audio data determined last time is greater than 0, it means that the distance between the target object and the terminal device has become closer. At this time, the beam state of the microphone can be adjusted to reduce the sound pickup distance of the microphone to improve the sound pickup effect for the target object; and the volume of the audio data of the target object in the first audio data can be reduced. If the difference between the volume of the audio data of the target object in the first audio data determined this time and the volume of the audio data of the target object in the first audio data determined last time is less than 0, it means that the distance between the target object and the terminal device has become farther. At this time, the beam state of the microphone can be adjusted to increase the sound pickup distance of the microphone to achieve long-distance sound pickup to improve the sound pickup effect for the target object; and the volume of the audio data of the target object in the first audio data can be increased. For details, please refer to the description in the embodiment of Figure 7 above, which will not be repeated here.

[0291] In one possible approach, an image module may detect whether the distance between a target object and a terminal device has changed based on the first video data. Specifically, when determining that the focal length of the camera has not changed, the image module may determine the area of ​​the target object in the current frame of the first video data (hereinafter referred to as Area 1) and the area of ​​the target object in the previous frame of the first video data (hereinafter referred to as Area 2); when the difference between Area 1 and Area 2 is greater than an area threshold, it may be determined that the distance between the target object and the terminal device has changed.

[0292] Among them, if area 1 minus area 2 is greater than 0, it means that the distance between the target object and the terminal device is getting closer. At this time, the beam state of the microphone can be adjusted to reduce the microphone's sound pickup distance to improve the sound pickup effect for the target object; and the volume of the audio data of the target object in the first audio data can be reduced. If area 1 minus area 2 is less than 0, it means that the distance between the target object and the terminal device is getting farther. At this time, the beam state of the microphone can be adjusted to increase the microphone's sound pickup distance to achieve long-distance sound pickup to improve the sound pickup effect for the target object; and the volume of the audio data of the target object in the first audio data can be increased. For details, please refer to the description in the embodiment of Figure 7 above, and no further details will be given here.

[0293] For example, during the real-time sound source tracking process, it may also be determined whether the distance between the target object and the shooting device has changed.

[0294] S905: Perform audio collection based on the adjusted beam to obtain third audio data.

[0295] In the embodiment of FIG. 9 , the first video data and the first audio data, the third video data and the third audio data may be stored as one multimedia file.

[0296] Exemplarily, the third audio data includes audio data of the target object and audio data of other objects.

[0297] It should be noted that for a terminal device including a front camera and a rear camera, the user generally does not adjust the focus (or does not support user focus adjustment) during video capture with the front camera; thus, the embodiment of FIG9 can be applied to scenarios where the terminal device uses the front camera for video capture. It should be understood that the embodiment of FIG9 can also be applied to scenarios where the terminal device uses the rear camera for video capture.

[0298] FIG10 is a schematic diagram illustrating an exemplary video shooting process.

[0299] S1001: Receive a second shooting operation and receive a selection operation for a target object.

[0300] In conjunction with the embodiment of FIG. 5A and with reference to FIG. 6C , illustratively, when a user clicks the confirmation option in the shooting mode prompt option 606 on the recording interface 605 , the mobile phone, in response to the user's operation, cancels the display of the shooting mode prompt option 606 and then displays an object selection prompt item 1101 , as shown in FIG. 11A . The object selection prompt item 1101 in FIG. 11A may include one or more controls, including but not limited to: a confirmation option and a cancel option. In addition, the object selection prompt item 1101 may also include a prompt message "Do you want to select the target object?"

[0301] 11A , illustratively, when the user determines the target object, the user can click the confirmation option in the object selection prompt item 1101 . The mobile phone cancels the display of the object selection prompt item 1101 in response to the user's operation. At this time, the displayed recording interface 605 is shown in FIG11B and FIG11C , and multiple objects are displayed in the preview box of the recording interface 605 .

[0302] 11B , illustratively, when the user wants to shoot one target object, he or she can click on a target object location in the preview box of the recording interface 605 . The mobile phone responds to the user's operation and displays a focus box 1102 in the recording interface 605 .

[0303] 11C , for example, when the user wants to shoot two target objects, he can click on the positions of the two target objects in the preview box of the recording interface 605 in sequence or simultaneously. The mobile phone responds to the user's operation and displays two focus boxes 1103 in the recording interface 605; and so on, which will not be repeated here.

[0304] Referring to Figure 6C again, for example, the user clicks the confirmation option in the shooting mode prompt option 606 and clicks the recording start option 607 in the video recording interface 605. The mobile phone responds to the user's operation and displays the video shooting interface 608. Then, it can display the object selection prompt item 1104, as shown in Figure 11D. The object selection prompt item 1104 is similar to the above-mentioned object selection prompt item 1101 and will not be repeated here.

[0305] Referring to Figure 11D, for example, when the user determines the target object, he can click the confirmation option in the object selection prompt item 1104. The mobile phone responds to the user's operation and cancels the display of the object selection prompt item 1104. At this time, multiple objects can be displayed in the video shooting interface 608, as shown in Figures 11E and 11F.

[0306] 11E , illustratively, when the user wants to shoot one target object, he can click on the location of a target object in the video shooting interface 608 , and the mobile phone displays a focus frame 1105 in the video shooting interface 608 in response to the user's operation.

[0307] 11F , for example, when the user wants to shoot two target objects, he can click on the locations of the two target objects in the video shooting interface 608. The mobile phone responds to the user's operation and displays two focus frames 1106 in the video shooting interface 608; and so on, which will not be repeated here.

[0308] That is to say, this application can shoot videos for one or more target objects.

[0309] Referring to FIG11A , for example, a user clicks the cancel option in object selection prompt 1101. In response to the user's operation, the mobile phone cancels the display of object selection prompt 1101. At this point, it can be determined that the user has not selected a target object. The terminal device can then use the object with the largest area in the image captured by the camera as the target object, or the object closest to the center of the image as the target object. This application does not limit this. Subsequently, the user clicks the recording start option 607. In response to the user's operation, the mobile phone executes S1002 and S1003 and displays the video capture interface 608, as shown in FIG6D .

[0310] 11D , exemplarily, the user clicks the cancel option in the object selection prompt item 1104 , and the mobile phone cancels the display of the object selection prompt item 1104 in response to the user's operation; at this time, it can be determined that the user has not selected the target object, and then, the target object can be determined by the terminal device, which will not be repeated here.

[0311] Based on the above description, in one possible manner, the selection operation for the target object may include the operation of clicking the confirmation option in the object selection prompt item 1101 (or the object selection prompt item 1104), and the operation of clicking the location of the target object in the recording interface 605 of Figure 11B or Figure 11C (or clicking the video shooting interface 608 of Figure 11E or Figure 11F).

[0312] It should be understood that the present application does not limit the order in which the user performs the shooting operation and the selection operation.

[0313] It should be understood that the focus frames in FIG. 11B , FIG. 11C , FIG. 11E , and FIG. 11F may not be displayed, and this application does not impose any limitation on this.

[0314] It should be understood that in one possible way, the user clicks on the location of the target object in the recording interface 605 of Figure 11B or Figure 11C (or clicks on the video shooting interface 608 of Figure 11E or Figure 11F), and the mobile phone responds to the user's operation and can also display a prompt message (such as "You have successfully selected the target object").

[0315] 5A and FIG6C , exemplarily, the user clicks the confirmation option in the shooting mode prompt option 606 in the video recording interface 605 , and the mobile phone responds to the user's operation behavior, cancels the display of the shooting mode prompt option 606 , and displays the object selection prompt item 1107 , as shown in FIG11G .

[0316] 11G , illustratively, when the user has determined the target object, they can click on object selection prompt 1107, or click on an area of ​​the preview frame of the recording interface 605 other than object selection prompt 1107. In response to the user's operation, the mobile phone cancels the display of object selection prompt 1107. At this point, the displayed recording interface 605 is shown in FIG11B and FIG11C , with multiple objects displayed in the preview frame of the recording interface 605. Alternatively, the user may choose not to click on object selection prompt 1107, or not to click on an area of ​​the preview frame of the recording interface 605 other than object selection prompt 1107. After waiting for a few seconds, the mobile phone cancels the display of object selection prompt 1107. At this point, the displayed recording interface 605 is shown in FIG11B and FIG11C , with multiple objects displayed in the preview frame of the recording interface 605.

[0317] Referring to Figure 11G, when the user clicks on the object selection prompt item 1107, or clicks on the area other than the object selection prompt item 1102 in the preview box of the recording interface 605, but does not click on the location of the target object in the preview box of the recording interface 605, but directly clicks on the recording start option 607, it can also be determined that the user has not selected the target object. At this time, the target object can be determined by the terminal device, which will not be repeated here.

[0318] Referring to Figure 6C again, for example, the user clicks the confirmation option in the shooting mode prompt option 606 and clicks the recording start option 607 in the video recording interface 605. In response to the user's operation, the mobile phone displays the video shooting interface 608 and then displays the object selection prompt item 1108, as shown in Figure 11H.

[0319] 11H , illustratively, when the user has determined the target object, they can click on the object selection prompt 1108, or click on an area other than the object selection prompt 1108 in the preview frame of the video shooting interface 608. In response to the user's operation, the mobile phone cancels the display of the object selection prompt 1108. At this time, the displayed video shooting interface 608 is shown in Figures 11E and 11F , where multiple objects are displayed in the preview frame of the video shooting interface 608. In addition, the user can also choose not to click on the object selection prompt 1108, or not to click on an area other than the object selection prompt 1108 in the preview frame of the video shooting interface 608. After waiting for a few seconds, the mobile phone will cancel the display of the object selection prompt 1108. At this time, the displayed video shooting interface 608 is shown in Figures 11E and 11F , where multiple objects are displayed in the preview frame of the video shooting interface 608.

[0320] Referring to Figure 11H, when the user clicks on the object selection prompt item 1108, or clicks on the area other than the object selection prompt item 1108 in the preview box of the video shooting interface 608, but does not click on the location of the target object in the video shooting interface 608, it can be determined that the user has not selected the target object. At this time, the target object can be determined by the terminal device, which will not be repeated here.

[0321] In one possible embodiment, the target object selection operation may include the operation of clicking the object selection prompt item 1107 (or the object selection prompt item 1108), and the operation of clicking the location of the target object in the recording interface 605 of Figure 11B or Figure 11C (or clicking the video shooting interface 608 of Figure 11E or Figure 11F).

[0322] In one possible embodiment, the target object selection operation may include the operation of clicking on an area in the preview box of the recording interface 605 other than the object selection prompt item 1107 (or the object selection prompt item 1108), and the operation of clicking on the location of the target object in the recording interface 605 of Figure 11B or Figure 11C (or clicking on the video shooting interface 608 of Figure 11E or Figure 11F).

[0323] S1002 : In response to a second shooting operation and a target object selection operation, first video data is collected.

[0324] S1003 : In response to the second shooting operation and the target object selection operation, performing directed sound pickup on the target object to collect first audio data.

[0325] For example, S1002 to S1003 may refer to the description of S503 to S504 above, which will not be repeated here.

[0326] It should be noted that when there are multiple target objects, the beam state of the microphone can be adjusted according to the position information of the multiple target objects so that the target pickup area of ​​the microphone is the intersection or union of the positions of the multiple target objects, or in other words, the target pickup area of ​​the microphone includes multiple target objects and as few other objects as possible.

[0327] It should be understood that it is also possible that the object selection prompt item does not need to be displayed, that is, the user does not need to select the target object, and the terminal device can determine the target object.

[0328] FIG12 is a schematic diagram illustrating an exemplary video shooting process.

[0329] S1201: Receive a second shooting operation.

[0330] S1202: In response to a second shooting operation, collect first video data.

[0331] S1203: In response to the second shooting operation, perform directed sound pickup on the target object to collect first audio data.

[0332] For example, S1201 to S1203 may refer to the description of S503 to S504 above, which will not be repeated here.

[0333] S1204: Receive a target object switching operation from the user.

[0334] 13A , illustratively, during the process of capturing a video of a target object, the captured video screen may be shown in a video capture interface 608 , where the target object is A. It should be noted that during the process of capturing a video of a target object, a focus frame 1301 in the video capture interface 608 may or may not be displayed. In order to illustrate the target object switching process, the present application shows the focus frame 1301.

[0335] 13B , for example, during a video capture of target object A, if the user needs to switch target objects (e.g., from A to B), the user can click on the location of user B in the preview box of the video capture interface 608. In response to the user's operation, the mobile phone displays a focus frame 1301 at the location of user B in the video capture interface 608, and executes S1205 and S1206. It should be understood that the focus frame in FIG13B may not be displayed, and this application does not limit this.

[0336] In addition, in one possible method, the user can click on the location of B in the preview box of the video shooting interface 608, and the mobile phone can also display a prompt message (such as "target object switching successful") in response to the user's operation behavior.

[0337] S1205 : In response to the target object switching operation, collect fourth video data for the switched target object.

[0338] Exemplarily, in response to the target object switching operation, the switched target object may be focused; then, fourth video data may be collected; and the fourth video data may include the switched target object.

[0339] S1206 : In response to the target object switching operation, perform directional sound pickup on the switched target object to collect fourth audio data.

[0340] For example, S1205 to S1206 may be similar to the above-mentioned process of S503 to S504, and will not be described in detail here.

[0341] Exemplarily, the fourth audio data may include the third target audio data and third other audio data.

[0342] In the embodiment of FIG12 , in one possible manner, the first video data and the first audio data, and the fourth video data and the fourth audio data, can be stored as one multimedia file. In another possible manner, the first video data and the first audio data can be stored as one multimedia file, and the fourth video data and the fourth audio data can be stored as another multimedia file.

[0343] In this way, during the video shooting process, the target object can be changed according to user needs without stopping the current video recording and reshooting another video, which simplifies user operations.

[0344] In one possible approach, the audio module may invoke the second audio processing model to perform audio post-processing (e.g., sound source separation of the target object, diffuse field noise extraction, mixing calculation, signal-to-noise ratio optimization, etc.) on the first audio data to obtain post-processed first audio data. The post-processed first audio data and the first video data may then be stored as a multimedia file.

[0345] In this way, during the video shooting process, by post-processing the audio data collected by the directional sound pickup, the signal-to-noise ratio of the audio data of the target object in the shot video can be further improved, thereby further improving the audio quality and clarity of the audio data of the target object in the shot video.

[0346] It should be understood that the present application may also perform audio post-processing on at least one of the second audio data, the third audio data, or the fourth audio data, and the present application does not impose any limitation on this.

[0347] It should be understood that any two embodiments in this application can be combined, and this application does not impose any limitation on this.

[0348] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the video shooting method in the above-mentioned embodiment.

[0349] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the video shooting method in the above-mentioned embodiment.

[0350] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the video shooting method in the above-mentioned method embodiments.

[0351] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0352] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0353] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0354] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0355] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0356] Any content of each embodiment of this application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0357] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0358] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0359] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0360] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0361] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A video shooting method, characterized in that: The method includes: The photographing device receives a first photographing operation; In response to the first shooting operation, the shooting device records first target audio data and first other audio data, wherein the first target audio data is audio data of a target object, and the first other audio data is audio data of other objects; The photographing device receives a second photographing operation; In response to the second shooting operation, the shooting device records second target audio data and second other audio data, wherein the second target audio data is audio data of the target object, and the second other audio data is audio data of the other object; The signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data.

2. The method according to claim 1, characterized in that The method further comprises: The photographing device receives a selection operation on the target object; In response to the selection operation on the target object, the photographing device determines the target object.

3. The method according to claim 1, characterized in that: The method further comprises: The photographing device determines any one of the object closest to the center of the photographing picture, the object closest to the photographing device, or the object with the largest occupancy in the photographing picture as the target object.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The photographing device receives a target object switching operation; In response to the target object switching operation, the shooting device records third target audio data and third other audio data, and the third target audio data is the audio data of the switched target object.

5. The method according to any one of claims 1 to 4, characterized in that: The target object is one or more.

6. The method according to any one of claims 1 to 5, characterized in that: The second target audio data is audio data collected by the shooting device through pointed sound pickup of the target object.

7. The method according to claim 6, characterized in that The method further comprises: In response to the second shooting operation, the shooting device records target video data, where the target video data is video data of the target object; The audio data collected by the shooting device through pointing and picking up sound from the target object specifically includes: The shooting device determines the target object based on the target video data; The photographing device determines a first distance according to the target object, where the first distance is the distance between the target object and the photographing device; The shooting device performs pointing and sound pickup on the target object according to the first distance to collect the second target audio data.

8. The method according to claim 7, characterized in that The method further comprises: The shooting device performs real-time sound source tracking on the target object; If the position of the target object changes, the shooting device re-determines the first distance and performs the step of pointing and picking up sound from the target object according to the first distance to collect the second target audio data.

9. A terminal device, characterized in that: The terminal device is used to execute the video shooting method executed by any shooting device in claim 1 to claim 8.

10. The terminal device according to claim 9, characterized in that: The terminal device includes at least three groups of microphones.

11. The terminal device according to claim 10, characterized in that: The distance between any two groups of microphones in the at least three groups of microphones is greater than a preset distance.

12. The terminal device according to claim 10 or 11, characterized in that: A set of microphones includes at least one microphone.

13. A computer storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the video shooting method according to any one of claims 1 to 8.

14. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a computer or a processor, the steps of the method according to any one of claims 1 to 8 are performed.

Images