Video shooting method and terminal equipment

Abstract

The embodiment of the invention provides a video shooting method and terminal equipment. The method comprises the following steps: firstly, the shooting equipment receives a first shooting operation; in response to the first shooting operation, the shooting equipment records first target audio data and first other audio data; then, the shooting equipment receives a second shooting operation; in response to the second shooting operation, the shooting equipment records second target audio data and second other audio data; wherein the signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data. Wherein the higher the signal-to-noise ratio of the audio data is, the better the audio quality (namely the reduction degree) and the better the definition (namely the recognition degree) are; that is to say, compared with the first target audio data, the audio quality and definition of the second target audio data are better.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a video shooting method and a terminal device. Background Art

[0002] With the popularization of terminal products (such as mobile phones, tablets, watches, etc.) and the advantages of terminal products such as portability and simple operation, more and more users will use terminal products to shoot videos, for example, shooting vlogs, recording lecture videos, recording live performances, live broadcasts, etc.

[0003] When the video shooting environment is noisy, or the sound source of the target object of the video shooting is not prominent, the audio quality and clarity of the video will be affected. When the video needs to be used later, professional software is needed to extract, edit and remove noise from the audio in the video. The video product efficiency is low, the production cycle is long, and the user experience is poor. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a video shooting method and terminal device. The method can improve the signal-to-noise ratio of the audio data of the target object in the shot video, thereby improving the quality (i.e., restoration degree) and clarity (i.e., recognizability) of the audio of the target object in the shot video.

[0005] In a first aspect, an embodiment of the present application provides a video shooting method, the method comprising: first, a shooting device receives a first shooting operation; in response to the first shooting operation, the shooting device records first target audio data and first other audio data, the first target audio data being the audio data of a target object, and the first other audio data being the audio data of other objects; then, the shooting device receives a second shooting operation; in response to the second shooting operation, the shooting device records second target audio data and second other audio data, the second target audio data being the audio data of the target object, and the second other audio data being the audio data of other objects; wherein the signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data.

[0006] The higher the signal-to-noise ratio of the audio data, the better the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio; that is, after receiving the second shooting operation, the audio quality (i.e., restoration) and clarity (i.e., recognizability) of the second target audio data recorded are better than those of the first target audio data. In this way, the video can be used directly later (for example, forwarded, shared to a social platform, etc.), without the need to use professional software to process the second target audio data in the video before use, which can simplify user operations and improve user experience.

[0007] Exemplarily, the first shooting operation may refer to an operation of clicking a recording start option in a recording interface of a shooting device.

[0008] Exemplarily, there may be multiple possibilities for the second shooting operation, which are specifically determined according to the user interface (UI) design of the shooting application in the shooting device. The present application does not limit what the second shooting operation is.

[0009] For example, the second shooting operation may be an operation of clicking on a confirmation option in a shooting mode prompt option that pops up on the recording interface after entering the recording interface.

[0010] For example, the second shooting operation may include, after entering the recording interface, clicking on a confirmation option in a shooting mode prompt option that pops up on the recording interface and clicking on a recording start option on the recording interface.

[0011] For example, the second shooting operation may include clicking a target recording option in an application interface of a camera application, and clicking a recording start option in a recording interface.

[0012] For example, the second shooting operation may be an operation of clicking a target object shooting option in the recording interface.

[0013] For example, the second shooting operation may include an operation of clicking a target object shooting option in the recording interface, and an operation of clicking a recording start option in the recording interface.

[0014] Exemplarily, in response to the first shooting operation, the shooting device may also record first target video data; and in response to the second shooting operation, the shooting device may also record second target video data.

[0015] Exemplarily, in response to the first shooting operation, the shooting device may call the front camera / rear camera to record the first target video data.

[0016] Exemplarily, in response to the second shooting operation, the shooting device may call the front camera / rear camera to record the second target video data.

[0017] Exemplarily, the first target video data, the first target audio data and the first other audio data may constitute the first audio-video data.

[0018] Exemplarily, the second target video data, the second target audio data and the second other audio data may constitute the second audio-video data.

[0019] Exemplarily, the photographing device may include, but is not limited to: mobile phones, tablet computers, smart watches, laptop computers, home cameras, etc., and this application does not impose any restrictions on this.

[0020] According to the first aspect, the method further includes: the photographing device receives a selection operation for the target object; in response to the selection operation for the target object, the photographing device determines the target object. In this way, the user can customize the selection of the target object to meet the user's personalized needs.

[0021] According to the first aspect, or any implementation of the first aspect above, the method further includes: the camera determines any one of the object closest to the center of the captured image, the object closest to the camera, or the object with the largest proportion in the captured image as the target object. In this way, the user does not need to select the target object, and the camera can automatically complete the selection of the target object, simplifying the user operation.

[0022] It should be understood that the shooting device may also use other methods to determine the target object, and this application does not limit this.

[0023] According to the first aspect, or any implementation of the first aspect above, the method further includes: the shooting device receives a target object switching operation; in response to the target object switching operation, the shooting device records third target audio data and third other audio data, and the third target audio data is the audio data of the switched target object. In this way, during the video shooting process, the target object can be changed according to user needs without stopping the current video recording and reshooting another video, which simplifies the user operation.

[0024] The third other audio data is audio data of other objects, and the other objects in the third other audio data are the same as or different from the other objects in the first other audio data and the second other audio data.

[0025] According to the first aspect, or any implementation of the first aspect above, the target object is one or more.

[0026] According to the first aspect, or any implementation of the first aspect above, the second target audio data is audio data collected by the shooting device for pointing and picking up sound from the target object. In this way, the signal-to-noise ratio of the second target audio data can be improved.

[0027] Exemplarily, the first target audio data may be audio data collected by a photographing device performing omnidirectional sound pickup.

[0028] According to the first aspect, or any implementation of the first aspect above, the method further includes: in response to the second shooting operation, the shooting device records target video data, the target video data is the video data of the target object; the shooting device performs pointed sound pickup on the target object to collect audio data, specifically including: the shooting device determines the target object based on the target video data; the shooting device determines a first distance according to the target object, the first distance is the distance between the target object and the shooting device; the shooting device performs pointed sound pickup on the target object according to the first distance to collect the second target audio data. In this way, pointed sound pickup of the target object can be achieved in the early stage of video shooting.

[0029] Exemplarily, the target video data here may refer to the second target video data mentioned above.

[0030] According to the first aspect, or any implementation of the first aspect above, the method further includes: the shooting device performs real-time sound source tracking on the target object; if the position of the target object changes, the shooting device re-determines the first distance, and performs the step of pointing and picking up the target object according to the first distance to collect the second target audio data. In this way, even if the position of the target object changes, the audio quality and clarity of the target object in the captured video can be guaranteed.

[0031] According to the first aspect, or any implementation of the first aspect above, the method can be applied to at least one scenario of live broadcast, shooting singing video, shooting sports video, shooting meeting video, shooting lecture video, shooting speech video or shooting interview video.

[0032] It should be understood that the present application can be applied to any scenario that requires video shooting, and the present application is not limited to this.

[0033] In a second aspect, an embodiment of the present application provides a video shooting method, which includes: first, a shooting device receives a second shooting operation; then, in response to the second shooting operation, the shooting device shoots first video data for a target object and performs pointed sound pickup for the target object to collect first audio data; the shooting device merges the first video data and the first audio data to obtain audio and video data.

[0034] That is to say, during the video shooting process, the present application collects audio data (i.e., first audio data) by performing directional sound pickup on the target object. Compared with the prior art of collecting audio data by omnidirectional sound pickup during video shooting, performing directional sound pickup on the target object can suppress the sounds of other objects, thereby improving the signal-to-noise ratio of the audio data of the target object, thereby improving the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio of the target object in the shot video. In this way, the video can be used directly later (for example, forwarded, shared to a social platform, etc.), without the need to use professional software to process the audio in the video before use, which can simplify user operations and improve user experience.

[0035] Exemplarily, the front camera / rear camera may be called to shoot first video data for the target object.

[0036] Exemplarily, the beam state of the microphone may be adjusted to achieve directional sound pickup for the target object to collect the first audio data, wherein the beam state may include a beam directional angle and a beam shape.

[0037] Exemplarily, the first audio data and the first video data can be stored in the same multimedia file, which is a video shot for the target object, and can also be called an audio-video file (or audio-video data). Each frame of audio in the first audio data has a timestamp, and each frame of image in the first video data has a timestamp, and the timestamps of the images and audio collected at the same time are the same; in this way, when playing the video shot for the target object, the playback module can play a frame of image and a frame of audio with the same timestamp at the same time, thereby achieving audio-visual synchronization.

[0038] According to the second aspect, the shooting device performs pointing and picking up sound from the target object to collect the first audio data, including: the shooting device determines the target object based on the first video data; the shooting device determines the first distance according to the target object; the first distance is the distance between the target object and the shooting device; the shooting device performs pointing and picking up sound from the target object according to the first distance to collect the first audio data. In this way, the pointing and picking up sound from the target object can be realized in the early stage of video shooting.

[0039] Exemplarily, the photographing device may determine the first distance based on the first M (M is a positive integer) frames of images of the first video data.

[0040] Exemplarily, the shooting device may also determine the first distance based on the initial video data (or the first target video data in the first aspect above). The initial video data may refer to the video data collected by the camera after entering the recording interface (at this time, the camera is started) and before performing the second shooting operation.

[0041] According to the second aspect, or any implementation of the second aspect above, the shooting device points to the target object and picks up the sound to collect the first audio data according to the first distance, specifically including: the shooting device determines the first angle of the target object relative to the shooting device according to the first distance; the shooting device adjusts the beam pointing angle of the microphone to the first angle; according to the preset rule and the first distance, the beam shape of the microphone is adjusted to the first beam shape; the microphone of the shooting device uses the first beam shape and the first angle to collect the first audio data. In this way, the target pickup area of ​​the microphone can include the target object and as few other objects as possible, thereby improving the signal-to-noise ratio of the audio data of the target object.

[0042] Exemplarily, the first angle of the target object relative to the shooting device may refer to the angle between the line between the center of the shooting device and the target object and the X-axis; wherein X is a line passing through the center of the shooting device and perpendicular to the back of the shooting device.

[0043] Exemplarily, the preset rule may be a rule learned in advance according to training, with the goal of increasing the sound pickup effect of the target object. For example, the preset rule is: when the distance is X1 meters, the short axis of the left beam is a1 mm and the short axis of the right beam is a2 mm, and / or, the long axis of the left beam is b1 mm and the long axis of the right beam is b2 mm; when the distance is X2 meters, the short axis of the left beam is a3 mm and the short axis of the right beam is a4 mm, and / or, the long axis of the left beam is b3 mm and the long axis of the right beam is b4 mm. Wherein, X1, X2, a1-a4 and b1-b4 are all positive integers.

[0044] According to the second aspect, or any implementation of the second aspect above, the shooting device performs real-time sound source tracking on the target object; if the position of the target object changes, the shooting device re-determines the first distance, and performs the step of pointing and picking up the target object according to the first distance to collect the first audio data. In this way, even if the position of the target object changes, the audio quality and clarity of the audio data of the target object in the captured video can be guaranteed.

[0045] Exemplarily, the second position information (that is, the position information after the change) of the target object at the current moment (or the current cycle) can be determined by sound source positioning. The second position information may include a second pitch angle (i.e., the angle between the line between the target object and the origin and the Z axis), a second horizontal angle (i.e., the angle between the line between the target object and the origin and the X axis), and a second distance (i.e., the distance between the target object and the terminal device); thereafter, the second distance may be used as the first distance.

[0046] According to the second aspect, or any implementation of the second aspect above, the method also includes: the shooting device receives a focus adjustment operation; in response to the focus adjustment operation, the shooting device adjusts the focus of the camera, and the camera uses the adjusted focus to shoot second video data for the target object, and the shooting device adjusts the beam state of the microphone, and the microphone uses the beam after the beam state is adjusted to point to the target object for sound pickup to collect second audio data, and the beam state includes a beam shape and a beam pointing angle.

[0047] In this way, when the distance between the target object and the terminal device becomes farther, the sound pickup distance of the microphone can be increased to achieve long-distance sound pickup, thereby improving the sound pickup effect for the target object; and increasing the volume of the audio data of the target object in the first audio data. And when the distance between the target object and the terminal device becomes closer, the sound pickup distance of the microphone can be reduced to improve the sound pickup effect for the target object; and reducing the volume of the audio data of the target object in the first audio data. In other words, the present application can perform corresponding "audio zoom" as the video zooms.

[0048] In addition, the visual and auditory synchronization of the distance between the target object and the shooting device can also be achieved, that is, the distance between the target object and the shooting device perceived by watching the video data and the distance between the target object and the shooting device perceived by listening to the audio data are equal or have a small difference. For example, when the actual distance between the target object and the shooting device is 5m, the user increases the focal length, and the proportion of the target object in the picture of the video data becomes larger; at this time, the distance between the target object and the shooting device perceived by watching the video data is 2m; however, since the prior art cannot achieve "audio zoom", the distance between the target object and the shooting device perceived by listening to the audio data collected by the prior art is 5m, while the distance between the target object and the shooting device perceived by listening to the audio data collected by the present application (i.e., the second audio data) is 2.5m.

[0049] According to the second aspect, or any implementation of the second aspect, the camera is a rear camera.

[0050] According to the second aspect, or any implementation of the second aspect above, the method further includes: when detecting that the distance between the target object and the shooting device changes, the shooting device adjusts the beam state of the microphone, and the microphone uses the beam after the beam state is adjusted to point to the target object to pick up sound to collect third audio data, and the beam state includes a beam shape and a beam pointing angle. In this way, when the user does not perform a focus adjustment operation (or the shooting device does not support the user to perform a focus adjustment operation), audio zoom can also be achieved.

[0051] For example, the volume of the target object's audio data can be used to detect whether the distance between the target object and the camera has changed; or, if the focal length of the camera has not changed, the proportion of the target object in the image of the video data can be used to detect whether the distance between the target object and the camera has changed, which is not limited in this application. In addition, in one method, during real-time sound source tracking, it can be determined whether the distance between the target object and the camera has changed.

[0052] According to the second aspect, or any implementation of the second aspect, the camera is a front camera.

[0053] According to the second aspect, or any implementation of the second aspect above, the method further includes: the photographing device receives a selection operation for the target object; in response to the selection operation for the target object, the photographing device determines the target object. In this way, the user can customize the selection of the target object to meet the user's personalized needs.

[0054] According to the second aspect, or any implementation of the second aspect above, the method also includes: the shooting device determines any one of the object closest to the center of the shooting picture, the object closest to the shooting device, or the object with the largest proportion in the shooting picture as the target object.

[0055] According to the second aspect, or any implementation of the second aspect, the target object is one or more.

[0056] Exemplarily, when there are multiple target objects, the beam state of the microphone can be adjusted according to the second position information of the multiple target objects so that the target pickup area of ​​the microphone is the intersection or union of the positions of the multiple target objects, or in other words, the target pickup area of ​​the microphone includes multiple target objects and as few other objects as possible.

[0057] According to the second aspect, or any implementation of the second aspect above, the method further includes: the shooting device receives a target object switching operation; in response to the target object switching operation, the shooting device shoots fourth video data for the switched target object and performs pointing and sound pickup for the switched target object to collect fourth audio data. In this way, during the video shooting process, the target object can be changed according to user needs without stopping the current video recording and reshooting another video, which simplifies the user operation.

[0058] It should be noted that the first audio data in the second aspect and any implementation of the second aspect may include the second target audio data and the second other audio data in the first aspect and any implementation of the first aspect. And the fourth audio data in the second aspect and any implementation of the second aspect may include the third target audio data and the third other audio data in the first aspect and any implementation of the first aspect. In addition, the second audio data and the third audio data may both include the audio data of the target object and the audio data of other objects.

[0059] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0060] In a third aspect, an embodiment of the present application provides a terminal device, which can be used to execute the method executed by the shooting device in the above-mentioned first aspect and any one of the implementations of the first aspect; or, to execute the method executed by the shooting device in the second aspect and any one of the implementations of the second aspect.

[0061] Exemplarily, terminal devices include but are not limited to: mobile phones, tablet computers, smart watches, laptop computers, home cameras, etc., and this application does not impose any restrictions on this.

[0062] According to a third aspect, the terminal device may include at least three groups of microphones.

[0063] According to the third aspect, or any implementation of the third aspect, a distance between any two groups of microphones in the at least three groups of microphones is greater than a preset distance.

[0064] Exemplarily, the preset distance is greater than 0 (that is, there is a certain distance between any two groups of microphones). The preset distance can be set according to requirements, and this application does not impose any limitation on this.

[0065] Exemplarily, the distribution of at least three groups of microphones in different terminal devices may be different, and this application does not impose any limitation on this.

[0066] According to the third aspect, or any implementation of the third aspect, a group of microphones includes at least one microphone.

[0067] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0068] The third aspect and any implementation of the third aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, which will not be repeated here.

[0069] In a fourth aspect, an embodiment of the present application provides a computer storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the video shooting method in the first aspect or any possible implementation of the first aspect.

[0070] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

[0071] In a fifth aspect, a chip is provided, wherein the chip includes at least one processing circuit and an interface, and the processing circuit can execute the video processing method of the first aspect and any one of the items in the first aspect.

[0072] The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0073] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the video shooting method in the first aspect or any possible implementation of the first aspect.

[0074] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a schematic diagram of the hardware structure of an electronic device shown as an example;

[0076] Figure 2 A schematic diagram of the software structure of an electronic device is shown as an example;

[0077] Figure 3 is a schematic diagram of microphone distribution shown as an example;

[0078] Figure 4A is a schematic diagram of an application scenario shown as an example;

[0079] Figure 4B is a schematic diagram of an application scenario shown as an example;

[0080] Figure 4C is a schematic diagram of an application scenario shown as an example;

[0081] Figure 4D is a schematic diagram of an application scenario shown as an example;

[0082] Figure 5A is a schematic diagram of a video shooting process shown as an example;

[0083] Figure 5B is a schematic diagram of a coordinate system shown as an example;

[0084] Figure 5C is a schematic diagram of a beam of a microphone shown as an example;

[0085] Figure 5D is a schematic diagram of a beam of a microphone shown as an example;

[0086] Figure 5E is a schematic diagram of a beam of a microphone shown as an example;

[0087] Fig. 5F is a schematic diagram of initial audio data shown as an example;

[0088] Figure 5G is a schematic diagram of initial audio data shown as an example;

[0089] Figure 5H is a schematic diagram of first audio data exemplarily shown;

[0090] Fig.5I is a schematic diagram of first audio data exemplarily shown;

[0091] Fig. 6A is a schematic diagram of a mobile phone interface shown as an example;

[0092] Figure 6Bis a schematic diagram of a mobile phone interface shown as an example;

[0093] Figure 6C is a schematic diagram of a mobile phone interface shown as an example;

[0094] Fig.6D is a schematic diagram of a mobile phone interface shown as an example;

[0095] Fig. 6E is a schematic diagram of a mobile phone interface shown as an example;

[0096] Fig. 6F is a schematic diagram of a mobile phone interface shown as an example;

[0097] Figure 6G is a schematic diagram of a mobile phone interface shown as an example;

[0098] Figure 7 is a schematic diagram of a video shooting process shown as an example;

[0099] Fig. 8A is a schematic diagram of a mobile phone interface shown as an example;

[0100] Figure 8B is a schematic diagram of a mobile phone interface shown as an example;

[0101] Figure 8C is a schematic diagram of a mobile phone interface shown as an example;

[0102] Fig.8D is a schematic diagram of a mobile phone interface shown as an example;

[0103] Fig. 8E is a schematic diagram of a mobile phone interface shown as an example;

[0104] Figure 8F is a schematic diagram of a beam of a microphone shown as an example;

[0105] Figure 8G is a schematic diagram of a beam of a microphone shown as an example;

[0106] Fig. 9 is a schematic diagram of a video shooting process shown as an example;

[0107] Fig.10 is a schematic diagram of a video shooting process shown as an example;

[0108] Fig.11A is a schematic diagram of a mobile phone interface shown as an example;

[0109] Fig. 11B is a schematic diagram of a mobile phone interface shown as an example;

[0110] Fig. 11C is a schematic diagram of a mobile phone interface shown as an example;

[0111] Fig.11D is a schematic diagram of a mobile phone interface shown as an example;

[0112] Fig.11E is a schematic diagram of a mobile phone interface shown as an example;

[0113] Fig.11F is a schematic diagram of a mobile phone interface shown as an example;

[0114] Fig.11G is a schematic diagram of a mobile phone interface shown as an example;

[0115] Fig.11H is a schematic diagram of a mobile phone interface shown as an example;

[0116] Fig.12 is a schematic diagram of a video shooting process shown as an example;

[0117] Fig.13A is a schematic diagram of a mobile phone interface shown as an example;

[0118] Fig. 13B The figure is a schematic diagram of a mobile phone interface shown as an example. DETAILED DESCRIPTION

[0119] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0120] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0121] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0122] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0123] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0124] Figure 1 1 shows a schematic diagram of the structure of the electronic device 100. It should be understood that Figure 1 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0125] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0126] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0127] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0128] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0129] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0130] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0131] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0132] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0133] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0134] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0135] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0136] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0137] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0138] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0139] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0140] The wireless communication module 160 can provide wireless communication solutions including WLAN (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0141] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0142] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0143] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0144] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0145] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0146] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. Among them, the N cameras 193 may include N1 front cameras and N2 rear cameras, where N1 and N2 are integers, and N1+N2=N.

[0147] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0148] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0149] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0150] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0151] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0152] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0153] Exemplarily, the electronic device may include three or more groups of microphones 170C, wherein a group of microphones 170C may include at least one microphone.

[0154] It should be noted that the distance between any two groups of microphones 170C in the electronic device is greater than the preset distance; wherein the preset distance is greater than 0 (that is, there is a certain distance between any two groups of microphones), and the preset distance can be set according to specific needs, and this application does not impose any restrictions on this.

[0155] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0156] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0157] The layered architecture of the electronic device 100 divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0158] The application layer can include a series of application packages.

[0159] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0160] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0161] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a media service, and the like.

[0162] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0163] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0164] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0165] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0166] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0167] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0168] Media services can be used to process multimedia data (such as audio data, video data, etc.).

[0169] AndroidRuntime includes core libraries and virtual machines. AndroidRuntime is responsible for scheduling and management of the Android system.

[0170] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0171] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0172] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGLES), 2D graphics engine (such as SGL), etc.

[0173] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0174] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0175] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0176] A 2D graphics engine is a drawing engine for 2D drawings.

[0177] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0178] Understandably, Figure 2 The components included in the system framework layer, system library and runtime layer shown do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine some components, or split some components, or arrange the components differently.

[0179] Figure 3 FIG. 1 is a schematic diagram showing an exemplary microphone distribution. Figure 3 In the embodiment, the electronic device is a mobile phone, the electronic device includes three groups of microphones, and each group of microphones includes one microphone.

[0180] Reference Figure 3 , exemplarily, the mobile phone may include three microphones: microphone 1, microphone 2, and microphone 3. In some embodiments, microphone 1 may be referred to as a top microphone, microphone 2 may be referred to as a bottom microphone, and microphone 3 may be referred to as a back microphone.

[0181] It should be noted that Figure 3 The position of the three microphones is only an example of the present application. The position of microphone 1 of different mobile phones may be slightly different, or the position of microphone 2 of different mobile phones may be slightly different, or the position of microphone 3 of different mobile phones may be slightly different.

[0182] In addition, when the electronic device is other terminal devices (such as tablets, smart watches, home cameras, etc.), the distribution of microphone 1, microphone 2 and microphone 3 may be different from the distribution of microphone 1, microphone 2 and microphone 3 in a mobile phone, and this application does not impose any restrictions on this.

[0183] Figure 4A The figure is a schematic diagram of an exemplary application scenario. Figure 4A The application scenario shown is shooting a singing video.

[0184] Reference Figure 4A In a possible scenario, while singer A and singer B are singing on the stage in the studio, audience 1 sitting in the audience can take a video of the chorus of singer A and singer B.

[0185] Reference Figure 4A In one possible scenario, while singer A and singer B are singing on the stage in the studio, audience 1 sitting in the audience can shoot a solo video of singer A, or shoot a solo video of singer B.

[0186] Figure 4B The figure is a schematic diagram of an exemplary application scenario. Figure 4B Shown is shooting a motion video.

[0187] For example, when a family is doing outdoor activities, parents usually take videos of their children playing / entertaining / competing to record their children's daily life. Figure 4B , for example, a parent is filming a video of his child juggling a ball (ping pong).

[0188] Figure 4C The figure is a schematic diagram of an exemplary application scenario. Figure 4C Shown is the capture of a conference video.

[0189] Reference Figure 4C ,For example, after the meeting starts, a video of the meeting speaker can be taken for ,the subsequent meeting summary.

[0190] Figure 4D The figure is a schematic diagram of an exemplary application scenario. Figure 4D Shown is a video of a text recitation.

[0191] Reference Figure 4D For example, in order to complete the school's recitation check-in task, parents can take a video of their children reciting the text.

[0192] It should be understood that FIG. 4A to FIG. 4D The terminal devices used for shooting include but are not limited to: mobile phones, tablet computers, smart watches, laptops, home cameras, etc., and this application does not impose any restrictions on this.

[0193] It should be understood that the present application may also include other video shooting scenarios, such as shooting lecture videos, live broadcasts, shooting speech videos, shooting interview videos, etc., and the present application does not impose any restrictions on this.

[0194] exist Figure 4AIn the scene, there are more audiences in the studio; Figure 4B In the scene, there are many people outdoors, and there are other noises such as the sound of wind blowing through the trees, vehicle horns, birds singing, etc.; this will cause a lot of ambient noise in the shooting environment, which in turn affects the audio quality of the captured video. Figure 4C The distance between the middle part of the conference participants and the terminal device is closer than the distance between the conference speaker and the terminal device; Figure 4D The distance between the parent and the terminal device is closer than the distance between the child and the terminal device, which causes the sound of the target object in the captured video to not be prominent (or the target object is not the main sound source).

[0195] In order to solve the above problems, the present application proposes a video shooting method, which can suppress the sounds of other objects and thereby improve the signal-to-noise ratio of the audio data of the target object, thereby improving the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio of the target object in the shot video; in this way, the video can be used directly later (for example, forwarded, shared to social platforms, etc.), without the need to use professional software to process the audio in the video before use, which can improve the user experience.

[0196] Figure 5A The figure is a schematic diagram showing an exemplary video shooting process.

[0197] S501: Receive a first shooting operation.

[0198] S502, receiving a response to a first shooting operation, recording first target audio data and first other audio data, the first target audio data being audio data of a target object, and the first other audio data being audio data of other objects.

[0199] S503: Receive a second shooting operation.

[0200] Figure 6A to Figure 6G The figure is a schematic diagram of a mobile phone interface shown as an example.

[0201] Reference Fig. 6A , exemplarily, 601 is the main interface of the mobile phone, and the main interface of the mobile phone includes one or more controls, including but not limited to: application icons (for example, application icons of Huawei video applications, application icons of browser applications, application icons 602 of camera applications), network logos, power logos, etc.

[0202] Continue to refer to Fig. 6A In one possible scenario, the user clicks on the application icon 602 of the camera application, and the mobile phone responds to the user's operation behavior and displays the application interface 603 of the camera application, such as Figure 6BAs shown. Exemplarily, the application interface 603 of the camera application includes one or more controls, including but not limited to: camera switching options, flash options, AI shooting options, settings options, night scene options, people options, photo options, and video recording options 604. In addition, the application interface 603 of the camera application may also include a preview box, which displays the image captured by the camera.

[0203] Continue to refer to Figure 6B In one possible manner, the user clicks on the recording option 604, and the mobile phone responds to the user's operation behavior and displays the recording interface 605, such as Figure 6C As shown. The video recording interface 605 includes one or more controls, including but not limited to: focus control options, flash options, settings options, recording start options 607, camera switching options, picture viewing options, and shooting mode prompt options 606. Exemplarily, the shooting mode prompt option 606 includes one or more controls, including but not limited to: confirmation options, cancel options, and prompt information (such as "whether to shoot video for the target object"). In addition, the video recording interface 605 may also include a preview box, which displays the image captured by the camera.

[0204] Continue to refer to Figure 6C For example, when a user needs to shoot a video for a target object (e.g. Figure 4A to Figure 4D When the user clicks the confirmation option in the shooting mode prompt option 606, the mobile phone responds to the user's operation behavior and uses the video shooting method involved in the present application to perform video recording (or video shooting) (i.e., execute S504), and displays the video shooting interface 608, such as Fig.6D Alternatively, when the user clicks the confirmation option in the shooting mode prompt option 606, the mobile phone cancels the display of the shooting mode prompt option 606 in response to the user's operation behavior; then, the user can click the recording start option 607, and the mobile phone executes S504 and displays the video shooting interface 608 in response to the user's operation behavior, as shown. Fig.6D shown.

[0205] Continue to refer to Figure 6C For example, when the user does not need to shoot a video for the target object, the user can directly click the recording start option 607. In response to the user's operation, the mobile phone uses the video shooting method of the prior art to shoot the video (i.e., execute S502) and displays the video shooting interface 608. Fig.6DAlternatively, when the user does not need to shoot a video of the target object, the user clicks the cancel option in the shooting mode prompt option 606, and the mobile phone responds to the user's operation and cancels the display of the shooting mode prompt option 606; then, the user can click the recording start option 607, and the mobile phone responds to the user's operation and uses the video shooting method of the prior art to shoot a video (i.e., execute S502) and displays the video shooting interface 608, as shown. Fig.6D shown.

[0206] The operation of clicking the recording start option 607 can be referred to as the first shooting operation. Alternatively, the operation of clicking the shooting mode prompt option 606 and the operation of clicking the recording start option 607 are referred to as the first shooting operation. The video shooting method using the prior art can record the first target audio data, the first other audio data and the first target video data.

[0207] Refer again Figure 6B In one possible way, the user clicks the recording option 604, and the mobile phone responds to the user's operation behavior and displays the recording interface 609, such as Fig. 6E The video recording interface 609 includes one or more controls, including but not limited to: focus control option, flash option, setting option, recording start option 611, camera switching option, picture viewing option and target object shooting option 610.

[0208] Continue to refer to Fig. 6E For example, when the user does not need to shoot a video for the target object, the user can directly click the recording start option 611. The mobile phone responds to the user's operation behavior and uses the video shooting method of the prior art (i.e., executes S502) to shoot the video and display the video shooting interface 608, such as Fig.6D When the user needs to shoot a video for a target object, the user clicks on the target object shooting option 610, and the mobile phone responds to the user's operation behavior, executes S504 and displays the video shooting interface 608, such as Fig.6D Alternatively, when the user needs to shoot a video of a target object, the user clicks on the target object shooting option 610 and the recording start option 611 in sequence, and the mobile phone responds to the user's operation behavior, executes S504 and displays the video shooting interface 608, as shown. Fig.6D shown.

[0209] Refer again Fig. 6A In one possible manner, the user clicks on the application icon 602 of the camera application, and the mobile phone responds to the user's operation behavior and displays the application interface 612 of the camera application, such as Fig. 6FAs shown. Exemplarily, the application interface 612 of the camera application includes one or more controls, including but not limited to: camera switching options, flash options, AI shooting options, settings options, night scene options, people options, photo options, normal video options, target video options 613, etc. In addition, the application interface 612 of the camera application may also include a preview box, which displays the image captured by the camera.

[0210] Continue to refer to Fig. 6F For example, when a user needs to shoot a video of a target object, the user clicks on the target video option 613, and the mobile phone displays a recording interface 614 in response to the user's operation. Figure 6G The video recording interface 614 includes one or more controls, including but not limited to: focus control option, flash option, setting option, recording start option 615, camera switching option, picture viewing option, etc. Then, the user clicks the recording start option 615, and the mobile phone executes S504 and displays the video shooting interface in response to the user's operation behavior.

[0211] Continue to refer to Fig. 6F For example, when the user does not need to shoot a video for the target object (that is, not distinguishing between the target object and other objects, or called ordinary shooting), the ordinary video option can be clicked. In this case, the first shooting operation can be the operation of clicking the ordinary video option.

[0212] It should be noted that the second shooting operation in S503 may be as follows:

[0213] In one possible manner, the second shooting operation of the user in S503 may be clicking Figure 6C The operation of confirming the option in the shooting mode prompt option 606 is shown.

[0214] In one possible manner, the second shooting operation of the user in S503 may include clicking Figure 6C In the shooting mode prompt option 606, confirm the operation of the option, and click Figure 6C The operation of recording startup option 607.

[0215] In one possible manner, the second shooting operation of the user in S503 may be clicking Fig. 6E Operation of the target object shooting option 610.

[0216] In one possible manner, the second shooting operation of the user in S503 may include clicking Fig. 6E Operation of the target object shooting option 610, and clicking Fig. 6E Operation of the recording start option 611 in.

[0217] In one possible manner, the second shooting operation of the user in S503 may include clicking Fig. 6F In the target video option 613, click Figure 6G Operation of the recording start option 615 in.

[0218] In one possible manner, the second shooting operation of the user in S503 may be clicking Figure 6G Operation of the recording start option 615 in.

[0219] S504 , in response to a second shooting operation, recording second target audio data and second other audio data, where the second target audio data is audio data for a target object, and the second other audio data is audio data for other objects.

[0220] Exemplarily, after receiving the second shooting operation from the user, the mobile phone may call the camera to collect the first video data (that is, the target video data or the second target video data) in response to the second shooting operation. The image in the first video data may include the target object.

[0221] Exemplarily, in response to the second shooting operation, the target object is targeted for sound pickup to collect the first audio data, wherein the first audio data includes the second target audio data and the second other audio data.

[0222] It should be noted that, refer to Figure 6B , the user clicks on the video recording option 604, and the mobile phone responds to the user's operation by displaying Figure 6C Recording interface 605 (or Fig. 6E At the same time as the recording interface 609 in the video recording is displayed, the camera and microphone of the mobile phone are also activated; at this time, the camera of the mobile phone can collect video data (hereinafter referred to as initial video data for the sake of distinction), and the microphone can also collect audio data (hereinafter referred to as initial audio data for the sake of distinction). And, refer to Fig. 6F , the user clicks on the target video option 613, and the mobile phone responds to the user operation by displaying Figure 6G At the same time as the recording interface 614 is displayed, the camera and microphone of the mobile phone are also started; at this time, the camera of the mobile phone can collect initial video data, and the microphone of the mobile phone can collect initial audio data.

[0223] It should also be noted that, during the process of the mobile phone's microphone collecting initial audio data or first target audio data (taking the collection of initial audio data as an example), the beam of the mobile phone's microphone is a fixed beam (it can also be said that the mobile phone's microphone is omnidirectionally picking up sound). Wherein, when the camera used to collect initial video data is a rear camera, the beam shape of the fixed beam can be a first preset shape, and the beam pointing angle of the fixed beam can be a first preset angle. Wherein, the first preset shape and the first preset angle can be obtained by training in advance (before the terminal device leaves the factory), and the first preset shape and the first preset angle are the beam shape and beam pointing angle with the best sound pickup effect when shooting with the rear camera (wherein, during the training process, the rear camera is used to shoot video data and the microphone is used to collect audio data; then, the signal-to-noise ratio of the audio data is determined; when the signal-to-noise ratio is lower than the signal-to-noise ratio threshold, the beam shape and beam pointing angle of the microphone are adjusted until the optimal beam shape and beam pointing angle are obtained). When the camera used to collect the initial video data is a front camera, the beam shape of the fixed beam may be a second preset shape, and the beam pointing angle of the fixed beam may be a second preset angle. The second preset shape and the second preset angle may also be obtained through pre-training, and the second preset shape and the second preset angle are the beam shape and the beam pointing angle with the best sound pickup effect when shooting with the front camera.

[0224] In order to facilitate the subsequent description, the three-dimensional coordinate system established based on the mobile phone is introduced. Figure 5B For example, the center of the mobile phone can be taken as the origin O, the line perpendicular to the back of the mobile phone and passing through the origin O can be taken as the X-axis, the line perpendicular to the top of the mobile phone and passing through the origin O can be taken as the Z-axis, and the line perpendicular to the side of the mobile phone and passing through the origin O can be taken as the Y-axis.

[0225] Figure 5C FIG. 1 is a schematic diagram of a beam of a microphone shown as an example.

[0226] For example, assuming that the shooting direction of the rear camera is ( Figure 5B The X-axis direction is 0 degrees. Figure 5C In the figure, the solid line is the left beam and the dotted line is the right beam. The beam of the microphone can be synthesized by the left beam and the right beam. The beam pointing angle of the left beam is 45 degrees, and the beam pointing angle of the right beam is 345 degrees. At this time, the beam pointing angle of the microphone beam synthesized by the left beam and the right beam is 0 degrees; the shapes of the left beam and the right beam are as follows Figure 5C shown.

[0227] Exemplarily, after receiving the second shooting operation from the user, the terminal device may respond to the second shooting operation by instructing the image module (or image processing module, which may be a module in the processor) to determine the video focal length information based on the first video data, wherein the video focal length information may include the distance between the target object and the terminal device (hereinafter referred to as the first distance). Specifically, the image module may call a pre-trained image processing model (wherein the image processing model may be a model in the prior art; this may also be described as calling the image processing model to load pre-trained model parameters) based on the first video data to determine the first distance. Specifically, the image processing model may perform target detection based on the first video to determine the first distance between the target object and the terminal device.

[0228] Subsequently, the image module can output the video focal length information to the audio module (or audio processing module or audio module), and the audio module can perform directional sound pickup for the target object according to the first distance in the video focal length information. Specifically, the audio module can call a pre-trained beam adjustment algorithm (which can also be described as calling a beam adjustment algorithm to load pre-trained algorithm parameters) to determine the first angle of the target object relative to the terminal device according to the first distance; then, according to the first angle, adjust the beam state of the microphone so that the target sound pickup area of ​​the microphone contains the target object and contains as few other objects as possible, so as to achieve directional sound pickup for the target object.

[0229] Exemplarily, the first angle of the target object relative to the terminal device may be as follows: Figure 5B As shown, the first angle is the angle between the line between the origin O and the target object and the X-axis; wherein the first angle can be called the first horizontal angle. Exemplarily, at least three microphones in the terminal device can determine the distance, horizontal angle and pitch angle between any point in the three-dimensional space and the terminal device; further, the terminal device can deduce the first angle of the target object relative to the terminal device based on the first distance of the target object relative to the terminal device.

[0230] Exemplarily, the beam state may include a beam pointing angle and a beam shape. Specifically, the beam state of the microphone may be adjusted according to a first angle, a preset rule, and a first distance; for example, the beam pointing angle of the microphone may be adjusted to a first angle, and the beam shape of the microphone may be adjusted to a first beam shape according to a preset rule and a first distance.

[0231] Exemplarily, the preset rule is a rule that the beam adjustment algorithm is pre-trained and learned to increase the sound pickup effect of the target object. For example, the preset rule is: when the distance is X1 meters, the short axis of the left beam is a1 mm and the short axis of the right beam is a2 mm, and / or, the long axis of the left beam is b1 mm and the long axis of the right beam is b2 mm; when the distance is X2 meters, the short axis of the left beam is a3 mm and the short axis of the right beam is a4 mm, and / or, the long axis of the left beam is b3 mm and the long axis of the right beam is b4 mm. Among them, X1, X2, a1~a4 and b1~b4 are all positive integers.

[0232] Figure 5D FIG. 1 is a schematic diagram of a beam of a microphone shown as an example.

[0233] Reference Figure 5D For example, assuming that the first angle is 30 degrees, after adjusting the beam state of the microphone according to the first angle, the preset rule and the first distance, the beam pointing angle of the left beam is 60 degrees, the beam pointing angle of the right beam is 0 degrees, and the beam pointing angle of the beam of the microphone synthesized by the left beam and the right beam is 30 degrees; the shapes of the left beam and the right beam are Figure 5D shown.

[0234] Relative to Figure 5C In terms of Figure 5D The long axis of the left beam (or right beam) remains unchanged. Figure 5D The minor axis of the center left beam (or right beam) is shorter.

[0235] It should be noted that the directional sound pickup performed by the audio module according to the first angle in the video focal length information output by the image module is the first directional sound pickup.

[0236] In the process of shooting a video of a target object, the target object may be in motion. Therefore, after the first pointing sound pickup, the audio module can determine the audio data of the target object from the first audio data collected after the first pointing sound pickup (wherein the first audio data may include the audio data of the target object (i.e., the first target audio data) and the audio data of other objects (i.e., the first other audio data)) according to the first distance in the video focal length information. Specifically, the audio module can first call the first audio processing model obtained by pre-training (wherein the first audio processing model can be a model in the prior art; it can also be described here as calling the first audio processing model to load the pre-trained model parameters), perform sound source positioning according to the first audio data, and determine the first position information of multiple objects (including the target object and other objects) contained in the first audio data. Among them, the first position information of a single object can include a third pitch angle (i.e., the line between the single object and the origin, the angle between the Z axis), a third horizontal angle (i.e., the line between the single object and the origin, the angle between the X axis) and a third distance (i.e., the distance between the single object and the terminal device). Next, the pre-trained second audio processing model is called (wherein the second audio processing model may be a model in the prior art; this may also be described as calling the second audio processing model to load the pre-trained model parameters), and the audio data of the target object is determined according to the first distance and the third distance of the multiple objects; for example, the audio data of the object whose third distance is closest to the first distance is used as the audio data of the target object. In addition, the second audio processing model may also extract feature information from the audio data of the target object to obtain the feature information of the target object, so as to determine the audio data of the target object from the first audio data collected subsequently.

[0237] Afterwards, the audio module can track the sound source of the target object. Exemplarily, the audio module can call the second audio processing model in real time (or periodically) to extract features from the audio data of multiple objects in the first audio data collected subsequently, and obtain feature information of multiple objects. Then, the feature information of multiple objects is compared with the feature information of the target object to determine the audio data of the target object from the first audio data collected subsequently; for example, the audio data of the object whose feature information is closest to the feature information of the target object is used as the audio data of the target object. Afterwards, the first audio processing model can locate the sound source according to the audio data of the target object and determine the second position information of the target object (that is, the result of sound source tracking). Among them, the second position information of the target object may include a second pitch angle (i.e., the line between the target object and the origin, and the angle between the Z axis), a second horizontal angle (i.e., the line between the target object and the origin, and the angle between the X axis) and a second distance (i.e., the distance between the target object and the terminal device). Then, the audio module calls the beam adjustment algorithm to determine whether the position of the target object has changed based on the second position information of the target object at the current moment (or current cycle) and the second position information of the target object at the previous moment (or previous cycle).

[0238] Exemplarily, when the second position information of the target object at the current moment (or current cycle) is greater than the difference information between the second position information of the target object at the previous moment (or previous cycle) and the second position information of the target object at the previous moment (or previous cycle) is greater than the difference threshold, it is determined that the position of the target object has changed; at this time, the audio module can call the beam adjustment algorithm, according to the second position information of the target object at the current moment (or current cycle) (that is, the position information after the change), re-adjust the beam state of the microphone (or understand it as re-determining the first distance and the first angle, that is, taking the second distance as the first distance, and taking the second horizontal angle as the first angle) to achieve directional pickup of the target object after the position change. By analogy, in the process of shooting a video of the target object, the audio module can capture the position of the target object, and constantly readjust the beam state of the microphone according to the current position information of the target object (that is, the second position information of the target object at the current moment (or current cycle)), so as to achieve dynamic directional pickup of the target object.

[0239] Specifically, the beam state of the microphone can be adjusted according to the second angle, the second distance and the preset rule; for example, the beam pointing angle of the microphone can be adjusted to the second angle, and the beam shape of the microphone can be adjusted according to the preset rule and the second distance.

[0240] Figure 5E FIG. 1 is a schematic diagram of a beam of a microphone shown as an example.

[0241] Reference Figure 5EFor example, assuming that the second angle is 330 degrees, after adjusting the beam state of the microphone according to the second angle, the second distance and the preset rule, the beam pointing angle of the left beam is 0 degrees, the beam pointing angle of the right beam is 300 degrees, and the beam pointing angle of the beam of the microphone synthesized by the left beam and the right beam is 330 degrees; the shapes of the left beam and the right beam are Figure 5E shown.

[0242] Relative to Figure 5D In terms of Figure 5E Neutralization Figure 5D The minor axes of the left beam (right beam) are equal, and Figure 5E Neutralization Figure 5D The long axes of the left beam (and right beam) are equal; Figure 5D The beam pointing angle of the middle microphone is 30 degrees. Figure 5E The beam pointing angle of the microphone beam is 330 degrees.

[0243] Exemplarily, when the difference information between the second position information of the target object at the current moment (or current period) and the second position information of the target object at the previous moment (or previous period) is less than or equal to the difference threshold, the first audio processing model can continue to be called to perform real-time (or periodic) sound source positioning according to the audio data of the target object to determine the second position information of the target object at the next moment (or next period).

[0244] Fig. 5F and Figure 5G FIG. 1 is a schematic diagram of exemplary first target audio data and first other audio data, wherein user A is the target object.

[0245] For example, the audio data of user A (such as Fig. 5F As shown, i.e., the first target audio data) and the audio data of user B (as shown Figure 5G As shown, namely the first other audio data). Fig. 5F and Figure 5G , the amplitude of the audio data of user A and the amplitude of the audio data of user B are not much different.

[0246] Figure 5H and Fig.5I FIG. 1 is a schematic diagram showing exemplary second target audio data and second other audio data.

[0247] Exemplarily, after the user A is pointedly picked up, the first audio data collected includes the audio data of the user A (such as Figure 5H As shown, that is, the second target audio data) and the audio data of user B (as shown Fig.5I As shown, that is, the second other audio data). Figure 5H and Fig.5I, the amplitude of the audio data of user A is greater than the amplitude of the audio data of user B.

[0248] It can be seen that after the targeted sound pickup is performed, the signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data; therefore, the sound pickup effect of the target object is better.

[0249] In this way, by performing directional sound pickup for the target object during video shooting, the sounds of other objects can be suppressed, thereby improving the signal-to-noise ratio of the audio data of the target object. Therefore, the sound pickup effect for the target object is better, and the quality (i.e., restoration) and clarity (i.e., recognizability) of the audio of the target object in the shot video can be improved. In this way, the video can be used directly later (for example, forwarded, shared to social platforms, etc.), without the need to use professional software to process the second target audio data in the video before use, which can simplify user operations and improve user experience.

[0250] Exemplarily, the first audio data and the first video data can be stored in the same multimedia file, which is a video shot for the target object, and can also be called an audio-video file (or audio-video data). Each frame of audio in the first audio data has a timestamp, and each frame of image in the first video data has a timestamp, and the timestamps of the images and audio collected at the same time are the same; in this way, when playing the video shot for the target object, the playback module can play a frame of image and a frame of audio with the same timestamp at the same time, thereby achieving audio-visual synchronization.

[0251] Figure 7 The figure is a schematic diagram showing an exemplary video shooting process.

[0252] S701: Receive a second shooting operation.

[0253] S702: In response to a second shooting operation, collect first video data.

[0254] S703: In response to the second shooting operation, point and pick up sound from the target object to collect first audio data.

[0255] Exemplarily, S701 to S703 may refer to the description of S503 to S504 above, which will not be repeated here.

[0256] S704, receiving a focus adjustment operation from the user.

[0257] Figure 8A to Figure 8E The figure is a schematic diagram of a mobile phone interface shown as an example.

[0258] For example, during the process of shooting a moving target object, the distance between the target object and the terminal device may change; for example, the distance between the target object and the terminal device may change from far to near, or from near to far. Therefore, in order to ensure that the size of the target object in the image of the first video data remains unchanged (or substantially unchanged), the user may perform a focus adjustment operation to adjust the shooting distance.

[0259] For example, when the distance between the target object and the terminal device changes from far to near, the target object occupies a larger proportion in the shooting interface 608. Fig. 8A As shown; the user can click the focus reduction option 801 in the video shooting interface 608 to reduce the target object (ie, reduce the proportion of the target object in the shooting interface 608, such as Figure 8B shown).

[0260] For example, when the distance between the target object and the terminal device changes from near to far, the target object occupies a smaller proportion in the shooting interface 608, such as Figure 8C As shown; the user can click the focus increase option 802 in the video shooting interface 608 to zoom in on the target object (ie, increase the proportion of the target object in the shooting interface 608, such as Figure 8B shown).

[0261] For example, when the distance between the target object and the terminal device changes from far to near, the target object occupies a larger proportion in the shooting interface 608. Fig.8D As shown; the user can perform a pinch operation (i.e., an operation to shorten the distance between two fingers) in the video shooting interface 608 to reduce the target object (i.e., reduce the proportion of the target object in the shooting interface 608, such as Figure 8B shown).

[0262] For example, when the distance between the target object and the terminal device changes from near to far, the target object occupies a smaller proportion in the shooting interface 608, such as Fig. 8E As shown; the user can perform a stretching operation (i.e., an operation of pulling the distance between two fingers) in the video shooting interface 608 to zoom in on the target object (i.e., increase the proportion of the target object in the shooting interface 608, such as Figure 8B shown).

[0263] That is, the focus adjustment operation in S704 may include any one of clicking the focus reduction option 801, clicking the focus increase option 802, a pinch operation, or a stretch operation.

[0264] S705 , in response to the focus adjustment operation, adjusting the beam state of the microphone.

[0265] Exemplarily, when the focus adjustment operation is clicking the focus reduction option 801 or a pinch operation, the terminal device can respond to the focus adjustment operation and instruct the audio module to call the beam adjustment algorithm to adjust the beam state of the microphone to reduce the pickup distance of the microphone, thereby improving the pickup effect for the target object; and reduce the volume of the audio data of the target object in the first audio data. Specifically, it can ensure that the beam pointing angle of the microphone remains unchanged, shorten the short axis of the left beam and the short axis of the right beam, and increase the long axis of the left beam and the long axis of the right beam.

[0266] Figure 8G FIG. 1 is a schematic diagram of a beam of a microphone shown as an example.

[0267] Combination Figure 5D and Figure 8G When the focus adjustment operation is clicking the focus reduction option 801 or a pinch operation, after the beam state of the microphone is adjusted, the beam pointing angle of the microphone remains 30 degrees, the short axes of the left beam and the right beam become shorter, and the long axes of the left beam and the right beam become longer.

[0268] Exemplarily, when the focus adjustment operation is clicking the focus increase option 802 or a stretching operation, the terminal device can respond to the focus adjustment operation and instruct the audio module to call the beam adjustment algorithm to adjust the beam state of the microphone to increase the microphone's sound pickup distance, achieve long-distance sound pickup, and improve the sound pickup effect for the target object; and increase the volume of the audio data of the target object in the first audio data. Specifically, the beam pointing of the microphone can be ensured to remain unchanged, and the short axis of the left beam and the short axis of the right beam can be shortened.

[0269] Figure 8F FIG. 4 is a schematic diagram of an exemplary beam. FIG.

[0270] Combination Figure 5D and Figure 8F When the focus adjustment operation is clicking the focus increase option 802 or a stretching operation, after the beam state of the microphone is adjusted, the beam pointing angle of the microphone remains 30 degrees, the short axes of the left beam and the right beam become shorter, and the long axes of the left beam and the right beam do not change.

[0271] Exemplarily, the image module can output video focal length information (wherein the video focal length information can include focal length information, which can be the target focal length or the focal length change corresponding to the focal length adjustment operation; for example, the current focal length of the camera is 20mm, and the focal length change corresponding to the focal length adjustment operation is +10mm, then the target focal length is 30mm) to the beam adjustment algorithm, and the beam adjustment algorithm adjusts the beam state of the microphone according to the focal length information.

[0272] S706: Perform audio collection based on the adjusted beam to obtain second audio data.

[0273] Exemplarily, after the beam state of the microphone is adjusted, the microphone can collect audio according to the adjusted beam to obtain the second video data.

[0274] S707, in response to the focus adjustment operation, adjusting the focus of the camera.

[0275] Exemplarily, the image module may determine a target focal length according to the focal length adjustment operation and the current focal length of the camera; and then adjust the focal length of the camera according to the target focal length.

[0276] S708, performing video capture based on the adjusted focal length to obtain second video data.

[0277] Afterwards, the camera can perform video acquisition according to the adjusted focal length to obtain second audio data, wherein the second audio data includes audio data of the target object and audio data of other objects.

[0278] exist Figure 7 In the embodiment, the first video data and the first audio data, the second video data and the second audio data may be stored as one multimedia file.

[0279] It should be noted that, for a terminal device including a front camera and a rear camera, a user can manually adjust the focal length during video shooting using the rear camera; Figure 7 The embodiments can be applied to a scenario where a terminal device uses a rear camera to shoot a video.

[0280] Fig. 9 The figure is a schematic diagram of a video shooting process shown as an example.

[0281] S901: Receive a second shooting operation.

[0282] S902: In response to a second shooting operation, collect first video data.

[0283] S903: In response to the second shooting operation, point and pick up sound from the target object to collect first audio data.

[0284] Illustratively, S901 to S903 may refer to the description of S503 to S504 above, which will not be repeated here.

[0285] S904: When it is detected that the distance between the target object and the terminal device changes, adjust the beam state of the microphone.

[0286] Exemplarily, the terminal device may detect in real time (or periodically) whether the distance between the target object and the terminal device changes to determine whether to adjust the beam state of the microphone.

[0287] In one possible manner, the audio module may call a beam adjustment algorithm to detect whether the distance between the target object and the terminal device has changed based on the first audio data. Specifically, the beam adjustment algorithm may determine the volume of the audio data of the target object in the first audio data; when the difference between the volume of the audio data of the target object in the first audio data determined this time and the volume of the audio data of the target object in the first audio data determined last time is greater than a volume threshold, it may be determined that the distance between the target object and the terminal device has changed.

[0288] Among them, if the difference between the volume of the audio data of the target object in the first audio data determined this time and the volume of the audio data of the target object in the first audio data determined last time is greater than 0, it means that the distance between the target object and the terminal device has become closer. At this time, the beam state of the microphone can be adjusted to reduce the sound pickup distance of the microphone to improve the sound pickup effect for the target object; and to reduce the volume of the audio data of the target object in the first audio data. If the difference between the volume of the audio data of the target object in the first audio data determined this time and the volume of the audio data of the target object in the first audio data determined last time is less than 0, it means that the distance between the target object and the terminal device has become farther. At this time, the beam state of the microphone can be adjusted to increase the sound pickup distance of the microphone to achieve long-distance sound pickup to improve the sound pickup effect for the target object; and to increase the volume of the audio data of the target object in the first audio data. For details, please refer to the above Figure 7 The description in the embodiments will not be repeated here.

[0289] In one possible manner, the image module can detect whether the distance between the target object and the terminal device has changed based on the first video data. Specifically, when the image module determines that the focal length of the camera has not changed, it can determine the area of ​​the target object in the current frame of the first video data (hereinafter referred to as area 1) and the area of ​​the target object in the previous frame of the first video data (hereinafter referred to as area 2); when the difference between area 1 and area 2 is greater than the area threshold, it can be determined that the distance between the target object and the terminal device has changed.

[0290] Among them, if area 1 minus area 2 is greater than 0, it means that the distance between the target object and the terminal device has become closer. At this time, the beam state of the microphone can be adjusted to reduce the sound pickup distance of the microphone to improve the sound pickup effect for the target object; and reduce the volume of the audio data of the target object in the first audio data. If area 1 minus area 2 is less than 0, it means that the distance between the target object and the terminal device has become farther. At this time, the beam state of the microphone can be adjusted to increase the sound pickup distance of the microphone to achieve long-distance sound pickup to improve the sound pickup effect for the target object; and increase the volume of the audio data of the target object in the first audio data. For details, please refer to the above Figure 7 The description in the embodiments will not be repeated here.

[0291] For example, during the real-time sound source tracking process, it can also be determined whether the distance between the target object and the shooting device changes.

[0292] S905: Perform audio collection based on the adjusted beam to obtain third audio data.

[0293] exist Fig. 9 In the embodiment, the first video data and the first audio data, the third video data and the third audio data may be stored as one multimedia file.

[0294] Exemplarily, the third audio data includes audio data of the target object and audio data of other objects.

[0295] It should be noted that, for a terminal device including a front camera and a rear camera, when the front camera is used to shoot a video, the user usually does not adjust the focal length (or the user is not supported to adjust the focal length); furthermore, Fig. 9 The embodiment can be applied to the scenario where the terminal device uses a front camera to shoot video. It should be understood that Fig. 9 The embodiment can also be applied to a scenario where a terminal device uses a rear camera to shoot a video.

[0296] Fig.10 The figure is a schematic diagram of a video shooting process shown as an example.

[0297] S1001, receiving a second shooting operation and receiving a selection operation for a target object.

[0298] Combination Figure 5A Examples, and references Figure 6C For example, the user clicks the confirmation option in the shooting mode prompt option 606 in the video recording interface 605. After the mobile phone cancels the display of the shooting mode prompt option 606 in response to the user's operation behavior, the object selection prompt item 1101 may be displayed, such as Fig.11A As shown. Among them, Fig.11A The object selection prompt item 1101 may include one or more controls, including but not limited to: a confirmation option and a cancel option, etc. In addition, the object selection prompt item 1101 may also include prompt information "whether to select the target object".

[0299] Reference Fig.11A For example, when the user determines the target object, the user can click the confirmation option in the object selection prompt item 1101. The mobile phone responds to the user's operation behavior and cancels the display of the object selection prompt item 1101. At this time, the displayed video recording interface 605 is as shown in FIG. Fig. 11B and Fig. 11C As shown, multiple objects are displayed in the preview box of the video recording interface 605.

[0300] Reference Fig. 11B For example, when the user wants to shoot one target object, he can click on a location of the target object in the preview box of the video recording interface 605. The mobile phone responds to the user's operation and displays a focus box 1102 in the video recording interface 605.

[0301] Reference Fig. 11C For example, when the user needs to shoot two target objects, he can click the positions of the two target objects in the preview box of the recording interface 605 in sequence or simultaneously. The mobile phone responds to the user's operation and displays two focus boxes 1103 in the recording interface 605; and so on.

[0302] Refer again Figure 6C For example, the user clicks the confirmation option in the shooting mode prompt option 606 and clicks the recording start option 607 in the video recording interface 605. After the mobile phone displays the video shooting interface 608 in response to the user's operation behavior, it can display the object selection prompt item 1104, such as Fig.11D As shown, the object selection prompt item 1104 is similar to the above-mentioned object selection prompt item 1101, and will not be described again here.

[0303] Reference Fig.11D , exemplarily, when the user determines the target object, the user can click the confirmation option in the object selection prompt item 1104. The mobile phone responds to the user's operation behavior and cancels the display of the object selection prompt item 1104. At this time, multiple objects can be displayed in the video shooting interface 608, such as Fig.11E and Fig.11F shown.

[0304] Reference Fig.11EFor example, when the user needs to shoot one target object, he can click on the location of a target object in the video shooting interface 608. The mobile phone responds to the user's operation and displays a focus frame 1105 in the video shooting interface 608.

[0305] Reference Fig.11F For example, when the user needs to shoot two target objects, he can click on the locations of the two target objects in the video shooting interface 608. The mobile phone responds to the user's operation and displays two focus frames 1106 in the video shooting interface 608; and so on, which will not be repeated here.

[0306] That is to say, the present application can shoot videos for one or more target objects.

[0307] Reference Fig.11A For example, the user clicks on the cancel option in the object selection prompt 1101, and the mobile phone responds to the user's operation and cancels the display of the object selection prompt 1101; at this time, it can be determined that the user has not selected the target object, and then the terminal device can use the object with the largest area in the image captured by the camera as the target object; or use the object closest to the center of the image as the target object; this application does not limit this. Afterwards, the user clicks on the recording start option 607, and the mobile phone responds to the user's operation and executes S1002 and S1003 and displays the video shooting interface 608, such as Fig.6D shown.

[0308] Reference Fig.11D For example, the user clicks on the cancel option in the object selection prompt item 1104, and the mobile phone responds to the user's operation and cancels the display of the object selection prompt item 1104; at this time, it can be determined that the user has not selected the target object, and then, the target object can be determined by the terminal device, which will not be repeated here.

[0309] Based on the above description, in one possible manner, the selection operation for the target object may include clicking the confirmation option in the object selection prompt item 1101 (or the object selection prompt item 1104), and clicking Fig. 11B or Fig. 11C Video recording interface 605 (or click Fig.11E or Fig.11F Operations on the location of the target object in the video shooting interface 608).

[0310] It should be understood that the present application does not limit the order in which the user performs the shooting operation and the selection operation.

[0311] It should be understood that Fig. 11B , Fig. 11C , Fig.11E and Fig.11FThe focus frame may not be displayed, and this application does not impose any restrictions on this.

[0312] It should be understood that in one possible way, the user clicks Fig. 11B or Fig. 11C Video recording interface 605 (or click Fig.11E or Fig.11F The mobile phone can also display a prompt message (such as "You have successfully selected the target object") in response to the user's operation.

[0313] Combination Figure 5A Examples, and references Figure 6C For example, the user clicks the confirmation option in the shooting mode prompt option 606 in the video recording interface 605, and the mobile phone responds to the user's operation behavior, cancels the display of the shooting mode prompt option 606, and displays the object selection prompt item 1107, such as Fig.11G shown.

[0314] Reference Fig.11G For example, when the user determines the target object, he can click the object selection prompt item 1107, or click the area other than the object selection prompt item 1107 in the preview frame of the recording interface 605. In response to the user's operation behavior, the mobile phone cancels the display of the object selection prompt item 1107. At this time, the displayed recording interface 605 is as shown in FIG. Fig. 11B and Fig. 11C As shown in FIG. 6 , a plurality of objects are displayed in the preview box of the recording interface 605. In addition, the user may not click the object selection prompt item 1107, or may not click the area other than the object selection prompt item 1107 in the preview box of the recording interface 605. After waiting for a few seconds, the mobile phone will cancel the display of the object selection prompt item 1107. At this time, the displayed recording interface 605 is as shown in FIG. Fig. 11B and Fig. 11C As shown, multiple objects are displayed in the preview box of the video recording interface 605.

[0315] Reference Fig.11G When the user clicks on the object selection prompt item 1107, or clicks on the area other than the object selection prompt item 1102 in the preview box of the recording interface 605, but does not click on the location of the target object in the preview box of the recording interface 605, but directly clicks on the recording start option 607, it can also be determined that the user has not selected the target object. At this time, the target object can be determined by the terminal device, which will not be repeated here.

[0316] Refer again Figure 6CFor example, the user clicks the confirmation option in the shooting mode prompt option 606 and clicks the recording start option 607 in the video recording interface 605. After the mobile phone displays the video shooting interface 608 in response to the user's operation behavior, it can display the object selection prompt item 1108, such as Fig.11H shown.

[0317] Reference Fig.11H , illustratively, when the user determines the target object, he can click on the object selection prompt item 1108, or click on the area other than the object selection prompt item 1108 in the preview box of the video shooting interface 608. In response to the user's operation behavior, the mobile phone cancels the display of the object selection prompt item 1108. At this time, the displayed video shooting interface 608 is as shown in FIG. Fig.11E and Fig.11F As shown in FIG. 6 , a plurality of objects are displayed in the preview box of the video shooting interface 608. In addition, the user may not click the object selection prompt item 1108, or may not click the area other than the object selection prompt item 1108 in the preview box of the video shooting interface 608. After waiting for a few seconds, the mobile phone will cancel the display of the object selection prompt item 1108. At this time, the displayed video shooting interface 608 is as shown in FIG. Fig.11E and Fig.11F As shown, multiple objects are displayed in the preview box of the video shooting interface 608.

[0318] Reference Fig.11H When the user clicks on the object selection prompt item 1108, or clicks on the area other than the object selection prompt item 1108 in the preview box of the video shooting interface 608, but does not click on the location of the target object in the video shooting interface 608, it can be determined that the user has not selected the target object. At this time, the target object can be determined by the terminal device, which will not be repeated here.

[0319] In one possible manner, the target object selection operation may include clicking the object selection prompt item 1107 (or the object selection prompt item 1108), and clicking Fig. 11B or Fig. 11C Video recording interface 605 (or click Fig.11E or Fig.11F Operations on the location of the target object in the video shooting interface 608).

[0320] In one possible manner, the target object selection operation may include clicking an area other than the object selection prompt item 1107 (or the object selection prompt item 1108) in the preview frame of the video recording interface 605, and clicking Fig. 11B or Fig. 11C Video recording interface 605 (or click Fig.11E or Fig.11F Operations on the location of the target object in the video shooting interface 608).

[0321] S1002: In response to a second shooting operation and a target object selection operation, first video data is acquired.

[0322] S1003: In response to a second shooting operation and a target object selection operation, point and pick up sound for the target object to collect first audio data.

[0323] Illustratively, S1002 to S1003 may refer to the description of S503 to S504 above, which will not be repeated here.

[0324] It should be noted that when there are multiple target objects, the beam state of the microphone can be adjusted according to the position information of the multiple target objects so that the target pickup area of ​​the microphone is the intersection or union of the positions of the multiple target objects, or in other words, the target pickup area of ​​the microphone includes multiple target objects and as few other objects as possible.

[0325] It should be understood that it is also possible not to display the object selection prompt item, that is, the user does not need to select the target object, and the terminal device can determine the target object.

[0326] Fig.12 The figure is a schematic diagram showing an exemplary video shooting process.

[0327] S1201: Receive a second shooting operation.

[0328] S1202: In response to a second shooting operation, collect first video data.

[0329] S1203: In response to the second shooting operation, point and pick up sound from the target object to collect first audio data.

[0330] Illustratively, S1201 to S1203 may refer to the description of S503 to S504 above, which will not be repeated here.

[0331] S1204: Receive a target object switching operation from the user.

[0332] Reference Fig.13A , exemplarily, during the process of shooting a video of the target object, the shot video screen may be as shown in the video shooting interface 608, where the target object is A. It should be noted that during the process of shooting a video of the target object, the focus frame 1301 in the video shooting interface 608 may be displayed or not, and the present application shows the focus frame 1301 in order to illustrate the target object switching process.

[0333] Reference Fig. 13BFor example, during the process of shooting a video of target object A, if the user needs to switch the target object (for example, switch the target object from A to B), the user can click the position of user B in the preview box of the video shooting interface 608. In response to the user's operation, the mobile phone displays a focus frame 1301 at the position of B in the video shooting interface 608, and executes S1205 and S1206. It should be understood that Fig. 13B The focus frame may not be displayed, and this application does not impose any restrictions on this.

[0334] In addition, in one possible manner, the user can click on the location of B in the preview box of the video shooting interface 608, and the mobile phone can also display a prompt message (such as "target object switching successful") in response to the user's operation behavior.

[0335] S1205 , in response to the target object switching operation, collecting fourth video data for the switched target object.

[0336] Exemplarily, in response to the target object switching operation, the switched target object may be focused; then, fourth video data may be acquired; and the fourth video data may include the switched target object.

[0337] S1206: In response to the target object switching operation, perform directional sound pickup for the switched target object to collect fourth audio data.

[0338] Exemplarily, S1205 to S1206 may be similar to the process of S503 to S504 described above, and will not be described in detail here.

[0339] Exemplarily, the fourth audio data may include the third target audio data and third other audio data mentioned above.

[0340] exist Fig.12 In an embodiment of the present invention, in one possible manner, the first video data and the first audio data, the fourth video data and the fourth audio data can be stored as one multimedia file. In one possible manner, the first video data and the first audio data are stored as one multimedia file; and the fourth video data and the fourth audio data are stored as another multimedia file.

[0341] In this way, during the video shooting process, the target object can be changed according to user needs without stopping the current video recording and reshooting another video, which simplifies user operations.

[0342] In one possible manner, the audio module may call the second audio processing model to perform audio post-processing (such as sound source separation of the target object, diffuse field noise extraction, mixing calculation, signal-to-noise ratio optimization, etc.) on the first audio data to obtain the post-processed first audio data. Afterwards, the post-processed first audio data and the first video data may be stored as a multimedia file.

[0343] In this way, during the video shooting process, by post-processing the audio data collected by the directional sound pickup, the signal-to-noise ratio of the audio data of the target object in the shot video can be further improved, thereby further improving the audio quality and clarity of the audio data of the target object in the shot video.

[0344] It should be understood that the present application may also perform audio post-processing on at least one of the second audio data, the third audio data, or the fourth audio data, and the present application does not impose any limitation on this.

[0345] It should be understood that any two embodiments in the present application can be combined, and the present application does not impose any limitation on this.

[0346] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the video shooting method in the above-mentioned embodiment.

[0347] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the video shooting method in the above-mentioned embodiment.

[0348] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the video shooting method in the above-mentioned method embodiments.

[0349] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0350] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0351] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0352] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0353] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0354] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0355] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0356] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0357] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (ReadOnly Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0358] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0359] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video shooting method, characterized in that: The method includes: The photographing device receives a first photographing operation; In response to the first shooting operation, the shooting device records first target audio data and first other audio data, wherein the first target audio data is audio data of a target object, and the first other audio data is audio data of other objects; The photographing device receives a second photographing operation; In response to the second shooting operation, the shooting device records second target audio data and second other audio data, wherein the second target audio data is audio data of the target object, and the second other audio data is audio data of the other object; The signal-to-noise ratio of the second target audio data is greater than the signal-to-noise ratio of the first target audio data.

2. The method according to claim 1, characterized in that The method further comprises: The photographing device receives a selection operation on the target object; In response to the selection operation on the target object, the photographing device determines the target object.

3. The method according to claim 1, characterized in that: The method further comprises: The photographing device determines any one of the object closest to the center of the photographing picture, the object closest to the photographing device, or the object with the largest occupancy in the photographing picture as the target object.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The photographing device receives a target object switching operation; In response to the target object switching operation, the shooting device records third target audio data and third other audio data, and the third target audio data is the audio data of the switched target object.

5. The method according to any one of claims 1 to 4, characterized in that: The target object is one or more.

6. The method according to any one of claims 1 to 5, characterized in that: The second target audio data is audio data collected by the shooting device through pointed sound pickup of the target object.

7. The method according to claim 6, characterized in that The method further comprises: In response to the second shooting operation, the shooting device records target video data, where the target video data is video data of the target object; The audio data collected by the shooting device through pointing and picking up sound from the target object specifically includes: The shooting device determines the target object based on the target video data; The photographing device determines a first distance according to the target object, where the first distance is the distance between the target object and the photographing device; The shooting device performs pointing and sound pickup on the target object according to the first distance to collect the second target audio data.

8. The method according to claim 7, characterized in that The method further comprises: The shooting device performs real-time sound source tracking on the target object; If the position of the target object changes, the shooting device re-determines the first distance and performs the step of pointing and picking up sound from the target object according to the first distance to collect the second target audio data.

9. A terminal device, characterized in that: The terminal device is used to execute the video shooting method executed by any shooting device in claim 1 to claim 8.

10. The terminal device according to claim 9, characterized in that: The terminal device includes at least three groups of microphones.

11. The terminal device according to claim 10, characterized in that: The distance between any two groups of microphones in the at least three groups of microphones is greater than a preset distance.

12. The terminal device according to claim 10 or 11, characterized in that: A set of microphones includes at least one microphone.

13. A computer storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the video shooting method according to any one of claims 1 to 8.

14. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a computer or a processor, the steps of the method according to any one of claims 1 to 8 are performed.

Images