A video recording noise processing method and electronic device

By using an auxiliary sensor to collect noise data during the lens module's focusing process and performing targeted noise reduction, the impact of global filtering on the audio quality of the video recording is resolved, thus improving the audio quality of the video recording.

CN123123529APending Publication Date: 2026-11-17HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510622552.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-14
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

While global filtering reduces motor noise in the recording, it also over-processes normal ambient noise, affecting the audio quality of the recording.

Method used

By using an auxiliary sensor to collect noise data during the lens module's focusing process, targeted noise reduction is performed, avoiding additional processing of audio data in other time periods.

Benefits of technology

It improves the overall audio quality of video recording, ensures reduced noise impact from the lens module and maintains audio data quality in other time periods, and achieves precise noise suppression during focusing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123529A_ABST
    Figure CN123123529A_ABST
Patent Text Reader

Abstract

A video recording noise processing method and electronic device are disclosed, relating to the field of electronic device technology. The video recording noise processing method includes: the electronic device receiving a first operation instructing the electronic device to record video; and responding to the first operation, acquiring target video through a lens module and acquiring first audio data through an audio module. The electronic device performs noise reduction processing on data within a target time period in the first audio data to obtain second audio data. The target time period is the time period during which the lens module focuses while acquiring the target video. In this application, the electronic device only performs targeted processing on audio data within the target time period where noise is generated by the lens module, avoiding additional processing on audio data not affected by lens module noise. This reduces the impact of noise generated by the lens module while ensuring the quality of audio data in other time periods, thereby improving the overall audio effect of the video recording.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to a video recording noise processing method and electronic equipment. Background Technology

[0002] During the recording process using electronic devices, the image is focused to obtain a clear video. The motor in the electronic device moves the lens assembly back and forth to achieve focusing, causing vibrations and generating noise. The microphone in the electronic device picks up this noise during recording. To reduce the noise generated by motor movement in the recorded video, global filtering and other processing methods are applied to the audio data collected by the microphone. However, global filtering can over-process normal ambient sound, thus affecting the audio quality of the video recording. Summary of the Invention

[0003] This application provides a video recording noise processing method and electronic device to solve the problem that the global filtering method over-processes normal ambient sound, thereby affecting the audio quality of the video recording.

[0004] Firstly, this application provides a video recording noise processing method. This method can be applied to a computer system or to an electronic device that supports the computer system in implementing the method, such as a mobile phone. In one possible example, the aforementioned electronic device includes a lens module and an audio module. The video recording noise processing method includes: the electronic device receiving a first operation instructing it to record video; and then, in response to the first operation, acquiring a target video recording through the lens module and acquiring first audio data through the audio module. The electronic device performs noise reduction processing on data within a target time period in the first audio data to obtain second audio data. The target time period is the time period during which the lens module focuses while acquiring the target video recording.

[0005] In this application, the electronic device only performs targeted processing on the audio data within the target time period in which the lens module generates noise, avoiding additional processing on the audio data that is not affected by the noise from the lens module. This reduces the impact of noise generated by the lens module while ensuring the quality of audio data in other time periods, thereby improving the overall audio effect of the recording.

[0006] In one possible scenario, the above-described video noise processing method further includes: the electronic device acquiring a focus start timestamp and a focus end timestamp. These focus start timestamps and focus end timestamps are used to indicate the target time period.

[0007] In this application, the electronic device can accurately obtain the target time period by using the focus start time stamp and focus end time stamp, and then perform noise reduction processing on the data in the first audio data that falls within the target time period. This can accurately reduce the noise impact generated by the lens module while ensuring the quality of audio data in other time periods, thereby further improving the overall audio effect of the recording.

[0008] In one possible example, the time period from the focus start timestamp to the focus end timestamp is the target time period.

[0009] In one possible example, the focus information collected by the electronic device includes a focus start timestamp and a focus end timestamp.

[0010] In one possible implementation, the audio module includes a main sensor and an auxiliary sensor. The main sensor is used to collect first sub-audio data, which indicates ambient sound and noise generated by the lens module. The auxiliary sensor is used to collect second sub-audio data, which indicates noise generated by the lens module. The first audio data includes both the first and second sub-audio data. The electronic device performs noise reduction processing on the data within a target time period in the first audio data to obtain the second audio data. This process includes: the electronic device using the second sub-audio data to perform noise reduction processing on the data within the target time period in the first sub-audio data to obtain the second audio data.

[0011] In this application, the electronic device uses second sub-audio data, which is used to indicate the noise generated by the lens module, to eliminate or reduce the noise generated by the lens module in the first sub-audio data. That is, it performs noise reduction processing on the first sub-audio data by means of cancellation, which can improve the elimination or reduction of the noise generated by the lens module in the first sub-audio data. As a result, the second audio data obtained contains no noise or contains only a small amount of noise, thereby improving the overall audio effect of the video recording.

[0012] In one possible scenario, the aforementioned ambient sound is used to indicate the sound sources around the location of the electronic device.

[0013] In one possible scenario, the auxiliary sensor may also collect ambient sound, wherein the noise generated by the lens module and collected by the auxiliary sensor is louder than the ambient sound.

[0014] In one possible example, the electronic device uses the second sub-audio data to denoise the data in the first sub-audio data that falls within a target time period, thereby obtaining the second audio data. This includes: using spectral subtraction in the frequency domain, subtracting the noise component from the noisy signal frequency of the first sub-audio data using the noise spectrum estimate of the second sub-audio data.

[0015] In another possible example, the electronic device uses the second sub-audio data to denoise the data in the first sub-audio data that falls within the target time period, thereby obtaining the second audio data. This includes: using the second sub-audio data as a reference input, dynamically adjusting the filter parameters through an adaptive algorithm (such as least mean square (LMS) or recursive least squares (RLS)) to suppress noise in the first sub-audio data.

[0016] In one possible scenario, the auxiliary sensor is closer to the lens module than the main sensor, and the vertical projection of the lens module on the mid-frame of the electronic device and the vertical projection of the auxiliary sensor on the mid-frame overlap or are adjacent to each other.

[0017] In this application, by setting an auxiliary sensor that is closer to the lens module than the main sensor, the auxiliary sensor can more accurately collect noise generated by the lens module during focusing. Then, based on the second sub-audio data collected by the auxiliary sensor, the first sub-audio data collected by the main sensor is processed to obtain audio data (second audio data) with reduced noise generated by the lens module, thereby improving the overall audio effect of the video recording.

[0018] In one possible scenario, the auxiliary sensor and the lens module are each packaged independently.

[0019] In one possible example, if the auxiliary sensor includes a microphone, there is a gap, such as a cavity, between the auxiliary sensor and the lens module.

[0020] For example, the microphone hole of the auxiliary sensor faces the lens module, thus creating a cavity between the auxiliary sensor and the lens module.

[0021] In another possible example, if the auxiliary sensor includes a bone conduction device, then the acquisition surface of the auxiliary sensor (such as the surface where the diaphragm is located in the bone conduction device) contacts the lens module.

[0022] In this application, by placing the auxiliary sensor close to the lens module (with a gap or contact between them), the auxiliary sensor can more accurately and sensitively collect the noise generated by the lens module, thereby improving the accuracy of the second sub-audio data collected by the auxiliary sensor. As a result, the noise generated by the lens module in the second audio data obtained from the second sub-audio data is weaker, thereby further improving the overall audio effect of the video recording.

[0023] In another possible scenario, the auxiliary sensor is encapsulated within the lens module. The lens module includes a drive assembly and a lens assembly, with the drive assembly used to drive the movement of the lens assembly. For example, the aforementioned drive assembly may include a motor, and the lens assembly may include multiple lenses, etc.

[0024] In one possible example, if the auxiliary sensor includes a microphone, there is a gap between the auxiliary sensor and the driving component.

[0025] For example, the microphone hole of the auxiliary sensor faces the drive assembly, thereby creating a cavity between the auxiliary sensor and the drive assembly.

[0026] In another possible example, if the auxiliary sensor includes a bone conduction device, then the acquisition surface of the auxiliary sensor is in contact with the drive component.

[0027] In this application, by placing the auxiliary sensor close to the driving component (with a gap or contact between them), the auxiliary sensor can more accurately and sensitively collect the noise generated by the movement of the driving component, thereby improving the accuracy of the second sub-audio data collected by the auxiliary sensor. As a result, the noise generated by the driving component in the second audio data obtained from the second sub-audio data is weaker, thereby further improving the overall audio effect of the video recording.

[0028] In one possible implementation, the electronic device performs noise reduction processing on the data in the first audio data that falls within a target time period to obtain the second audio data. This includes: the electronic device determining a target noise reduction level that matches the focus step size of the lens module from the mapping relationship between focus step size and noise reduction level, and then performing noise reduction on the data in the first audio data that falls within the target time period according to the target noise reduction level to obtain the second audio data.

[0029] In this application, the electronic device performs noise reduction processing on the data in the first audio data within the target time period according to the focus step size and the matched target noise reduction level. This achieves targeted processing of the first audio data according to the focus step size, further reducing the impact of noise generated by the lens module on the final audio data (second audio data), thereby improving the overall audio effect of the recording.

[0030] In one possible scenario, the degree of noise reduction can be determined by parameters, filter type, etc. For example, if the noise reduction process is a moving average or Gaussian filter, the window size directly affects the smoothing strength. A larger window means a wider range of data is averaged, resulting in stronger noise suppression. Similarly, in a Gaussian filter, the standard deviation controls the filtering strength. A larger standard deviation means a wider filtering range and stronger noise suppression. In a low-pass filter, a lower cutoff frequency means more thorough filtering of high-frequency noise, resulting in stronger noise suppression.

[0031] In one possible example, the focus step size is used to indicate the amount of focal length change during the lens module's transition from focus point A to focus point B.

[0032] In one possible implementation, the electronic device performs noise reduction processing on data within a target time period in the first audio data to obtain second audio data. This includes: if the volume indicated by the first audio data is less than a target volume, then performing noise reduction processing on the data within the target time period in the first audio data according to a first noise reduction level to obtain the second audio data. If the volume indicated by the first audio data is greater than or equal to the target volume, then performing noise reduction processing on the data within the target time period in the first audio data according to a second noise reduction level to obtain the second audio data. The first noise reduction level is greater than the second noise reduction level.

[0033] In this application, the electronic device performs noise reduction processing on the first audio data according to the magnitude relationship between the volume indicated by the first audio data and the target volume, and performs noise reduction processing on the first audio data according to the corresponding noise reduction level. This realizes adaptive processing of the first audio data according to the volume indicated by the first audio data, improves the flexibility of video recording noise processing, and reduces the power consumption of the electronic device in video recording noise processing.

[0034] In one possible example, the target volume mentioned above is the default value, or it can be configured by the user.

[0035] In one possible example, the volume indicated by the first audio data is the maximum volume in the first audio data.

[0036] Secondly, this application provides an electronic device. The electronic device includes a mid-frame, a lens assembly, a main sensor, and an auxiliary sensor. The lens assembly is connected to the mid-frame and is used for shooting; the main sensor is connected to the mid-frame and is used to collect a first sound from the lens module during shooting; the auxiliary sensor is connected to the mid-frame and is used to collect a second sound from the lens module during shooting. The auxiliary sensor is closer to the lens module than the main sensor, and the vertical projection of the lens module onto the mid-frame and the vertical projection of the auxiliary sensor onto the mid-frame overlap or are adjacent.

[0037] In this application, by setting an auxiliary sensor that is closer to the lens module than the main sensor, the auxiliary sensor can more accurately collect noise generated by the lens module during focusing. Based on the noise collected by the auxiliary sensor, the sound collected by the main sensor can be processed to obtain audio data that reduces the noise generated by the lens module, thereby improving the overall audio effect of the video recording.

[0038] In one possible implementation, the middle frame has a first end and a second end that are positioned opposite each other along the length direction, and the main sensor is connected to the first end or the second end.

[0039] In one possible implementation, the auxiliary sensor and lens module are arranged along the thickness direction of the mid-frame.

[0040] In one possible implementation, the acquisition surface of the auxiliary sensor is in contact with the lens module, or there is a gap between the auxiliary sensor and the lens module.

[0041] In one possible implementation, the lens module includes a drive assembly and a lens assembly, the drive assembly being used to drive the movement of the lens assembly. The acquisition surface of the auxiliary sensor is in contact with the drive assembly, or there is a gap between the auxiliary sensor and the drive assembly.

[0042] Thirdly, this application provides a chip. The chip includes: a processor and an interface circuit; the interface circuit is used to acquire a first operation, and the processor is used to execute the method in the first aspect or any possible implementation of the first aspect according to the first operation.

[0043] Fourthly, this application provides an electronic device. The electronic device includes a memory and a processor, the memory storing computer instructions; when the processor executes the computer instructions, it implements the method described in the first aspect or any possible implementation of the first aspect.

[0044] Fifthly, this application provides a computer-readable storage medium. The storage medium stores a computer program or instructions that, when executed by a processing device, implement the method described in the first aspect or any possible implementation thereof.

[0045] Sixthly, this application provides a computer program product. The computer program product includes a computer program or instructions that, when executed by a processing device, implement the method described in the first aspect or any possible implementation thereof.

[0046] The beneficial effects of aspects two through six above can be referred to in the first aspect or any possible implementation of the first aspect, and will not be elaborated here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of a lens module included in an electronic device.

[0048] Figure 2 This is a schematic diagram of an electronic device;

[0049] Figure 3 This is a schematic diagram of the structure of a mobile phone provided in an embodiment of this application;

[0050] Figure 4 A flowchart illustrating a video noise processing method provided in this application embodiment. Figure 1 ;

[0051] Figure 5 A flowchart illustrating a video noise processing method provided in this application embodiment. Figure 2 ;

[0052] Figure 6 A schematic diagram of the structure of an electronic device provided in this application embodiment. Figure 1 ;

[0053] Figure 7 Positional relationship between the lens module and the auxiliary sensor provided in the embodiments of this application Figure 1 ;

[0054] Figure 8 Positional relationship between the lens module and the auxiliary sensor provided in the embodiments of this application Figure 2 ;

[0055] Figure 9 A schematic diagram of the structure of an electronic device provided in this application embodiment. Figure 2 ;

[0056] Figure 10 A schematic diagram illustrating the sound pickup range of an omnidirectional microphone and a directional microphone provided in the embodiments of this application;

[0057] Figure 11 A schematic diagram of the structure of an electronic device provided in this application embodiment. Figure 3 . Detailed Implementation

[0058] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0059] A lens module includes a lens (group), motor, filter, image sensor, bracket, and circuit board, and is used for taking pictures. It is worth noting that when an electronic device includes multiple lens modules, the area where these multiple lens modules converge can be called the camera decoration (DECO) area. For example... Figure 1 As shown, Figure 1 This is a schematic diagram of a lens module included in an electronic device. Figure 1 The electronic device shown includes three lens modules, such as lens module 101, lens module 102, and lens module 103. The aforementioned camera DECO area includes lens module 101, lens module 102, and lens module 103.

[0060] It is worth noting that, Figure 1The lens module shown is only the light entrance hole of the lens module. Below the camera DECO area (along the thickness direction of the electronic device), there are also drive components, lens components, etc. included in the lens module.

[0061] The motors in lens modules mostly use voice coil motors. Due to the electromagnetic interaction between the coil and the magnet, voice coil motors induce slight vibrations, which in turn generate noise. In one possible scenario, considering the characteristics of the camera function, a sliding shaft motor is used to achieve autofocus, and a ball bearing or sliding shaft motor is used for optical image stabilization (OIS). The friction interface introduced by these types of motors involves switching between dynamic and static friction, which increases the nonlinear friction of the lens module. The current fluctuations driving the motor are more severe compared to voice coil motors, resulting in increased motor vibration and further increased noise.

[0062] To improve the directionality of audio recording during filming on electronic devices, a rear microphone is often placed in the camera DECO area of ​​the device, with additional microphones placed at the top and bottom along the length of the device. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of an electronic device. Figure 2 The electronic device shown includes: a top microphone 201, a bottom microphone 202, a rear microphone 203, and a camera DECO area 204. The rear microphone 203 is deployed near the camera DECO area 204, or within the camera DECO area 204.

[0063] The aforementioned camera DECO area 204 is the collection of areas where cameras deployed on the back of electronic devices are located. That is, one camera DECO area 204 corresponds to one rear microphone 203.

[0064] The top microphone 201, bottom microphone 202, and rear microphone 203 mentioned above are all omnidirectional microphones. When the motor in the lens module in the camera DECO area generates noise, the microphones can easily pick up the noise, resulting in noise in the video recording.

[0065] To reduce noise generated by motor movement in the captured video, global filtering and other processing methods were applied to the audio data collected by the microphone. However, global filtering resulted in over-processing of normal ambient sound, thus affecting the audio quality of the video recording.

[0066] Based on this, this application provides a video recording noise processing method applied to an electronic device, which includes a lens module and an audio module. In one possible example, the video recording noise processing method provided by this application processes the audio data within the target time period when the lens module generates noise, from the first audio data collected by the audio module during the lens module's shooting, to obtain second audio data. This application performs targeted processing on the audio data within the target time period when the lens module generates noise, avoiding additional processing on audio data unaffected by lens module noise, ensuring the quality of audio data in other time periods, thereby improving the overall audio effect of the video recording.

[0067] To address the aforementioned microphone layout issues, this application also provides an electronic device. This device includes a main sensor and an auxiliary sensor. The auxiliary sensor is positioned closer to the lens module of the electronic device than the main sensor, with the vertical projections of the lens module onto the mid-frame overlapping or adjacent to those of the auxiliary sensor onto the mid-frame. Both the main and auxiliary sensors are used to collect sound from the lens module during the shooting process. In this application, by placing the auxiliary sensor closer to the lens module, the auxiliary sensor can more accurately collect noise generated by the lens module during focusing. Furthermore, based on the noise collected by the auxiliary sensor, the sound collected by the main sensor can be processed to obtain audio with reduced noise from the lens module, thus improving the overall audio quality of the video recording.

[0068] The aforementioned electronic devices may be: terminals, computing devices, drones, smartphones, laptops, tablets, personal desktop computers, or augmented reality (AR) devices, etc., with camera functions.

[0069] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0070] Please refer to Figure 3 , Figure 3 This is a schematic diagram of a mobile phone structure provided in an embodiment of this application. The methods in the following embodiments can be implemented in a mobile phone having the above-described hardware structure.

[0071] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, bone conduction device 170E, lens module 180, etc.

[0072] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0074] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0075] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0076] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, audio and video files can be stored on the external memory card.

[0077] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as video recording / camera recording, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0078] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0079] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0080] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0081] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 121, external memory, lens module 180, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0082] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0083] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0084] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0085] The modem processor may include a modulator and a demodulator. The modulator modulates a low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through a display screen included in the electronic device 100. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0086] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0087] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0088] In one possible embodiment, the electronic device 100 further includes a display screen. The electronic device 100 implements display functions via a GPU, a display screen, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0089] The display screen is used to display images, videos, etc. The display screen includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N displays, where N is a positive integer greater than 1.

[0090] Electronic device 100 can achieve shooting functions through ISP, lens module 180, video codec, GPU, and application processor. Alternatively, electronic device 100 can achieve shooting functions through ISP, audio module 170, lens module 180, video codec, GPU, and application processor.

[0091] The ISP (Image Signal Processor) is used to process data fed back from the lens module 180. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, etc. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the lens module 180.

[0092] Lens module 180 is used to capture still images or videos. An object passes through the lens to generate an optical image that is projected onto a photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, electronic device 100 may include one or N lens modules 180, where N is a positive integer greater than 1.

[0093] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0094] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0095] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0096] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0097] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0098] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0099] In one possible embodiment, the electronic device 100 may further include a bone conduction sensor. The bone conduction sensor can acquire vibration signals. In some embodiments, the bone conduction sensor can acquire vibration signals from the vibrating bone fragments of the human vocal cords. The bone conduction sensor can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor can also be disposed in headphones, forming bone conduction headphones. The audio module 170 can parse speech signals based on the vibration signals from the vibrating bone fragments of the vocal cords acquired by the bone conduction sensor to realize voice functionality. The application processor can parse heart rate information based on the blood pressure signals acquired by the bone conduction sensor to realize heart rate detection functionality. In some embodiments, the audio module 170 can acquire noise caused by motor movement in the lens module 180 based on the bone conduction sensor, and then use the aforementioned noise to perform noise reduction processing on the audio data collected by the lens module 180 during shooting to obtain noise-reduced audio data.

[0100] Figure 4 The video noise processing method shown can be applied to Figure 3 In the electronic device shown. For example... Figure 4 As shown, Figure 4 A flowchart illustrating a video noise processing method provided in this application embodiment. Figure 1 In this embodiment, the electronic device 100 includes a lens module and an audio module, wherein the lens module may be... Figure 3 The lens module 180 and audio module can be... Figure 3 The microphone is 170C or the bone conduction device is 170E. Figure 4 The low-altitude airspace risk quantification method shown includes the following steps S410-S430.

[0101] S410, Electronic device 100 receives the first operation.

[0102] In one possible scenario, the first operation is used to instruct the use of an electronic device to record video.

[0103] For the content of the first operation, three possible examples are provided below.

[0104] Example 1: The first action is for the user to activate the camera function of the electronic device. For example, the first action is for the user to tap "Camera" displayed on the user interface of the electronic device.

[0105] Example 2: The first operation is for the user to activate the camera function of the electronic device. For example, the first operation is for the user to click the "Record" or "Video" control in the user interface corresponding to the camera.

[0106] Example 3: The first operation is the target focus determined by the user. For example, if the camera is currently at focus A, and the user needs to switch the focus, they click on the target area within the viewfinder of the camera's user interface. The focus corresponding to this target area is the target focus, which is different from focus A.

[0107] It is worth noting that the above content is merely an example and should not be construed as limiting this application. In other embodiments of this application, the first operation may also be a user triggering a button on the electronic device, which may be a camera shortcut key, the first operation may be a user shaking the electronic device 100, or the first operation may be a user's voice command to the electronic device, such as "open the camera to record video".

[0108] For the content of the first operation, three possible examples are provided below.

[0109] Example 1: Electronic device 100 receives a trigger operation A from the user on the main user interface for the control component corresponding to the camera. Trigger operation A may be a click operation by the user on the touch screen included in electronic device 100, or a confirmation operation by the user on the control component via the keyboard, etc.

[0110] Example 2: Electronic device 100 acquires a user's trigger operation B on the "record recording" or "video recording" control component on the user interface corresponding to the camera, or a trigger operation C on an area within the viewfinder of the user interface. Both trigger operations B and C can be user click operations on the touchscreen, etc.

[0111] Example 3: When the user interface of electronic device 100 is in the same position as the user interface corresponding to the camera, the attitude data of electronic device 100 recorded by the accelerometer and / or gyroscope in electronic device 100 is acquired. If the attitude data indicates that electronic device 100 is jittering, then electronic device 100 acquires the first data.

[0112] It is worth noting that the above content is merely an example and should not be construed as limiting this application. In other embodiments of this application, the electronic device 100 obtains the user's trigger operation on the button indicating the camera shortcut key to obtain the first operation.

[0113] S420, electronic device 100 responds to the first operation by acquiring target video through the lens module and acquiring first audio data through the audio module.

[0114] In one possible scenario, the lens module, through the cooperation of the lens assembly, drive assembly, image sensor, etc., captures target video recordings, and while the electronic device is capturing target video recordings, the audio module continuously captures sound sources around the electronic device and / or sounds emitted by the electronic device, thereby obtaining the first audio data.

[0115] The content of capturing target video using the lens module and capturing first audio data using the audio module can be referred to conventional technical methods in this field, and will not be elaborated here.

[0116] In one possible scenario, in response to the first operation, the electronic device 100 will also acquire the focus information of the lens module during the acquisition of target video recording.

[0117] The focus information is used to indicate the target time period during which the lens module generates noise.

[0118] In one possible scenario, focus information includes a focus timestamp. This focus timestamp includes a focus start timestamp and a focus end timestamp. The focus start timestamp and focus end timestamp are used to indicate a target time period. For example, the time period from the focus start timestamp to the focus end timestamp is the target time period.

[0119] Taking the first operation as the user activating the camera function of electronic device 100 as an example, after activating the camera function, electronic device 100 will perform automatic focusing. This focusing process requires the motor in the lens module to move the lens group in the lens module, and the motor will generate noise during the movement of the lens group. In response to the first operation, electronic device 100 obtains the focusing information of the lens module, including: the focusing start time stamp of receiving the first operation, and the focusing end time stamp of the electronic device 100 after controlling the motor to move the lens group in response to the first operation. The time period from the aforementioned focusing start time stamp to the focusing end time stamp is the target time period.

[0120] Taking the first operation as user-induced shaking of electronic device 100 as an example, after detecting the shaking, electronic device 100 will perform autofocus (AF) according to the autofocus algorithm. During the focusing process, the motor will generate noise. In response to the first operation, electronic device 100 acquires the focusing information of the lens module, including: the focusing start time stamp when the first operation is received, and the focusing end time stamp after the autofocus is completed in response to the first operation. The period from the aforementioned focusing start time stamp to the focusing end time stamp is the target time period.

[0121] In another possible scenario, the electronic device 100, in response to the first operation, acquires a focus timestamp indicating the target time period. That is, the focus timestamp shown in this scenario is not included in the focus information, and the electronic device 100 can directly acquire the focus timestamp.

[0122] S430 and electronic device 100 perform noise reduction processing on the data in the first audio data that falls within the target time period to obtain the second audio data.

[0123] The target time period is the time during which the lens module focuses while capturing video of the target.

[0124] In one possible example, the electronic device 100 performs noise reduction processing such as smoothing or filtering on the data in the first audio data that falls within the target time period to obtain the second audio data.

[0125] For example, if the volume indicated by the first audio data is less than the target volume, the data in the first audio data that falls within the target time period is denoised according to the first denoising level to obtain the second audio data.

[0126] The aforementioned level of noise reduction can be determined by parameters, filter type, etc. For example, if the noise reduction process is a moving average or Gaussian filtering, the window size directly affects the smoothing strength. A larger window means a wider range of data is averaged, resulting in stronger noise suppression.

[0127] For example, in a Gaussian filter, the filter strength is controlled by the standard deviation. The larger the standard deviation, the wider the filtering range and the stronger the noise suppression effect. In a low-pass filter, the lower the cutoff frequency, the more thoroughly high-frequency noise is filtered out, that is, the stronger the noise suppression effect.

[0128] The following are three possible examples of the volume decibels indicated by the first audio data.

[0129] Example 1: The volume indicated by the first audio data above is the maximum volume of the first audio data.

[0130] Example 2: The volume indicated by the first audio data is any one of the first N volumes among the multiple volumes indicated by the first audio data. The aforementioned arrangement is in descending order, and N is greater than or equal to 1.

[0131] Example 3: The volume indicated by the first audio data is any one of the last N volumes among the multiple volumes indicated by the first audio data. The aforementioned arrangement is in ascending order.

[0132] It is worth noting that the above content is merely an example and should not be construed as limiting this application. In other embodiments of this application, the volume indicated by the first audio data can be any volume indicated by the audio data collected by the audio module, and the target volume can be 10 decibels (dB).

[0133] For example, if the volume indicated by the first audio data is greater than the target volume, the data in the first audio data that falls within the target time period is denoised according to the second noise reduction level to obtain the second audio data.

[0134] For the content of this example, please refer to the description of the case where the volume indicated by the first audio data is less than the target volume, which will not be repeated here.

[0135] Among them, the first level of noise reduction is greater than the second level of noise reduction.

[0136] For example, the window indicating the first level of noise reduction is larger than the window indicating the second level of noise reduction.

[0137] For example, the standard deviation of the Gaussian filter indicating the first level of noise reduction is greater than the standard deviation of the Gaussian filter indicating the second level of noise reduction.

[0138] For example, the cutoff frequency of the low-pass filter indicated by the first noise reduction level is lower than the cutoff frequency of the low-pass filter indicated by the second noise reduction level.

[0139] It is worth noting that the above content is merely an example and should not be construed as limiting this application.

[0140] It is worth noting that if the volume indicated by the first audio data is equal to the target volume, the electronic device can perform noise reduction processing on the data in the first audio data within the target time period according to the user's configuration, based on the first noise reduction level or the second noise reduction level, to obtain the second audio data.

[0141] In one possible embodiment, the aforementioned focus information is further used to indicate the focus step size of the lens module. Furthermore, the electronic device 100 performs noise reduction processing on the data within the target time period in the first audio data to obtain the second audio data. This includes: the electronic device 100 determining a target noise reduction level matching the focus step size of the lens module from the mapping relationship between the focus step size and the noise reduction level; and performing noise reduction on the data within the target time period in the first audio data according to the target noise reduction level to obtain the second audio data.

[0142] The focus step size mentioned above indicates the change in focal length during the lens module's transition from focus point A to focus point B. For example, if the focal length (distance from the focal point to the optical center of the lens module) corresponding to focus point A is 28 mm and the focal length corresponding to focus point B is 35 mm, then the focus step size is 35 - 28 = 7 mm.

[0143] In one possible scenario, the above mapping relationship is the default mapping relationship, or it can be configured by the user.

[0144] In one possible example, the degree of noise reduction in the above mapping is used to indicate at least one of the following: window size, filter type, and its parameters (such as standard deviation, cutoff frequency).

[0145] In one possible example, the electronic device 100 performs noise reduction on the data within a target time period in the first audio data according to the target noise reduction level, to obtain the second audio data. This includes: the electronic device 100 uses the window size (e.g., 10), filter type (e.g., Gaussian filter), and standard deviation size (e.g., 2) indicated by the target noise reduction level to perform noise reduction on the data within the target time period in the first audio data, to obtain the second audio data.

[0146] For example, electronic device 100 uses a Gaussian filter to denoise the first audio data within the target time period, and uses a window of size 10 and a standard deviation of 2 during the denoising process.

[0147] In one possible embodiment, the focus information is also used to indicate focal length information. When the audio module includes multiple sensors, the focal length information is used to determine the weight of the audio data collected by each sensor, and then the aforementioned multiple audio data are weighted and summed to obtain the recorded audio data, such as the first audio data described above.

[0148] It is worth noting that the electronic device 100 can perform weighted summation, noise reduction, and other processing on the audio data simultaneously during the recording process, or after the recording is completed.

[0149] In one possible embodiment, the audio module includes a main sensor and an auxiliary sensor. The main sensor is used to collect first sub-audio data, which indicates ambient sound and noise generated by the lens module. The auxiliary sensor is used to collect second sub-audio data, which indicates noise in the lens module parameters. The first audio data includes both the first and second sub-audio data.

[0150] In one possible scenario, the auxiliary sensor is closer to the lens module than the main sensor, and the vertical projection of the lens module on the mid-frame of the electronic device and the vertical projection of the auxiliary sensor on the mid-frame overlap or are adjacent to each other.

[0151] In one possible example, the auxiliary sensor is a bone conduction device. The sensor's acquisition surface is in contact with the lens module, and the noise of the lens module is collected by acquiring the vibration of the lens module.

[0152] In this example, the sensor's acquisition surface can contact either side of the lens module. For instance, when the auxiliary sensor contacts the light-incident surface of the lens module, the auxiliary sensor will be offset from the lens module's light-incident aperture.

[0153] In one possible example, if the auxiliary sensor is a microphone, there is a gap between the auxiliary sensor and the lens module, which is used to conduct noise generated by the lens module.

[0154] For example, when the auxiliary sensor is in close contact with the lens module, a cavity can be formed between the auxiliary sensor and the lens module. This cavity is used to transmit the noise generated by the lens module to the auxiliary sensor.

[0155] The above scenario illustrates the case where the lens module is packaged independently. In another possible scenario, the lens module is packaged together with the auxiliary sensor. In this scenario, the lens module includes a drive component (such as a motor) and a lens assembly, with the drive component used to drive the lens assembly.

[0156] In one possible example, if the auxiliary sensor is a bone conduction device, then the sensor's acquisition surface is in contact with the drive assembly.

[0157] In another possible example, if the auxiliary sensor is a microphone, then there is a gap between the auxiliary sensor and the driving component.

[0158] It is worth noting that even in environments with high ambient noise levels, the auxiliary sensor may still detect ambient noise. Specifically, the ratio of noise generated by the lens module to ambient noise is greater than or equal to 0 dB.

[0159] In one possible implementation, the electronic device 100 performs noise reduction processing on the data in the first audio data that falls within the target time period to obtain the second audio data, including: the electronic device 100 uses the second sub-audio data to perform noise reduction processing on the data in the first sub-audio data that falls within the target time period to obtain the second audio data.

[0160] In one possible example, electronic device 100 uses spectral subtraction in the frequency domain to subtract the noise component from the noisy signal frequency of the first sub-audio data using the noise spectrum estimate of the second sub-audio data, thereby obtaining the second audio data.

[0161] In another possible example, the electronic device 100 uses the second sub-audio data as a reference input and dynamically adjusts the filter parameters through an adaptive algorithm (such as LMS, RLS) to suppress noise in the first sub-audio data, thereby obtaining the second audio data.

[0162] exist Figure 4 Based on the content shown, this application also provides detailed information on a video recording noise processing method. For example... Figure 5 As shown, Figure 5 A flowchart illustrating a video noise processing method provided in this application embodiment. Figure 2 . Figure 5 The content shown includes the following steps ①-⑥.

[0163] Step 1: The electronic device 100 responds to the first operation and enters the recording interface to record.

[0164] The content of the first operation can be referred to the description of the first operation in S410 above, and will not be repeated here.

[0165] In one possible example, the recording interface is the user interface corresponding to the camera, which implements the recording function.

[0166] Step 2: Electronic device 100 issues a focus command through an autofocus algorithm.

[0167] In one possible example, the focus command is used to instruct the lens module to focus on the target focus in order to obtain a clearer image.

[0168] Step 3: The lens module in the electronic device 100 responds to the focus command, drives the motor in the lens module to focus, and determines the focus information.

[0169] For the content of focus information, please refer to the above description of focus information, and we will not go into it again here.

[0170] In one possible example, the lens frame containing the lens module transmits focus information to the audio frame, which then processes the information using the audio hardware abstraction layer, as described in steps ④ and ⑤ below. Through this communication (focus information) between the lens frame and the audio frame, the target time period in which noise is generated can be accurately located.

[0171] Step 4: The electronic device 100 determines the target time period of the intermittent noise generated by the lens module's focusing based on the focusing information and the first audio data collected by the audio module.

[0172] In one possible scenario, even if the focus information indicates a focus timestamp, the time delay between the motor receiving the focus command and the motor vibrating to focus may lead to inaccuracies in determining the target time period using only the focus timestamp. Motor vibration can be used to collect the noise caused by the motor vibration and record it in the first audio data.

[0173] This embodiment utilizes focus information and first audio data to more accurately determine the aforementioned target time period.

[0174] Step 5: Electronic device 100 performs noise reduction processing on the data in the first audio data that falls within the target time period to obtain the second audio data.

[0175] For details on step ⑤, please refer to [link / reference]. Figure 4 The description of S430 in the text will not be repeated here.

[0176] Step 6: Electronic device 100 outputs the noise-reduced target video.

[0177] In one possible example, the aforementioned denoised target video includes second audio data.

[0178] In one possible embodiment, this application provides three possible electronic devices, which can implement... Figure 4 , Figure 5 The content shown.

[0179] In the first possible implementation, such as Figure 6 As shown, Figure 6 A schematic diagram of the structure of an electronic device provided in this application embodiment. Figure 1 Electronic device 1 includes: a mid-frame 610, a lens module 620, a main sensor 630, and an auxiliary sensor 640. The lens module 620 is connected to the mid-frame 610 and is used for shooting. The main sensor 630 is connected to the mid-frame 610 and is used to collect the first sound from the lens module during shooting. The auxiliary sensor 640 is connected to the mid-frame 610 and is used to collect the second sound from the lens module during shooting.

[0180] In one possible scenario, the auxiliary sensor 640 is closer to the lens module 620 than the main sensor 630, and thus the auxiliary sensor 640 can collect stronger noise than the main sensor 630, which is generated by the vibration of the motor in the lens module 620.

[0181] Furthermore, the vertical projection of the lens module 620 on the mid-frame 610 and the vertical projection of the auxiliary sensor 640 on the mid-frame 610 overlap or are adjacent to each other.

[0182] In one possible example, the auxiliary sensor 640 is in contact with any side of the lens module 620. For example, when the auxiliary sensor 640 is in contact with the light-incident surface of the lens module 620, the auxiliary sensor 640 avoids the light-incident aperture of the lens module 620.

[0183] In one possible scenario, such as Figure 7 As shown, Figure 7 Positional relationship between the lens module and the auxiliary sensor provided in the embodiments of this application Figure 1 . Figure 7 The vertical projection of the lens module 620 on the mid-frame 610 and the vertical projection of the auxiliary sensor 640 on the mid-frame 610 overlap, as if the auxiliary sensor 640 is in contact with the light-incident surface of the lens module 620.

[0184] In one possible example, if the lens module 620 is packaged separately, the auxiliary sensor 640 can be attached to the surface of the lens module 620 by means of dispensing, adhesive backing of flexible printed circuit (FPC), foam pressing and sealing, etc.

[0185] In another possible example, where the lens module 620 and the auxiliary sensor are packaged together, the auxiliary sensor 640 can be attached to the surface of the drive component in the lens module 620 by means of dispensing, adhesive backing of flexible printed circuit (FPC), foam pressing and sealing, etc.

[0186] For example, after the lens module 620 and the auxiliary sensor 640 are attached, there is still a gap between them. For instance, the auxiliary sensor 640 and / or the lens module 620 have grooves on their attachment surfaces, so that after the auxiliary sensor 640 and the lens module 620 are attached, there is a space between them. This cavity is used to transmit noise generated by vibration of the drive component to the auxiliary sensor 640. For example, the receiving surface of the auxiliary sensor 640 (microphone) (the surface where the diaphragm is located in the microphone) faces the lens module 620, and the auxiliary sensor 640 has no gap leading to the outside of the electronic device.

[0187] Similarly, after the auxiliary sensor 640 and the drive assembly are attached, there is still a gap between them.

[0188] In another possible scenario, the distance between the vertical projection of the lens module 620 onto the mid-frame 610 and the vertical projection of the auxiliary sensor 640 onto the mid-frame 610 is greater than 0 and less than 20 mm.

[0189] In this case, the auxiliary sensor 640 can be sealed to a camera DECO, motherboard, or motherboard bracket that includes multiple lens modules.

[0190] In one possible scenario, the middle frame 610 has a first end and a second end that are disposed opposite to each other along the length direction, and the main sensor 630 is connected to the first end or the second end.

[0191] In one possible example, the main sensor 630 includes sensor 631 and sensor 632. Sensor 631 is connected to the first end, and sensor 632 is connected to the second end. Figure 6 As shown, sensor 631 is deployed on the top (first end) of electronic device 100, and sensor 632 is deployed on the bottom of electronic device 100.

[0192] In another possible example, the main sensor 630 also includes a sensor 633, which is deployed within the camera DECO area or on the mid-frame at a preset distance from the camera DECO area.

[0193] The aforementioned auxiliary sensor 640 can be an omnidirectional microphone or a directional microphone. When the auxiliary sensor 640 is a directional microphone, its pointing direction is the direction in which the lens module 620 records video.

[0194] It is worth noting that the above description only uses an example where the electronic device 100 includes only one lens module 620. In other embodiments of this application, an electronic device 100 may include multiple lens modules. When the electronic device 100 includes multiple lens modules, an auxiliary sensor can be provided for each of the multiple lens modules to collect the noise generated by the lens module during the focusing process. Alternatively, when the multiple lens modules included in the electronic device 100 are arranged adjacent to each other, a single auxiliary sensor can be used to collect the noise generated by the multiple lens modules during the focusing process.

[0195] The aforementioned main sensor 620 can be an omnidirectional microphone, allowing the electronic device 100 to simultaneously pick up external sounds during video recording. Alternatively, sensors 631 and 632 can be omnidirectional microphones, while sensor 633 can be a directional microphone.

[0196] In one possible implementation, the electronic device 100 utilizes the noise generated by the lens module during focusing, collected by the auxiliary sensor 640, to perform noise reduction (smoothing or filtering) on ​​the audio collected by sensors 631, 632, and 633. Based on the focal length information in the focusing information, the electronic device 100 determines the weights of the audio collected by sensors 631, 632, and 633, and then, according to these weights, performs a weighted summation of the audio collected by sensors 631, 632, and 633 to obtain the aforementioned second audio data.

[0197] Furthermore, the electronic device 100 can also perform noise reduction (removing ambient noise (such as vehicle noise)) on the second audio data to obtain the third audio data. The aforementioned noise reduction process may include filtering and deep learning-based noise reduction.

[0198] In the second possible implementation, such as Figure 8 As shown, Figure 8 Positional relationship between the lens module and the auxiliary sensor provided in the embodiments of this application Figure 2 The difference between this and the first possible implementation is that: 1) after the auxiliary sensor 640 and the lens module 620 come into contact, there is no gap between them, or after the auxiliary sensor 640 comes into contact with the driving component, there is no gap between them. For example, the acquisition surface of the auxiliary sensor (such as the surface where the diaphragm is located in a bone conduction device) comes into contact with the lens module.

[0199] 2) The auxiliary sensor 640 is a bone conduction device. Therefore, when the lens module 620 is shooting, the auxiliary sensor 640 can obtain the vibration generated by the lens module 620 during the shooting process through the solid medium, thereby obtaining the noise generated by the lens module 620 during the shooting process.

[0200] In the third possible implementation, such as Figure 9 As shown, Figure 9 A schematic diagram of the structure of an electronic device provided in this application embodiment. Figure 2 The difference between this implementation and the first possible implementation is that: 1) the audio module in this implementation only includes the main sensor 630. Figure 9 The main sensor includes sensors 631 and 632 disposed at both ends along the length of the electronic device, and sensor 633 disposed within or near the camera DECO area.

[0201] If sensor 633 is located near camera DECO, sensor 633 is offset from camera DECO along the length and / or width of electronic device, and the distance between sensor 633 and camera DECO is a preset distance, such as 1-40 mm.

[0202] 2) Sensor 633 is a directional microphone, pointing in the same direction as the recording direction of lens module 620.

[0203] In one possible scenario, the camera DECO area will avoid the pickup angle of sensor 633 (e.g., 60°-180°) to prevent the recording of noise generated by the motor in the camera DECO.

[0204] In one possible example, the aforementioned directional microphone can be any of the following: a directional microphone for a micro-electro-mechanical system (MEMS) such as a differential pressure type, a hot wire type, or a ciliary type.

[0205] When the lens module in the DECO area of ​​the camera is recording / shooting, the directional microphone mainly picks up the sound source in front of the camera. It is less sensitive to the sound source at the rear and sides (such as the noise generated by the motor in the lens module). As a result, the noise generated by the motor in the lens module has less interference with the directional microphone, so that at least one microphone (sensor 631, sensor 632, sensor 633) can pick up a relatively clean sound source.

[0206] 3) The electronic device 100 uses the audio data collected by the sensor 633 as a reference to process and synthesize the audio data collected by the sensor 631 and the sensor 632.

[0207] In one possible example, the above processing is used to indicate smoothing, filtering, etc.

[0208] In the fourth possible implementation, the difference from the fourth possible implementation mentioned above is that: 1) Sensor 633 is an omnidirectional microphone.

[0209] 2) A directional microphone is added to the top or bottom of the electronic device along its length, and the direction of the directional microphone is consistent with the recording direction of the lens module 620. For example, the sound-receiving hole of the aforementioned directional microphone is located on the back panel of the electronic device, that is, on the back panel where the light-receiving hole of the lens module 620 is located.

[0210] For example, the pickup angle of a directional microphone is 60°-180°.

[0211] Regarding the pickup range of omnidirectional and directional microphones, the following is a possible example. Figure 10 As shown, Figure 10 This is a schematic diagram showing the sound pickup range of an omnidirectional microphone and a directional microphone provided in an embodiment of this application. Figure 10 (a) shows a schematic diagram of the pickup range of an omnidirectional microphone, that is, an omnidirectional microphone can pick up sound from sound sources around the microphone in 360 degrees, as shown by the bold line in the figure. Figure 10 (b) shows a schematic diagram of the pickup range of an omnidirectional microphone. That is, a directional microphone mainly picks up sound from the sound source in the direction the microphone is pointing, and its pickup range is a "cardioid" range, as shown by the bold line in the figure.

[0212] It is understood that, in order to achieve the aforementioned video recording noise processing functions, electronic devices include hardware and / or software modules that perform the respective functions. Based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0213] This embodiment can divide the electronic device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0214] This application also provides a video recording noise processing apparatus, which is used to perform various functions or steps performed by the electronic device in the above method embodiments. The apparatus includes: a receiving module, an acquiring module, and a processing module. The receiving module is used to receive a first operation; the first operation instructs the electronic device to record video. The second acquiring module is used to, in response to the first operation, acquire target video through a lens module and acquire first audio data through an audio module. The processing module is used to perform noise reduction processing on data within a target time period in the first audio data to obtain second audio data; wherein, the target time period is the time period during which the lens module focuses while acquiring the target video.

[0215] This application also provides an electronic device including the aforementioned video recording noise processing apparatus. The electronic device may be a processor in other devices with shooting capabilities. Optionally, the electronic device may be a vehicle, a smartphone, a tablet, AR glasses, or other electronic devices. The electronic device can obtain a first operation from the vehicle to execute the various functions or steps performed by the electronic device in the above-described method embodiments according to the first operation.

[0216] This application also provides an electronic device, such as... Figure 11 As shown, Figure 11 A schematic diagram of the structure of an electronic device provided in this application embodiment. Figure 3 The electronic device 1000 includes a memory 1102, a processor 1104, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the electronic device to perform the various functions or steps in the above-described method embodiments.

[0217] In one possible scenario, the electronic device 100 also includes a bus 1106 connecting the memory 1102 and the processor 1104, and a communication interface 1108, which is connected to the memory 1102 and the processor 1104 via the bus 1106.

[0218] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. The interface circuit can read instructions stored in a memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform various functions or steps in the above method embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0219] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct an electronic device to perform a video recording noise processing method.

[0220] This application also provides a computer program product containing instructions. This computer program product may be a software or program product containing instructions, capable of running on an electronic device or stored on any usable medium. When the computer program product is run on at least one electronic device, it causes the at least one electronic device to perform the above-described video recording noise processing method.

[0221] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0223] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0224] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0225] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0226] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0227] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing video noise, characterized in that, The method is applied to an electronic device, the electronic device including a lens module and an audio module, and the method includes: Receive a first operation; the first operation is used to instruct the electronic device to record video. In response to the first operation, the target video is captured through the lens module, and the first audio data is captured through the audio module; The data in the first audio data that falls within the target time period is subjected to noise reduction processing to obtain the second audio data; wherein, the target time period is the time period during which the lens module focuses while capturing the target video recording.

2. The method according to claim 1, characterized in that, The method further includes: acquiring a focus start time stamp and a focus end time stamp, wherein the focus start time stamp and the focus end time stamp are used to indicate the target time period.

3. The method according to claim 1 or 2, characterized in that, The audio module includes a main sensor and an auxiliary sensor; the main sensor is used to collect first sub-audio data, which is used to indicate ambient sound and noise generated by the lens module; the auxiliary sensor is used to collect second sub-audio data, which is used to indicate noise generated by the lens module; the first audio data includes the first sub-audio data and the second sub-audio data. The step of performing noise reduction processing on the data within the target time period in the first audio data to obtain the second audio data includes: Using the second sub-audio data, noise reduction processing is performed on the data in the first sub-audio data that falls within the target time period to obtain the second audio data.

4. The method according to claim 3, characterized in that, The auxiliary sensor is closer to the lens module than the main sensor; the vertical projection of the lens module on the mid-frame of the electronic device and the vertical projection of the auxiliary sensor on the mid-frame overlap or are adjacent.

5. The method according to claim 3 or 4, characterized in that, The sampling surface of the auxiliary sensor is in contact with the lens module, or there is a gap between the auxiliary sensor and the lens module.

6. The method according to claim 3 or 4, characterized in that, The lens module includes a drive component and a lens component, the drive component being used to drive the lens component to move; the acquisition surface of the auxiliary sensor is in contact with the drive component, or there is a gap between the auxiliary sensor and the drive component.

7. The method according to claim 3 or 4, characterized in that, The auxiliary sensor includes a microphone, or a bone conduction device.

8. The method according to any one of claims 1 to 7, characterized in that, The step of performing noise reduction processing on the data within the target time period in the first audio data to obtain the second audio data includes: Based on the mapping relationship between focus step size and noise reduction level, the target noise reduction level that matches the focus step size of the lens module is determined; Based on the target noise reduction level, the data in the first audio data that falls within the target time period is denoised to obtain the second audio data.

9. The method according to any one of claims 1 to 8, characterized in that, The step of performing noise reduction processing on the data within the target time period in the first audio data to obtain the second audio data includes: If the volume indicated by the first audio data is less than the target volume, then the data in the first audio data that falls within the target time period is subjected to noise reduction processing according to the first noise reduction level to obtain the second audio data; If the volume indicated by the first audio data is greater than or equal to the target volume, then the data in the first audio data that falls within the target time period is denoised according to the second noise reduction level to obtain the second audio data; wherein, the first noise reduction level is greater than the second noise reduction level.

10. An electronic device, characterized in that, The electronic device includes: Mid-frame The lens module is connected to the mid-frame and is used for shooting. The main sensor, connected to the mid-frame, is used to collect the first sound from the lens module during the shooting process; An auxiliary sensor, connected to the mid-frame, is used to collect the second sound from the lens module during the shooting process; The auxiliary sensor is closer to the lens module than the main sensor; the vertical projection of the lens module on the mid-frame and the vertical projection of the auxiliary sensor on the mid-frame overlap or are adjacent.

11. The device according to claim 10, characterized in that, The middle frame has a first end and a second end that are arranged opposite to each other along the length direction, and the main sensor is connected to the first end or the second end.

12. The device according to claim 10 or 11, characterized in that, The auxiliary sensor and the lens module are arranged along the thickness direction of the mid-frame.

13. The device according to any one of claims 10 to 12, characterized in that, The sampling surface of the auxiliary sensor is in contact with the lens module, or there is a gap between the auxiliary sensor and the lens module.

14. The device according to any one of claims 10 to 12, characterized in that, The lens module includes a drive component and a lens component, the drive component being used to drive the lens component to move; the acquisition surface of the auxiliary sensor is in contact with the drive component, or there is a gap between the auxiliary sensor and the drive component.

15. An electronic device, characterized in that, Including processor and memory; The processor is configured to execute instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 9.

16. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by a computing device, implement the method of any one of claims 1 to 9.

17. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a computing device, the method of any one of claims 1 to 9 is implemented.