Signal processing method and apparatus, and device

Abstract

Disclosed are a signal processing method and apparatus, and a device, which relate to the field of media. A sound pickup device includes a plurality of microphones, wherein the plurality of microphones have different orientations. The method comprises: acquiring a plurality of signals collected by a plurality of microphones, wherein the plurality of signals comprise a first signal collected by a first microphone and a second signal collected by a second microphone; and in response to the first signal being less affected by wind noise than the second signal, outputting a processed signal obtained from the first signal. Therefore, on the basis of the characteristic that wind noise attenuates rapidly with angle changes, by means of configuring the plurality of microphones in the sound pickup device to have different orientations, the impact of wind noise on a signal collected by at least one of the plurality of microphones is reduced. A processed signal obtained from the signal that is least affected by wind noise is output. Compared with optimizing only an algorithm or optimizing only a hardware structure, combining hardware structure design with algorithm scheme optimization effectively improves the effect of removing wind noise, thereby reducing the impact of wind noise on an output signal.

Description

Signal processing method, device and equipment

[0001] The present application claims priority from the Chinese patent application No. 202411377863.0 filed on September 29, 2024, and entitled "Signal processing method, device and equipment", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of media, in particular to a signal processing method, device and equipment. BACKGROUND

[0003] Currently, the sound pickup device may be affected by wind noise in the process of picking up sound. Usually, the effect of wind noise is reduced by changing the structure of the sound pickup device or by algorithm. However, the effect of wind noise reduction is very small, and the listening experience is poor. SUMMARY

[0004] The present application provides a signal processing method, device and equipment, thereby effectively reducing wind noise and improving listening experience.

[0005] In a first aspect, a signal processing method is provided, applied to a sound pickup device, the sound pickup device comprising a plurality of microphones, the orientations of the plurality of microphones being different, the method comprising: obtaining a plurality of signals collected by the plurality of microphones under wind noise, wherein one microphone corresponds to one signal, the plurality of signals comprising a first signal collected by a first microphone and a second signal collected by a second microphone; in response to the first signal being less affected by wind noise than the second signal, outputting a processed signal obtained from the first signal.

[0006] The scheme provided by the present application is based on the characteristic that wind noise rapidly decreases with the change of angle, by setting the orientations of the plurality of microphones in the sound pickup device to be different, the signal collected by at least one microphone of the plurality of microphones is less affected by wind noise. The processed signal obtained from the signal least affected by wind noise is output. Compared with optimizing only the algorithm or only the hardware structure, the combination of hardware structure design and algorithm scheme optimization effectively improves the effect of wind noise reduction and reduces the influence of wind noise on the output signal. For larger wind noise (such as wind noise above three or four levels), the wind noise suppression effect of the present application is better.

[0007] In a possible implementation, the first signal being less affected by wind noise than the second signal includes: the energy of the first signal on a first frequency band being less than the energy of the second signal on the first frequency band, and the gain of the first signal on the first frequency band being greater than the gain of the second signal on the first frequency band, the first frequency band being a frequency band in which wind noise is located.

[0008] In another possible implementation, the wind noise affecting the first signal includes energy of the first signal in the first frequency band and gain of the first signal.

[0009] In another possible implementation, the processed signal is obtained from the first signal when the first signal satisfies a first condition, and the first condition is satisfied when the energy of the first signal in the first frequency band is less than a first threshold and the gain of the first signal in the first frequency band is within a second gain range.

[0010] In some embodiments, the first signal is processed according to the gain of the first signal, and the processed signal is output.

[0011] Wind noise mostly exists in low frequencies. The energy and gain of the signal in the low frequencies are used to determine whether the signal is affected by wind noise, thereby improving the accuracy of wind noise identification. When the signal satisfies the first condition, it indicates that the signal is less affected by wind noise. Then, the signal is processed according to the gain, and the signal less affected by wind noise is output, thereby effectively improving the effect of wind noise removal and reducing the influence of wind noise on the output signal.

[0012] In another embodiment, the first signal is input into an artificial intelligence model, and the gain of the first signal in the first frequency band is output. The first signal is processed according to the gain of the first signal, and the processed signal is output, including: the first signal is processed according to the gain of the first signal in the first frequency band, and the processed signal is output.

[0013] The gain of the signal collected by the microphone is identified by the artificial intelligence model, which is more accurate than the traditional signal processing method for identifying the gain. Wind noise can be removed at a larger wind speed, thereby improving the accuracy of gain identification.

[0014] In another possible implementation, the first condition is satisfied when the difference between the energy of the first signal in the first frequency band and the energy of the first signal in the second frequency band is less than a second threshold.

[0015] The low-frequency energy and high-frequency energy of the signal are used to determine whether the signal is affected by wind noise, thereby further improving the accuracy of wind noise identification.

[0016] In another possible implementation, the processed signal is obtained from the first signal when the first signal satisfies a third condition, and the third condition is satisfied when the energy of the first signal in the first frequency band is less than the energy of the second signal in the first frequency band, the gain of the first signal in the first frequency band is greater than the gain of the second signal in the first frequency band, or the difference between the gain of the first signal in the first frequency band and the gain of the second signal in the first frequency band is greater than a third threshold.

[0017] In another possible implementation, the method further includes: in response to the wind noise impact on the first signal being greater than the wind noise impact on the second signal, outputting a processed signal obtained from the second signal.

[0018] In some embodiments, the second signal is processed according to a gain of the second signal, and the processed signal is outputted.

[0019] The wind noise impact on the signals collected by the plurality of microphones varies in any way. According to the signal processing method provided in the present application, the signal with the least wind noise impact is selected from the plurality of signals collected by the plurality of microphones, and a processed signal obtained from the signal with the least wind noise impact is outputted, thereby effectively reducing wind noise and improving listening experience.

[0020] In another possible implementation, the energy of the processed signal on the first frequency band is less than the first threshold, and the gain variation of the processed signal on the first frequency band is within the first gain range.

[0021] The energy of the processed signal on the first frequency band being less than the first threshold and the gain variation of the processed signal on the first frequency band being within the gain range indicate that the outputted processed signal is less affected by wind noise and the gain variation is stable, thereby effectively improving listening experience.

[0022] In another possible implementation, in a case where the first signal and the second signal satisfy a second condition, the processed signal is obtained from the first signal and the second signal, and the second condition includes that the energy on the first frequency band is greater than the first threshold and the gain on the first frequency band is outside the second gain range.

[0023] In another possible implementation, the energy of the processed signal on the first frequency band is obtained from the energy of the first signal on the first frequency band and the energy of the second signal on the first frequency band, and the gain of the processed signal on the first frequency band is obtained from the gain of the first signal on the first frequency band and the gain of the second signal on the first frequency band.

[0024] In a case where the wind noise impact on the signals collected by the plurality of microphones is relatively large, the plurality of signals are combined and processed, that is, the signal with the least wind noise impact is integrated from the plurality of signals, and a new signal is combined as a processed signal, thereby improving the wind noise suppression effect, outputting the signal with the least wind noise impact, effectively improving the wind noise reduction effect, reducing the wind noise impact on the output signal, and improving listening experience.

[0025] In another possible implementation, the plurality of microphones include a first microphone and a second microphone facing in opposite directions.

[0026] In another possible implementation, the plurality of microphones include a first microphone and a second microphone facing in perpendicular directions.

[0027] In another possible implementation, the multiple microphones include a first microphone and a second microphone facing an arbitrary included angle.

[0028] Due to the characteristic that the wind noise rapidly decreases with the angle, by setting different orientations of the multiple microphones in the sound pickup device, the signal collected by at least one microphone of the multiple microphones is less affected by the wind noise. In combination with the hardware structure design and algorithm optimization, the effect of wind noise reduction is effectively improved, and the influence of the wind noise on the output signal is reduced.

[0029] In a second aspect, a signal processing apparatus is provided for implementing the various methods described above. The signal processing apparatus includes modules, units, or means corresponding to the modules, units, or means for implementing the methods described above, which can be implemented by hardware, software, or by a combination of hardware and software. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0030] In some possible designs, the signal processing apparatus includes a processing module and an interface module. The interface module is configured to perform the functions of sending and / or receiving in any of the aspects and any possible implementation thereof. The processing module can be configured to perform the processing functions in any of the aspects and any possible implementation thereof.

[0031] For example, the processing module is configured to obtain multiple signals collected by multiple microphones in a wind noise environment, where one microphone corresponds to one signal, and the multiple signals include a first signal collected by a first microphone and a second signal collected by a second microphone; and the processing module is configured to output, through the interface module, a processed signal obtained from the first signal in response to the wind noise affecting the first signal being less than the wind noise affecting the second signal.

[0032] In a possible implementation, the wind noise affecting the first signal being less than the wind noise affecting the second signal includes that an energy of the first signal in a first frequency band is less than an energy of the second signal in the first frequency band, and a gain of the first signal in the first frequency band is greater than a gain of the second signal in the first frequency band, the first frequency band being a frequency band in which the wind noise is located.

[0033] In another possible implementation, the method further includes that the processing module is further configured to output, through the interface module, a processed signal obtained from the second signal in response to the wind noise affecting the first signal being greater than the wind noise affecting the second signal.

[0034] In another possible implementation, an energy of the processed signal in the first frequency band is less than a first threshold, and a gain variation of the processed signal in the first frequency band is within a gain range.

[0035] In a third aspect, a signal processing apparatus is provided, comprising a processor and a memory, the memory being configured to store computer instructions that, when executed by the processor, cause the communication apparatus to perform the method of any one of the first aspect. The memory can be coupled with the processor, or can be independent of the processor.

[0036] In a fourth aspect, a pickup device is provided, comprising a plurality of microphones with different orientations and a processor configured to perform the method of any one of the first aspect.

[0037] In a possible implementation, the plurality of microphones comprises a first microphone and a second microphone with opposite orientations.

[0038] In another possible implementation, the plurality of microphones comprises a first microphone and a second microphone with perpendicular orientations.

[0039] In another possible implementation, the plurality of microphones comprises a first microphone and a second microphone with arbitrary included angles.

[0040] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program or instructions, when executed on a signal processing apparatus, causes the signal processing apparatus to perform the method of any one of the first aspect or any implementation thereof.

[0041] In a sixth aspect, a computer program product is provided, which comprises instructions, when executed on a signal processing apparatus, causes the signal processing apparatus to perform the method of any one of the first aspect or any implementation thereof.

[0042] The technical effects brought by any implementation of the second aspect to the sixth aspect can be referred to the technical effects brought by the corresponding implementation of the first aspect, which will not be repeated here.

[0043] Any one of the various possible implementations of the above aspects can be combined, provided that the schemes are not contradictory. BRIEF DESCRIPTION OF DRAWINGS

[0044] FIG. 1 is a structural schematic diagram of a pickup device provided by the present application;

[0045] FIG. 2 is a schematic diagram of a microphone distribution provided by the present application;

[0046] FIG. 3 is a flowchart of a signal processing method provided by the present application;

[0047] FIG. 4 is a schematic diagram of an artificial intelligence wind noise reduction model provided by the present application;

[0048] FIG. 5 is a structural schematic diagram of a signal processing apparatus provided by the present application. Detailed Implementation

[0049] To facilitate understanding, the main terms used in this application will be explained first.

[0050] Wind noise, also known as wind noise or aerodynamic noise, is generated by the interaction between moving objects in the flow field, or by the interaction between fluids caused by the turbulent motion of the fluid itself.

[0051] For example, scenarios affected by wind noise include, but are not limited to: audio frequency acquisition, the interior of a car traveling at high speed, ultra-high-speed aircraft, and wind tunnel equipment.

[0052] The generation mechanism of wind noise varies in different scenarios. For example, in audio acquisition scenarios, wind noise is mainly caused by turbulent airflow near the microphone diaphragm, resulting in a relatively high signal level from the microphone. Wind noise primarily occurs in the low-frequency range (e.g., 0–500 Hz) and rapidly attenuates towards higher frequencies. Sudden gusts often cause wind noise lasting from a few to several hundred milliseconds. Due to the suddenness of gusts, wind noise can generate high amplitudes exceeding the nominal speech amplitude, thus exhibiting significant non-stationary characteristics.

[0053] A microphone, also known as a transducer, microphone, or microphone, is an energy conversion device that converts sound signals into electrical signals.

[0054] Gain is designed to increase the transmission strength of sound, making it loud enough to travel a long distance. For example, by increasing the amplitude of a signal, a weak sound is made louder, thus maintaining clarity and audibility during recording or transmission. In some examples, gain is used to indicate the difference between a noisy input signal (frequency domain signal) and a noisy signal.

[0055] To address the issue of wind noise affecting the signal acquired by microphones, this application provides a signal processing method applicable to a sound pickup device requiring outdoor sound pickup. The sound pickup device includes multiple microphones with different orientations. The method includes: acquiring multiple signals acquired by the multiple microphones under wind noise conditions, wherein one microphone corresponds to one signal, and the multiple signals include a first signal acquired by a first microphone and a second signal acquired by a second microphone; and outputting a processed signal obtained from the first signal in response to the first signal being less affected by wind noise than the second signal.

[0056] The scheme provided in the application is based on the characteristic that wind noise rapidly decreases with the change of angle, and by setting different orientations of multiple microphones in the sound pickup device, the influence of wind noise on the signals collected by at least one of the multiple microphones is reduced. A processed signal obtained from the signal least affected by wind noise is output. Compared with only optimizing the algorithm or only optimizing the hardware structure, the combination of hardware structure design and algorithm scheme optimization effectively improves the wind noise reduction effect and reduces the influence of wind noise on the output signal. For larger wind noise (such as wind noise above three or four levels), the wind noise suppression effect of the application is better.

[0057] The sound pickup device described in the application includes a wireless microphone, a monitoring device, or a wearable device, etc. with a sound pickup function. There is a strong demand for reducing wind noise in outdoor live broadcast scenarios using wireless microphones. There is a strong demand for reducing wind noise in outdoor monitoring scenarios of monitoring devices in far-field sound pickup scenarios. There is a strong demand for reducing wind noise in outdoor communication scenarios of wearable devices.

[0058] The implementation of the embodiments of the application will be described in detail below in combination with the accompanying drawings of the specification.

[0059] Fig. 1 is a structural schematic diagram of a sound pickup device provided in the application. As shown in Fig. 1, the sound pickup device 100 includes a processor 110, a microphone 120, a storage medium 130, and an interface circuit 140. The processor 110, the microphone 120, the storage medium 130, and the interface circuit 140 are connected.

[0060] In the embodiment, the sound pickup device 100 contains multiple microphones 120. The orientations of the multiple microphones 120 are different.

[0061] In some embodiments, the orientations of the multiple microphones 120 have an included angle, for example, the orientations of the sound pickup holes of the multiple microphones 120 have an included angle, to ensure that when wind noise is generated, one microphone is relatively less affected by wind noise. The application does not limit the angle of the included angle between the orientations of the microphones. The orientations of the multiple microphones 120 are preferably set to have a larger included angle.

[0062] For example, the multiple microphones include a first microphone and a second microphone with opposite orientations. As shown in (a) of Fig. 2, the two microphones are distributed on different planes. Microphone 1 is arranged on plane 1, and microphone 2 is arranged on plane 2. Plane 1 and plane 2 are parallel, and the orientation of microphone 1 and the orientation of microphone 2 are opposite. In the case where the wind blows towards microphone 2, microphone 2 is greatly affected by wind noise, and microphone 1 is less affected by wind noise.

[0063] For another example, the plurality of microphones includes a first microphone and a second microphone with arbitrary angles. As shown in (c) of FIG. 2, the plurality of microphones are arranged in a ring. The plurality of microphones have different orientations.

[0064] For another example, the plurality of microphones includes a first microphone and a second microphone with arbitrary angles. As shown in (c) of FIG. 2, the plurality of microphones are arranged in a ring. The plurality of microphones have different orientations.

[0065] In some embodiments, at least one microphone of the plurality of microphones is arranged at a wind-avoiding position of the sound pickup device. For example, the microphone is arranged at a position close to the skin on the sound pickup device.

[0066] With the increase of the number of microphones, the effect of wind noise reduction is more and more obvious, and the effect of wind noise reduction is improved by increasing the number of microphones.

[0067] In some embodiments, the processor 110 is a CPU, and the processor 110 is also other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor is a microprocessor or any conventional processor.

[0068] In this application, the processor 110 is used to calculate the gain required for wind noise reduction of the signals collected by the plurality of microphones, and to use the characteristics that the wind noise affects the plurality of microphones with different orientations differently, to integrate the processing results of the signals collected by the plurality of microphones, and to output the signals after wind noise reduction.

[0069] As an example, the sound pickup device 100 includes a plurality of processors. The processor is a multi-CPU processor. For example, CPU0 and CPU1 in FIG. 1. The processor here can refer to one or more devices, circuits, and / or computing units for processing data (such as computer program instructions).

[0070] As an example, the sound pickup device 100 further includes an NPU, a DPU, or one or more integrated circuits for controlling the execution of the program of the present application. For example, the sound pickup device 100 further calculates the gain required for wind noise reduction of the signals collected by the plurality of microphones according to an artificial neural network.

[0071] It is worth mentioning that only one processor 110 is included in the pickup device 100 in FIG. 1 as an example, and here the processor 110 is used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined according to business needs.

[0072] The interface circuit 140 is used to realize the communication between the pickup device 100 and external devices or equipment.

[0073] The storage medium 130 can be used to store relevant information in the signal processing process, such as gain, signal energy, amplitude, etc. For example, a magnetic disk such as a solid state hard disk.

[0074] Optionally, the pickup device can also include a Wi-Fi module, a Bluetooth module, an input unit, a display unit, a sensor, etc., which will not be described here. Those skilled in the art can understand that the structure of the wireless communication device shown in the figure does not constitute a limitation on the wireless communication device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0075] The input unit can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the wireless communication device. The input unit can include a touch screen and other input devices. The touch screen can collect user touch operations on or near it and drive the corresponding connection device according to the pre-set program. For example, the touch operation can include user operation on or near the touch screen using a finger, a stylus or any suitable object or accessory. Optionally, other input devices can include one or more of a physical keyboard, a function key, a trackball, a mouse, a joystick, etc., such as the function key including a volume control button, a power switch button, etc.

[0076] The display unit can be used to display information input by the user or information provided to the user, as well as various menus of the wireless communication device, etc. In one example, the display unit can include a display screen, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch screen can cover the display screen, and when the touch screen detects a touch operation on or near it, it transmits to the processor to determine the type of touch event, and then the processor provides corresponding visual output on the display screen according to the type of touch event. Although in the figure, the touch screen and the display screen are realized as two independent components to realize the input and output functions of the wireless communication device, in some embodiments, the touch screen and the display screen can be integrated to realize the input and output functions of the wireless communication device.

[0077] The signal processing method provided by the embodiment of the present application is described below in combination with the pickup device shown in FIG. 1. It is assumed herein that the pickup device comprises a plurality of microphones, and the plurality of microphones have different orientations. FIG. 3 is a flowchart of a signal processing method provided by an embodiment of the present application. As shown in FIG. 3, the method can comprise the following steps.

[0078] In step 310, a plurality of signals collected by a plurality of microphones are acquired.

[0079] The plurality of microphones collect signals of the same sound source to obtain a plurality of signals. One microphone corresponds to one signal. Since the plurality of microphones have different orientations, the plurality of signals collected by the plurality of microphones are affected by wind noise differently. The plurality of signals include signals affected by wind noise less and signals affected by wind noise more. It is assumed herein that the plurality of signals include a first signal collected by a first microphone and a second signal collected by a second microphone.

[0080] In step 320, in response to the first signal being affected by wind noise less than the second signal, a processed signal obtained from the first signal is output.

[0081] After the plurality of signals collected by the plurality of microphones are acquired, the wind noise affecting each signal is determined respectively, the signal with the least wind noise is selected from the plurality of signals, and a processed signal obtained from the signal with the least wind noise is output.

[0082] For example, the wind noise affecting each two signals in the plurality of signals is compared, and the signal with the least wind noise is selected from the plurality of signals. For example, the plurality of signals include three signals, the wind noise affecting the first signal is compared with the wind noise affecting the second signal, the wind noise affecting the first signal is compared with the wind noise affecting the third signal, the wind noise affecting the second signal is compared with the wind noise affecting the third signal, if the wind noise affecting the first signal is the least, i.e., the wind noise affecting the first signal is less than the wind noise affecting the second signal, and the wind noise affecting the first signal is less than the wind noise affecting the third signal, a processed signal obtained from the first signal is output.

[0083] Since wind noise mostly exists in low frequency, if wind noise exists, the energy of the signal in low frequency is larger. The gain of each frequency point of the signal and the characteristics (such as energy or amplitude) of the signal have a corresponding relationship. In some embodiments, the wind noise affecting the signal is determined according to the energy and gain of the signal in low frequency. Herein, a first frequency band in which wind noise exists is taken as an example for illustration. The energy and gain of the signal in the first frequency band represent the energy and gain of the signal in the frequency band in which wind noise exists. Alternatively, the gain described in the embodiments of the present application can be replaced by amplitude. That is, the wind noise affecting the signal is determined according to the energy and amplitude of the signal in low frequency.

[0084] The following describes the selection of the signal with the minimum wind noise effect by taking the first signal and the second signal as examples. Any one of the multiple signals can refer to the description.

[0085] After obtaining the first signal collected by the first microphone and the second signal collected by the second microphone, the gain of the first signal and the gain of the second signal are obtained, and then the wind noise effect on the first signal is determined according to the energy and the gain of the first signal in the first frequency band; the wind noise effect on the first signal is determined according to the energy and the gain of the second signal in the first frequency band.

[0086] In some embodiments, the signal is input into an artificial intelligence model, and the gain of the signal is output. The artificial intelligence model is an artificial intelligence wind noise reduction model (AIWNR). The artificial intelligence model is trained according to a training set, so that the artificial intelligence model has the function of predicting the gain of the signal affected by wind noise. The training set includes signals affected by wind noise and signals not affected by wind noise. For example, as shown in FIG. 4, an artificial intelligence wind noise reduction model provided by the present application includes multiple gated recurrent units (GRU) and deep neural networks (DNN). The gain of the signal collected by the microphone is identified by the artificial intelligence model, which is more accurate than the traditional signal processing method in identifying the gain, and reduces wind noise at a larger wind speed, thereby improving the identification accuracy of the gain.

[0087] The energy of the first signal in the first frequency band is greater than the first threshold value, and the gain of the first signal in the first frequency band is outside the second gain range, indicating that the possibility of wind noise is relatively large. The gain of the first signal in the first frequency band outside the second gain range, for example, means that the gain of the first signal in the first frequency band is greater than the maximum value of the second gain range or the gain of the first signal in the first frequency band is less than the minimum value of the second gain range. For example, the second gain range is 0.5-1.5. The gain is 1.5, indicating that the signal is not affected by wind noise, the gain is 1, indicating that the signal is less affected by wind noise, and the gain is greater than 1.5 or less than 0.5, indicating that the signal is greatly affected by wind noise.

[0088] The energy of the first signal in the first frequency band is less than the first threshold value, and the gain of the first signal in the first frequency band is within the second gain range, indicating that the possibility of wind noise is relatively small. The signal collected by the microphone may be less affected by wind noise.

[0089] The energy of the first signal in the first frequency band is greater than the first threshold value, and the gain of the first signal in the first frequency band is within the second gain range, indicating that the possibility of wind noise is relatively small, and there are other noise sources around the microphone.

[0090] Optionally, the first signal is determined to be affected by wind noise according to a difference between the energy of the first signal in the first frequency band and the energy of the first signal in the second frequency band. For example, if the difference between the energy of the first signal in the first frequency band and the energy of the first signal in the second frequency band is greater than or equal to the second threshold, it is determined that the first signal is affected by wind noise. Conversely, if the difference between the energy of the first signal in the first frequency band and the energy of the first signal in the second frequency band is less than the second threshold, it is determined that the first signal is less affected by wind noise or the first signal is not affected by wind noise.

[0091] Optionally, the first signal and the second signal are down-sampled, the gain of the first signal is obtained according to the down-sampled first signal, and the gain of the second signal is obtained according to the down-sampled second signal, and then it is determined whether each signal is affected by wind noise. For example, the first signal is a signal with a sampling rate of 96k, and the first signal is down-sampled to 16k by one or more times. Down-sampling can reduce the amount of data, improve the frequency domain resolution, calculate the gain based on the down-sampled signal, and improve the accuracy of the gain.

[0092] Optionally, in the case where the first signal and the second signal are not affected by wind noise, a processed signal obtained from the first signal or the second signal is output.

[0093] In some other embodiments, in the case where the multiple signals are affected by wind noise but the multiple signals are affected by wind noise to different degrees, a processed signal is obtained from at least one of the multiple signals. The processed signal is obtained from at least one of the multiple signals, the energy of the processed signal in the first frequency band is less than the first threshold, and the gain variation of the processed signal in the first frequency band is within the first gain range. The first gain range and the second gain range are different or the same, and the gain range is set according to experience.

[0094] In the first possible implementation, in the case where the multiple signals are affected by wind noise to different degrees, a processed signal is output according to the signal in the multiple signals that is affected by wind noise the least.

[0095] After it is determined that the first signal is affected by wind noise and the second signal is affected by wind noise, the first signal is compared with the second signal in terms of wind noise. If the first signal is affected by wind noise less than the second signal, a processed signal obtained from the first signal is output.

[0096] For example, in the case where the energy of the first signal in the first frequency band is less than the energy of the second signal in the first frequency band, and the gain of the first signal in the first frequency band is greater than the gain of the second signal in the first frequency band, it is determined that the first signal is affected by wind noise less than the second signal.

[0097] The first signal satisfies a first condition. The first condition includes that the energy of the first frequency band signal is less than a first threshold value and the gain of the first frequency band signal is within a second gain range. This indicates that the first signal is less likely to be affected by wind noise. Optionally, the second signal can or can not satisfy the first condition. If the second signal satisfies the first condition, it indicates that the second signal is less likely to be affected by wind noise, but the first signal is less affected by wind noise than the second signal. If the second signal does not satisfy the first condition, it indicates that the second signal is more likely to be affected by wind noise.

[0098] Optionally, if the difference between the energy of the first signal in the first frequency band and the energy of the first signal in a second frequency band is less than a second threshold value, it further indicates that the first signal is less likely to be affected by wind noise. The second frequency band is a frequency band in which a speech signal is located.

[0099] If it is determined that the first signal is less likely to be affected by wind noise, the first signal is processed according to the gain of the first signal, and a processed signal is output. For example, the first signal is processed according to the gain of the first signal in the first frequency band, and a processed signal is output. Here, the first signal is a signal that has not been down-sampled.

[0100] Optionally, if the energy of the first signal in the first frequency band is less than the energy of the second signal in the first frequency band, and the difference between the gain of the first signal in the first frequency band and the gain of the second signal in the first frequency band is greater than a third threshold value, it further indicates that the first signal is less likely to be affected by wind noise, and the first signal is processed according to the gain of the first signal, and a processed signal is output.

[0101] After the processed signal obtained from the first signal is output, if the first signal is more affected by wind noise than the second signal, a processed signal obtained from the second signal is output.

[0102] In some embodiments, if the wind noise of the first signal changes from small to large, and the wind noise of the second signal changes from large to small, the signal collected by the first microphone is converted to the signal collected by the second microphone. For example, if the energy of the first signal in the first frequency band is greater than a first threshold value and the gain in the first frequency band is outside a second gain range, it indicates that the first signal is more likely to be affected by wind noise. If the energy of the second signal in the first frequency band is less than the first threshold value and the gain in the first frequency band is within the second gain range, it indicates that the second signal is less likely to be affected by wind noise, and the first signal is more affected by wind noise than the second signal. The second signal is processed according to the gain of the second signal in the first frequency band, and a processed signal is output.

[0103] Understandably, the first signal and the second signal satisfy the first condition in different time lengths, that is, the processed signal includes the signal collected by the first microphone and the signal collected by the second microphone.

[0104] Optionally, the processed signal is up-sampled, and an up-sampled signal is output.

[0105] Since the output processed signal is a signal less affected by wind noise, it is not affected by the wind noise changes of other signals. When the wind direction changes, the signal is processed in real time according to the changes of the gain and the low-frequency energy (such as the energy of the first frequency band) to ensure fast response and continuous stability of the processed signal, effectively reducing the wind noise and improving the listening experience.

[0106] The embodiment of the present application provides a test method for an output signal of a sound pickup device. A first signal and a second signal are generated, and the voice content of the first signal and the voice content of the second signal are different, so as to identify whether the output processed signal is obtained from the first signal or the second signal according to different language content. The first signal and the second signal are respectively added with noise, so that the first signal is less affected by wind noise than the second signal. The first signal is input to the first microphone, and the second signal is input to the second microphone, so that the output processed signal includes the voice content of the first signal. Further, the wind noise of the second signal is reduced, and the wind noise of the first signal is increased, so that the first signal is more affected by wind noise than the second signal. The first signal is input to the first microphone, and the second signal is input to the second microphone, so that the output processed signal includes the voice content of the second signal. Thus, by applying different wind noises to the two signals, the two signals contain different voice content, and it is identified that the output processed signal is obtained from the signal least affected by wind noise. The sound pickup device is tested by using the test method, and the operability and the evidence of the test method are improved. In addition, the energy and the gain change of the processed signal on the first frequency band are detected, the energy of the processed signal on the first frequency band is less than a first threshold, and the gain change of the processed signal on the first frequency band is within a first gain range.

[0107] In a second possible implementation, when the differences of wind noise effects on the multiple signals are similar, a processed signal is output according to at least two signals of the multiple signals. That is, the signals less affected by wind noise in the multiple signals are combined to form a new signal as the processed signal.

[0108] When the first signal and the second signal satisfy the second condition, the processed signal is obtained from the first signal and the second signal, and the second condition includes that the energy on the first frequency band is greater than the first threshold and the gain on the first frequency band is outside the second gain range.

[0109] The first signal has an energy greater than the first threshold on the first frequency band and a gain out of the second gain range on the first frequency band, and the second signal has an energy greater than the first threshold on the first frequency band and a gain out of the second gain range on the first frequency band, which indicates that the first signal and the second signal are more likely to be affected by wind noise, and the signals of the frequency bands of the first signal and the second signal less affected by wind noise are spliced to output a processed signal. The first signal and the second signal are spliced according to the energy and the gain of the first signal and the energy and the gain of the second signal. The energy of the processed signal on the first frequency band is obtained from the energy of the first signal on the first frequency band and the energy of the second signal on the first frequency band, and the gain of the processed signal on the first frequency band is obtained from the gain of the first signal on the first frequency band and the gain of the second signal on the first frequency band.

[0110] In some embodiments, the weight of the first signal and the weight of the second signal are determined according to the energy and the gain of the first signal on the first frequency band and the energy and the gain of the second signal on the first frequency band; and the signals of the frequency bands of the first signal and the second signal are spliced according to the weight of the first signal and the weight of the second signal to output a processed signal. The weight is used to indicate the degree to which the signal is less affected by wind noise. For example, the greater the weight, the less the degree to which the signal is less affected by wind noise. Alternatively, the weight is used to indicate the importance of the output signal.

[0111] For example, the first frequency band is divided into three frequency bands, and the weight of the first signal and the weight of the second signal are calculated according to the ratio of the energy and the ratio of the gain in each frequency band.

[0112] In the frequency band 1, the ratio 1 is obtained according to the energy of the first signal and the energy of the second signal, the ratio 2 is obtained according to the gain of the first signal and the gain of the second signal, and the weight 1 of the first signal in the frequency band 1 and the weight 1' of the second signal in the frequency band 1 are calculated according to the ratio 1 and the ratio 2.

[0113] In the frequency band 2, the ratio 1 is obtained according to the energy of the first signal and the energy of the second signal, the ratio 2 is obtained according to the gain of the first signal and the gain of the second signal, and the weight 1 of the first signal in the frequency band 2 and the weight 1' of the second signal in the frequency band 2 are calculated according to the ratio 1 and the ratio 2.

[0114] In the frequency band 3, the ratio 1 is obtained according to the energy of the first signal and the energy of the second signal, the ratio 2 is obtained according to the gain of the first signal and the gain of the second signal, and the weight 1 of the first signal in the frequency band 3 and the weight 1' of the second signal in the frequency band 3 are calculated according to the ratio 1 and the ratio 2.

[0115] For each frequency band, the energy and gain of the multi-path signals are respectively weighted and averaged according to the weights, to obtain the signal energy and gain of the frequency band. For example, the energy of the first path signal in frequency band 1 multiplied by weight 1 and the energy of the second path signal in frequency band 1 multiplied by weight 1' obtain the new signal energy in frequency band 1. The gain of the first path signal in frequency band 1 multiplied by weight 1 and the gain of the second path signal in frequency band 1 multiplied by weight 1' obtain the new gain of the signal in frequency band 1.

[0116] Optionally, the signal energy and gain of the plurality of frequency bands are smoothed to obtain a new signal and a new gain. The new signal is processed according to the new gain to obtain a processed signal.

[0117] Optionally, after obtaining the processed signal, the processed signal is up-sampled to meet the requirements of the output signal. The Fourier transform is used to convert the signal from the time domain to the frequency domain signal. The inverse Fourier transform is used to convert the signal from the frequency domain to the time domain signal.

[0118] In the case of wind direction change, according to the change of the gain and energy of the signal in the first frequency band, the above-mentioned method is processed in real time, which ensures fast response and also ensures the continuity and stability of the signal. Effectively improve the effect of wind noise reduction, reduce the influence of wind noise of the output signal, and improve the listening experience. Optionally, the above-mentioned signal processing method is periodically (e.g., milliseconds) executed to ensure the quality of the output signal and reduce the influence of wind noise on the signal.

[0119] The embodiment of the present application provides another test method for output signal of a pickup device. Assuming that a first path signal and a second path signal are generated, the speech content of the first path signal and the speech content of the second path signal are different, so as to identify whether the processed output signal is obtained from the first path signal or the second path signal according to different language content. The first path signal and the second path signal are respectively added with noise, so that the first path signal and the second path signal both do not meet the first condition, the wind noise influence on the first path signal is similar to the wind noise influence on the second path signal, the processed output signal includes the speech content of the first path signal and the speech content of the second path signal, indicating that the signals with less wind noise influence in the multi-path signals are combined into a new signal as the processed output signal. Further, the wind noise of the second path signal is reduced and the wind noise of the first path signal is increased, so that the wind noise influence on the first path signal is greater than the wind noise influence on the second path signal, and the processed output signal includes the speech content of the second path signal. Further, the wind noise of the second path signal is increased and the wind noise of the first path signal is reduced, so that the wind noise influence on the first path signal is less than the wind noise influence on the second path signal, and the processed output signal includes the speech content of the first path signal.

[0120] The various embodiments of the present application can be implemented independently or in combination, without limitation. If not specifically stated and there is no logical conflict, the terms and / or descriptions provided in different embodiments of the present application are consistent and can be mutually referred to. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0121] It can be understood that, in the embodiments of the present application, the execution subject can execute part or all of the steps in the embodiments of the present application, and these steps or operations are only examples, and the embodiments of the present application can also execute other operations or variations of various operations. In addition, each step can be executed in a different order as presented in the embodiments of the present application, and it is possible that not all operations in the embodiments of the present application are executed.

[0122] The above mainly introduces the scheme provided by the embodiments of the present application from the perspective of interaction between devices. It can be understood that, in order to realize the above functions, each device comprises a hardware structure and / or a software module for executing each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed in the present application, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0123] The embodiments of the present application can divide the functional modules of each device according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be realized in the form of hardware or software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division manner.

[0124] In the case of dividing each functional module according to each function, FIG. 5 shows a signal processing apparatus 500, which can execute the actions performed by the pickup device in the method shown in FIG. 3. All related contents of each step involved in the above method embodiment can be referred to the function description of the corresponding functional module, and the technical effects that can be obtained can be referred to the above method embodiment, which will not be described here.

[0125] The signal processing apparatus 500 can include an interface module 510, a processing module 520 and a storage module 530.

[0126] The processing module 520 is configured to acquire a plurality of signals collected by a plurality of microphones under wind noise, wherein one microphone corresponds to one signal, and the plurality of signals include a first signal collected by a first microphone and a second signal collected by a second microphone; and the processing module 520 is configured to output a processed signal obtained from the first signal through the interface module 510 in response to the wind noise affecting the first signal being less than the wind noise affecting the second signal. For example, the interface module 510 is configured to perform step 320 in FIG. 3. For example, the processing module 520 is configured to perform step 310 in FIG. 3.

[0127] The processing module 520 is configured to determine whether wind noise exists in each signal according to energy of the collected signal on a low frequency and gain, and output the processed signal according to the energy of the collected signal and the gain.

[0128] The storage module 530 is configured to store information related to a signal processing process, for example, gain, signal energy, amplitude, and the like.

[0129] It should be understood that the signal processing apparatus 500 in the embodiments of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. When the signal processing method shown in FIG. 3 is implemented by software, the signal processing apparatus 500 and each module thereof can also be a software module.

[0130] More detailed descriptions of the interface module, the processing module, and the storage module can be directly obtained by referring to the related descriptions in the method embodiments shown in FIG. 3, and thus will not be described herein.

[0131] The embodiments of the present application further provide a computer program product, which can implement the functions of any of the method embodiments when executed by a computer.

[0132] The embodiments of the present application further provide a computer program, which can implement the functions of any of the method embodiments when executed by a computer.

[0133] The embodiments of the present application further provide a computer readable storage medium. All or part of the processes of the above method embodiments can be instructed by a computer program to relevant hardware to complete, the program can be stored in the above computer readable storage medium, and the program can include the processes of the above method embodiments when executed. The computer readable storage medium can be an internal storage unit of the terminal (including a data sending terminal and / or a data receiving terminal) of any of the above embodiments, for example, a hard disk or a memory of the terminal. The computer readable storage medium can also be an external storage device of the terminal, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the terminal. The computer readable storage medium is used to store the above computer program and other programs and data required by the terminal. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0134] It should be noted that the terms "first" and "second" and the like in the specification of the present application, claims and drawings are used to distinguish different objects, and are not used to describe a specific order. "First", "second" are only for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. In the description of the present embodiment, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0135] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units that are not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0136] It should be understood that, in the application, "at least one" means one or more. "Multiple" means two or more. "At least two" means two or three and more. "And / or" is used to describe the association between the associated objects, which means that there can be three relationships. For example, "A and / or B" can mean that there are three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple. "When" and "if" both mean that under certain objective circumstances, the corresponding processing will be done, not limited to time, and does not require a judgment action when implemented, nor does it mean that there are other limitations.

[0137] In the embodiments of the present application, the words such as "exemplary" or "for example" are used to mean serving as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words such as "exemplary" or "for example" are intended to present the relevant concept in a specific manner for ease of understanding.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example for illustration. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0139] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not implemented. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components displayed as units may be one physical unit or multiple physical units, that is, may be located in one place, or also may be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0141] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0142] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical scheme of the embodiment of the present application can be embodied in the form of a software product in essence or all or part of the technical scheme. The software product is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various storage program codes.

Claims

1. A signal processing method, characterized by, The method is applied to a sound pickup device, the sound pickup device comprises a plurality of microphones with different orientations, and the method comprises the following steps: obtaining a plurality of signals collected by the plurality of microphones under wind noise, wherein one microphone corresponds to one signal, the plurality of signals comprise a first signal collected by a first microphone and a second signal collected by a second microphone; in response to the first signal being less affected by wind noise than the second signal, outputting a processed signal obtained from the first signal.

2. The method of claim 1, wherein, The first signal is less affected by wind noise than the second signal, which comprises: the energy of the first signal in a first frequency band is less than the energy of the second signal in the first frequency band, and the gain of the first signal in the first frequency band is greater than the gain of the second signal in the first frequency band, the first frequency band being the frequency band where wind noise is located.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: in response to the first signal being more affected by wind noise than the second signal, outputting a processed signal obtained from the second signal.

4. The method according to any one of claims 1-3, characterized in that, The energy of the processed signal in the first frequency band is less than a first threshold, and the gain variation of the processed signal in the first frequency band is within a gain range.

5. The method according to any one of claims 1-4, characterized in that, The plurality of microphones comprise a first microphone and a second microphone with opposite orientations.

6. The method according to any one of claims 1-4, characterized in that, The plurality of microphones comprise a first microphone and a second microphone with perpendicular orientations.

7. A signal processing device, characterized by The method comprises a processing module and an interface module. The processing module is configured to obtain a plurality of signals collected by the plurality of microphones under wind noise, wherein one microphone corresponds to one signal, the plurality of signals comprise a first signal collected by a first microphone and a second signal collected by a second microphone. The processing module is configured to output a processed signal obtained from the first signal through the interface module in response to the first signal being less affected by wind noise than the second signal.

8. A pickup device, characterized in that The sound pickup device comprises a plurality of microphones with different orientations and a processor, and the processor is configured to execute the method according to any one of claims 1-6.

9. The pickup device according to claim 8, characterized in that The plurality of microphones comprise a first microphone and a second microphone with opposite orientations.

10. A computer program product, characterised in that, The computer program product comprises computer instructions; when part or all of the computer instructions are run on a computer, the computer is enabled to implement the method according to any one of claims 1-6.

Images