Far-field pickup method, device, equipment and medium
Through microphone array signal processing technology, the target speaker's orientation is automatically estimated, the main beam is focused and interference is suppressed, solving the problem of poor far-field sound pickup effect and improving the long-distance sound pickup quality and speech recognition accuracy.
Patent Information
- Application Number
- CN202310722615.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-16
- Publication Date
- 2024-12-17
AI Technical Summary
When the microphone is more than 2 meters away from the target sound source, the signal-to-noise ratio and direct reverberation ratio are low, resulting in poor far-field sound pickup effect.
By obtaining the original audio, performing voice endpoint detection and wind noise detection, determining whether audio processing is needed, positioning the sound source position and generating audio beams. Based on microphone array signal processing technology, the speaker's orientation is automatically estimated, the main beam focuses and suppresses interference. The noise reduction module eliminates noise.
It improves the quality of long-distance sound pickup and speech recognition accuracy, is suitable for different noise scenarios, and improves speech clarity and recognition accuracy.
Smart Images
Figure CN119152873A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of far-field sound pickup, and in particular to a far-field sound pickup method, a far-field sound pickup device, an electronic device, and a computer-readable storage medium. Background Art
[0002] At present, the location of the target sound source in many application scenarios in life is more than 2 meters away from the microphone, such as common head-mounted devices or wearable devices with sound collection systems, smart voice speakers, etc. When the target sound source, such as a person speaking, stands far away from the microphone, a more prominent problem is that the signal-to-noise ratio and direct reverberation ratio of the target sound are very low, and the far-field sound pickup effect is poor. Summary of the invention
[0003] The main purpose of the present application is to provide a far-field sound pickup method, a far-field sound pickup device, an electronic device and a computer-readable storage medium, aiming to solve the technical problem that it is difficult to achieve far-field sound pickup efficiently and accurately.
[0004] To achieve the above object, the present application provides a far-field sound pickup method, the method comprising:
[0005] Acquire original audio, and determine whether audio processing needs to be performed on the original audio;
[0006] After determining that the original audio needs to be processed, locating the sound source position of the original audio, and generating an audio beam based on the sound source position and the position of a microphone for collecting the original audio;
[0007] After determining that the audio beam has high noise, noise suppression is performed on the audio beam to output target audio.
[0008] Exemplarily, the step of determining whether audio processing needs to be performed on the original audio includes:
[0009] Performing voice endpoint detection and wind noise detection on the original audio;
[0010] Based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection, it is determined whether audio processing needs to be performed on the original audio.
[0011] Exemplarily, the step of determining whether it is necessary to perform audio processing on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection includes:
[0012] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise does not exist, it is determined that audio processing needs to be performed on the original audio.
[0013] Exemplarily, the step of determining whether it is necessary to perform audio processing on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection includes:
[0014] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise exists, wind noise reduction processing is performed on the original audio to output the target audio.
[0015] Exemplarily, the step of determining whether it is necessary to perform audio processing on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection includes:
[0016] If the first detection result is that there is no voice endpoint, it is determined that there is no need to perform audio processing on the original audio.
[0017] Exemplarily, the step of locating the sound source position of the original audio includes:
[0018] The sound wave incident angle information of the original audio is determined, and the sound wave incident angle information is determined as the sound source position of the original audio.
[0019] Exemplarily, after the step of generating an audio beam based on the sound source position and the microphone position for collecting the original audio, the method further comprises:
[0020] After determining that the audio beam does not have high noise, the original audio is output as the target audio.
[0021] The present application also provides a far-field sound pickup device, the device comprising:
[0022] A determination module, used for acquiring original audio and determining whether audio processing needs to be performed on the original audio;
[0023] A generating module, configured to locate the sound source position of the original audio after determining that the original audio needs to be processed, and generate an audio beam based on the sound source position and the position of a microphone for collecting the original audio;
[0024] The output module is used to suppress the noise of the audio beam and output the target audio after determining that the audio beam has high noise.
[0025] The present application also provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the far-field sound pickup method described above when executed by the processor.
[0026] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the far-field sound pickup method described above are implemented.
[0027] The embodiments of the present application propose a far-field sound pickup method, a far-field sound pickup device, an electronic device, and a computer-readable storage medium, which obtain original audio and determine whether audio processing is required for the original audio; after determining that audio processing is required for the original audio, locate the sound source position of the original audio, and generate an audio beam based on the sound source position and the position of a microphone for collecting the original audio; after determining that high noise exists in the audio beam, suppress noise on the audio beam and output target audio.
[0028] This application focuses on improving the quality of long-distance sound pickup and improving speech recognition or translation effects, and proposes a technical solution for a far-field sound pickup method. First, after determining that the acquired original audio needs to be processed, the sound source position of the original audio is located, and an audio beam is generated based on the sound source position and the position of the microphone that collects the original audio; then, after determining that the audio beam has high noise, the audio beam is noise suppressed to output the target audio.
[0029] This solution uses microphone array signal processing technology to automatically estimate the target speaker's position, focus the main beam on the speaker's direction and suppress other interference, and further eliminate noise through the noise reduction module to improve speech clarity. In this way, wind noise, high noise, quiet or low noise scenes can be identified for different audio processing, and the accuracy of speech recognition in different scenes can be improved more meticulously. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a structural diagram of an operating device of a hardware operating environment involved in an embodiment of the present application;
[0031] Figure 2 This is a flow chart of an embodiment of a far-field sound pickup method according to an embodiment of the present application;
[0032] Figure 3 This is a flow chart of another embodiment of the far-field sound pickup method involved in the embodiment of the present application;
[0033] Figure 4 This is a schematic diagram of an algorithm block diagram of an embodiment of a far-field sound pickup method according to an embodiment of the present application;
[0034] Figure 5 It is a schematic diagram of a far-field sound pickup device involved in an embodiment of the present application.
[0035] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0036] At present, the location of the target sound source in many application scenarios in life is more than 2 meters away from the microphone, such as common head-mounted devices or wearable devices with sound collection systems, smart voice speakers, etc. When the target sound source, such as a person speaking, stands far away from the microphone, a more prominent problem is that the signal-to-noise ratio and direct reverberation ratio of the target sound are very low. When the distance between the target sound source and the microphone doubles, the sound pressure level of the target sound at the microphone will be 6dB lower.
[0037] Therefore, there is currently a technical problem that it is difficult to achieve far-field sound pickup efficiently and accurately.
[0038] This application focuses on improving the quality of long-distance sound pickup and improving speech recognition or translation effects, and proposes a technical solution for a far-field sound pickup method. First, after determining that the acquired original audio needs to be processed, the sound source position of the original audio is located, and an audio beam is generated based on the sound source position and the position of the microphone that collects the original audio; then, after determining that the audio beam has high noise, the audio beam is noise suppressed to output the target audio.
[0039] This solution uses microphone array signal processing technology to automatically estimate the target speaker's position, focus the main beam on the speaker's direction and suppress other interference, and further eliminate noise through the noise reduction module to improve speech clarity. In this way, wind noise, high noise, quiet or low noise scenes can be identified for different audio processing methods, and the accuracy of speech recognition in different scenes can be improved more meticulously.
[0040] This application proposes a far-field sound pickup solution based on a microphone array, which integrates noise scene recognition and multi-algorithm module comprehensive processing. Experiments show that the algorithm output can obtain better voice signal quality, thereby obtaining a higher voice recognition accuracy.
[0041] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0042] Reference Figure 1 , Figure 1 This is a schematic diagram of the operating equipment structure of the hardware operating environment involved in the embodiment of the present application.
[0043] like Figure 1As shown, the operating device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0044] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the operating device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0045] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and a computer program.
[0046] exist Figure 1 In the running device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the running device of the present application can be set in the running device, and the running device calls the computer program stored in the memory 1005 through the processor 1001, and performs the following operations:
[0047] Acquire original audio, and determine whether audio processing needs to be performed on the original audio;
[0048] After determining that the original audio needs to be processed, locating the sound source position of the original audio, and generating an audio beam based on the sound source position and the position of a microphone for collecting the original audio;
[0049] After determining that the audio beam has high noise, noise suppression is performed on the audio beam to output target audio.
[0050] In one embodiment, the processor 1001 may call a computer program stored in the memory 1005 and further perform the following operations:
[0051] The step of determining whether audio processing needs to be performed on the original audio comprises:
[0052] Performing voice endpoint detection and wind noise detection on the original audio;
[0053] Based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection, it is determined whether audio processing needs to be performed on the original audio.
[0054] In one embodiment, the processor 1001 may call a computer program stored in the memory 1005 and further perform the following operations:
[0055] The step of determining whether audio processing needs to be performed on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection comprises:
[0056] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise does not exist, it is determined that audio processing needs to be performed on the original audio.
[0057] In one embodiment, the processor 1001 may call a computer program stored in the memory 1005 and further perform the following operations:
[0058] The step of determining whether audio processing needs to be performed on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection comprises:
[0059] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise exists, wind noise reduction processing is performed on the original audio to output the target audio.
[0060] In one embodiment, the processor 1001 may call a computer program stored in the memory 1005 and further perform the following operations:
[0061] The step of determining whether audio processing needs to be performed on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection comprises:
[0062] If the first detection result is that there is no voice endpoint, it is determined that there is no need to perform audio processing on the original audio.
[0063] In one embodiment, the processor 1001 may call a computer program stored in the memory 1005 and further perform the following operations:
[0064] The step of locating the sound source position of the original audio comprises:
[0065] The sound wave incident angle information of the original audio is determined, and the sound wave incident angle information is determined as the sound source position of the original audio.
[0066] In one embodiment, the processor 1001 may call a computer program stored in the memory 1005 and further perform the following operations:
[0067] After the step of generating an audio beam based on the sound source position and the microphone position for collecting the original audio, the method further comprises:
[0068] After determining that the audio beam does not have high noise, the original audio is output as the target audio.
[0069] The present application embodiment provides a far-field sound pickup method, referring to Figure 2 In one embodiment of the far-field sound pickup method, the method comprises:
[0070] Step S10, obtaining original audio, and determining whether audio processing needs to be performed on the original audio.
[0071] The original audio may be far-field audio collected at a distance of more than 2 meters from the microphone. In this embodiment, the method for collecting and acquiring the original audio is not limited. The audio processing performed on the original audio includes but is not limited to noise reduction, filtering, transformation, encoding and decoding, and enhancement. In one embodiment, whether the original audio needs to be processed is determined based on the detection results of voice endpoint detection and wind noise detection on the original audio.
[0072] Step S20: after determining that the original audio needs to be processed, locate the sound source position of the original audio, and generate an audio beam based on the sound source position and the position of a microphone for collecting the original audio.
[0073] After determining that the original audio needs to be processed, the sound source position of the original audio is located, and an audio beam is generated according to the sound source position and the position of a microphone for collecting the original audio.
[0074] In one embodiment, in quiet or noisy scenarios, the fusion of array multi-channel MIC data into one channel can enhance the signal itself, suppress noise, and improve the signal-to-noise ratio. Among them, the MIC coordinate information pre-cured in the remote sound pickup program and the angle of DOA (Direction Of Arrival) estimation are required to generate the audio beam.
[0075] Step S30: after determining that the audio beam has high noise, perform noise suppression on the audio beam and output target audio.
[0076] After the audio beam is generated, it is determined whether the audio beam has high decibel noise (high noise) with a noise volume greater than a preset threshold. After determining that the audio beam has high noise, a high noise flag is output to suppress noise in the audio beam, thereby outputting the target audio.
[0077] In this embodiment, the electronic device that implements the far-field sound pickup method can be a head-mounted device such as VR, AR, etc., or a headset, a wearable device, a smart speaker, etc.
[0078] In one embodiment, noise / wind noise suppression is used to eliminate noise as much as possible while avoiding distortion of the speech itself. Experiments have shown that speech distortion has a greater impact on speech recognition, but usually noise reduction algorithms will inevitably have distortion because there is a certain deviation between the estimated noise and the actual noise. Therefore, in this embodiment, on the one hand, through high noise judgment processing, the noise suppression module is entered only when the noise is large; on the other hand, a method combining AI noise reduction and traditional noise reduction is used to suppress noise as much as possible while avoiding speech distortion.
[0079] This application focuses on improving the quality of long-distance sound pickup and improving speech recognition or translation effects, and proposes a technical solution for a far-field sound pickup method. First, after determining that the acquired original audio needs to be processed, the sound source position of the original audio is located, and an audio beam is generated based on the sound source position and the position of the microphone that collects the original audio; then, after determining that the audio beam has high noise, the audio beam is noise suppressed to output the target audio.
[0080] This solution uses microphone array signal processing technology to automatically estimate the target speaker's position, focus the main beam on the speaker's direction and suppress other interference, and further eliminate noise through the noise reduction module to improve speech clarity. In this way, it can identify wind noise, high noise, quiet or low noise scenes for different sound effects processing, and more meticulously improve the accuracy of speech recognition in different scenes.
[0081] The present application embodiment provides a far-field sound pickup method. In another embodiment of the far-field sound pickup method, referring to Figure 3 The step of determining whether audio processing needs to be performed on the original audio comprises:
[0082] Step S101, performing voice endpoint detection and wind noise detection on the original audio;
[0083] Step S102: Determine whether audio processing is required for the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection.
[0084] Voice endpoint detection is used to detect whether someone is speaking. If no one is speaking, no recognition or translation will be sent. This process can avoid incorrect recognition in noisy scenes, especially in noise environments like cafes. Among them, voice endpoint detection can use VOA (Voice Activity Detection) technology to detect whether there is sound. In addition, wind noise detection is used to detect whether it is a wind noise scene. Therefore, the first detection result of the voice endpoint detection and the second detection result of the wind noise detection are used to determine whether the original audio needs to be processed. In this embodiment, the method of performing voice endpoint detection and wind noise detection on the original audio is not limited.
[0085] Exemplarily, the step of determining whether it is necessary to perform audio processing on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection includes:
[0086] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise does not exist, it is determined that audio processing needs to be performed on the original audio.
[0087] If the first detection result of performing speech endpoint detection on the original audio is that a speech endpoint exists, and the second detection result of performing wind noise detection on the original audio is that no wind noise exists, it can be confirmed that audio processing is required for the original audio.
[0088] Exemplarily, the step of determining whether it is necessary to perform audio processing on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection includes:
[0089] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise exists, wind noise reduction processing is performed on the original audio to output the target audio.
[0090] If the first detection result of voice endpoint detection on the original audio is the presence of voice endpoints, and the second detection result of wind noise detection on the original audio is the presence of wind noise, wind noise reduction processing is directly performed on the original audio without performing other audio processing, thereby outputting the target audio.
[0091] Exemplarily, the step of determining whether it is necessary to perform audio processing on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection includes:
[0092] If the first detection result is that there is no voice endpoint, it is determined that there is no need to perform audio processing on the original audio.
[0093] If the first detection result of voice endpoint detection on the original audio is that there is no voice endpoint, it can be confirmed that there is no need to perform audio processing on the original audio, thereby avoiding redundant processing and misrecognition.
[0094] In this embodiment, based on the first detection result of voice endpoint detection on the original audio and the second detection result of wind noise detection on the original audio, it is determined whether the original audio needs to be processed. Thus, by identifying wind noise, high noise, quiet or low noise scenes for different process processing, the accuracy of speech recognition in different scenes is improved more meticulously.
[0095] The embodiment of the present application provides a far-field sound pickup method. In another embodiment of the far-field sound pickup method, the step of locating the sound source position of the original audio includes:
[0096] The sound wave incident angle information of the original audio is determined, and the sound wave incident angle information is determined as the sound source position of the original audio.
[0097] When locating the sound source of the original audio, the azimuth information of the target speaker is estimated, mainly the incident angle information of the sound wave. This angle information will be used as the input of the audio beamforming to generate the audio beam.
[0098] An embodiment of the present application provides a far-field sound pickup method. In another embodiment of the far-field sound pickup method, after the step of generating an audio beam based on the sound source position and the microphone position for collecting the original audio, the method includes:
[0099] After determining that the audio beam does not have high noise, the original audio is output as the target audio.
[0100] In this embodiment, the high noise flag is output by detecting the noise level. When it is quiet or the noise is not loud, the algorithm process skips the noise reduction module, because the noise reduction module usually has more or less speech loss. Experiments show that when it is quiet or the noise is not loud, the recognition rate will be better without noise reduction. Therefore, after determining that there is no high noise in the audio beam, the original audio can be directly output as the target audio.
[0101] In an application scenario of a far-field sound pickup method of the present application, it is proposed as follows Figure 4 The algorithm block diagram shown in the figure:
[0102] 1. Voice endpoint detection module
[0103] It is used to detect whether someone is speaking. If no one is speaking, the speech will not be sent for recognition or translation. This can avoid false recognition in noisy scenes, especially in pub noise like in a coffee shop. Through the VAD module, in the non-speaking segment, the speech will not be sent for recognition, thus avoiding unnecessary false recognition.
[0104] 2. Wind noise detection module
[0105] This module is used to detect whether it is a wind noise scene. If it is a wind noise scene, the algorithm processing flow jumps to the wind noise suppression module without beamforming (the other output arrow of the wind noise detection module in the figure goes to beamforming, which is used to notify the beamforming module that beamforming is not required). The reason for not going through beamforming is that in wind noise scenes, the effect is usually worse after beam fusion. In actual operation, you can select a channel with the least wind noise content from the multi-channel microphone array and send it to the wind noise suppression module.
[0106] 3. High noise detection module
[0107] This module is used to detect the noise level and output a high noise flag. When it is quiet or the noise is not loud, the algorithm will skip the noise reduction module because the noise reduction module usually has more or less speech loss. Experiments show that when it is quiet or the noise is not loud, the recognition rate will be better without noise reduction.
[0108] 4. Sound source localization module
[0109] This module is used to estimate the position information of the target speaker, mainly the incident angle information of the sound wave. This angle information will be sent to the beamforming module as input.
[0110] 5. Beamforming module
[0111] In quiet or noisy scenes, this module fuses the multi-channel MIC data of the array into one channel, which can enhance the signal itself, suppress noise, and improve the signal-to-noise ratio. This module requires the MIC coordinate information and the angle estimated by the DOA module (the sound source localization module in the figure).
[0112] 6. Noise / wind noise suppression module
[0113] Noise / wind noise suppression, eliminate noise as much as possible while avoiding distortion of the voice itself. Through experiments, it is found that voice distortion has a greater impact on voice recognition, but usually noise reduction algorithms will inevitably have distortion because there is a certain deviation between the estimated noise and the actual noise. Therefore, on the one hand, through the high noise judgment module, the noise suppression module is entered only when the noise is large; on the other hand, a combination of AI noise reduction and traditional noise reduction is used to suppress noise as much as possible while avoiding voice distortion.
[0114] Therefore, in this application scenario, an effective far-field sound pickup technology method is proposed, which is based on microphone array signal processing technology to identify wind noise, high noise, quiet or low noise scenes, perform different process processing, and more meticulously improve the accuracy of speech recognition in different scenarios.
[0115] Reference Figure 5 In addition, the embodiment of the present application further provides a far-field sound pickup device, the far-field sound pickup device comprising:
[0116] A determination module M1 is used to obtain original audio and determine whether audio processing needs to be performed on the original audio;
[0117] A generating module M2 is used to locate the sound source position of the original audio after determining that the original audio needs to be processed, and generate an audio beam based on the sound source position and the position of the microphone for collecting the original audio;
[0118] The output module M3 is used to suppress the noise of the audio beam and output the target audio after determining that the audio beam has high noise.
[0119] Exemplarily, the determining module is further used for:
[0120] Performing voice endpoint detection and wind noise detection on the original audio;
[0121] Based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection, it is determined whether audio processing needs to be performed on the original audio.
[0122] Exemplarily, the determining module is further used for:
[0123] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise does not exist, it is determined that audio processing needs to be performed on the original audio.
[0124] Exemplarily, the determining module is further used for:
[0125] If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise exists, wind noise reduction processing is performed on the original audio to output the target audio.
[0126] Exemplarily, the determining module is further used for:
[0127] If the first detection result is that there is no voice endpoint, it is determined that there is no need to perform audio processing on the original audio.
[0128] Exemplarily, the generating module is further used for:
[0129] The sound wave incident angle information of the original audio is determined, and the sound wave incident angle information is determined as the sound source position of the original audio.
[0130] Exemplarily, the output module is further used for:
[0131] After the step of generating an audio beam based on the sound source position and the microphone position for collecting the original audio:
[0132] After determining that the audio beam does not have high noise, the original audio is output as the target audio.
[0133] The far-field sound pickup device provided by the present application adopts the far-field sound pickup method in the above embodiment to solve the technical problem that it is difficult to achieve far-field sound pickup efficiently and accurately. Compared with conventional technology, the beneficial effects of the far-field sound pickup device provided by the embodiment of the present application are the same as the beneficial effects of the far-field sound pickup method provided by the above embodiment, and other technical features in the far-field sound pickup device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0134] In addition, an embodiment of the present application also provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the far-field sound pickup method as described above when executed by the processor.
[0135] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the far-field sound pickup method as described above are implemented.
[0136] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0137] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to conventional technology can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, including a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0138] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A far-field sound pickup method, characterized in that: The method comprises: Acquire original audio, and determine whether audio processing needs to be performed on the original audio; After determining that the original audio needs to be processed, locating the sound source position of the original audio, and generating an audio beam based on the sound source position and the position of a microphone for collecting the original audio; After determining that the audio beam has high noise, noise suppression is performed on the audio beam to output target audio.
2. The far-field sound pickup method according to claim 1, characterized in that: The step of determining whether audio processing needs to be performed on the original audio comprises: Performing voice endpoint detection and wind noise detection on the original audio; Based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection, it is determined whether audio processing needs to be performed on the original audio.
3. The far-field sound pickup method according to claim 2, characterized in that: The step of determining whether audio processing needs to be performed on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection comprises: If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise does not exist, it is determined that audio processing needs to be performed on the original audio.
4. The far-field sound pickup method according to claim 2, characterized in that: The step of determining whether audio processing needs to be performed on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection comprises: If the first detection result is that a voice endpoint exists, and the second detection result is that wind noise exists, wind noise reduction processing is performed on the original audio to output the target audio.
5. The far-field sound pickup method according to claim 2, characterized in that: The step of determining whether audio processing needs to be performed on the original audio based on the first detection result of the voice endpoint detection and the second detection result of the wind noise detection comprises: If the first detection result is that there is no voice endpoint, it is determined that there is no need to perform audio processing on the original audio.
6. The far-field sound pickup method according to claim 1, characterized in that: The step of locating the sound source position of the original audio comprises: The sound wave incident angle information of the original audio is determined, and the sound wave incident angle information is determined as the sound source position of the original audio.
7. The far-field sound pickup method according to claim 1, characterized in that: After the step of generating an audio beam based on the sound source position and the microphone position for collecting the original audio, the method further comprises: After determining that the audio beam does not have high noise, the original audio is output as the target audio.
8. A far-field sound pickup device, characterized in that: The device comprises: A determination module, used to obtain original audio and determine whether audio processing needs to be performed on the original audio; A generating module, configured to locate the sound source position of the original audio after determining that the original audio needs to be processed, and generate an audio beam based on the sound source position and the position of a microphone for collecting the original audio; The output module is used to suppress the noise of the audio beam and output the target audio after determining that the audio beam has high noise.
9. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the far-field sound pickup method according to any one of claims 1 to 7 when executed by the processor.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the far-field sound pickup method according to any one of claims 1 to 7 are implemented.