Human voice positioning method, electronic device and storage medium

Abstract

The application discloses a human voice positioning method and an electronic device and a storage medium, wherein the method comprises the following steps: converting an acquired audio signal into a spectrogram signal, and separating the spectrogram signal; filtering out non-human voice signals in each direction of the separated spectrogram signal respectively through a neural network, and judging the existence probability of human voice signals in each direction of the spectrogram signal; determining a direction with the maximum human voice signal existence probability and outputting a mask signal of the direction with the maximum human voice signal existence probability; and estimating the azimuth information of the human voice signal based on at least the direction with the maximum human voice signal existence probability and the corresponding mask signal. According to the application, the mask signal of the direction with the maximum probability is output by using the neural network, the azimuth information of the human voice signal is estimated according to the mask signal of the direction with the maximum probability, and when external noise is large, the sound source positioning result can also accurately point to the target human voice source, so that the accuracy of target human voice positioning is further improved.

Description

Technical Field

[0001] This invention belongs to the field of sound source localization technology, and particularly relates to a human voice localization method, electronic device, and storage medium. Background Technology

[0002] In existing technologies, the main methods for human voice localization include beam selection algorithms, inter-channel phase difference (IPD) algorithms, generalized cross-correlation (GCC) algorithms, and multiple signal classification (MUSIC) algorithms, among other classic sound source localization estimation methods. A commonly used classic sound source localization method, represented by beam selection algorithms, involves: ① Performing a Fourier transform on the speech received by the microphone to obtain the spectrogram signal of the original speech; ② Projecting the spectrogram signal of the original speech onto multiple candidate directions using a fixed beamforming algorithm, outputting multi-channel speech spectrogram signals, where each channel corresponds to a candidate direction; ③ Calculating the energy of the spectrogram in each direction, and outputting the direction corresponding to the signal of the spectrogram with the highest energy as the direction of the human voice.

[0003] Existing traditional algorithms typically rely solely on energy to determine whether a signal is a human voice. When significant external noise is present, noise energy often dominates, making it easy for the sound source localization result to incorrectly point to the noise source instead of the target human voice. Furthermore, beamforming algorithms have high computational and memory requirements and low discrimination within the same direction, limiting the number of candidate regions and resulting in low accuracy. Traditional algorithms, when human voice direction localization accuracy is low, use IPD, GCC, or MUSIC algorithms to assist in estimating the voice's location through phase. However, IPD and GCC algorithms also have limited accuracy, while MUSIC requires significant computational power and does not support single-frame sound source localization, requiring multi-frame estimation and introducing additional latency. When interference resistance is poor, energy is used to determine the start and end times of the human voice, estimating the sound source location only within that timeframe. However, this approach remains unsuitable in interference-prone scenarios.

[0004] The inventors discovered that the above-mentioned technology provides only one capability. The accuracy of sound source localization in traditional algorithms is not high. Generally, the number of localized areas is equal to the number of microphones. At the same time, it has poor resistance to interference and is easily affected by environmental noise. Summary of the Invention

[0005] The embodiments of the present invention are intended to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a voice localization method, comprising: converting an acquired audio signal into a spectrogram signal, and separating the spectrogram signal; filtering out non-voice signal frequency points in each direction of the separated spectrogram signal using a neural network, and determining the probability of the presence of a voice signal in each direction of the spectrogram signal, determining the direction with the highest probability of the presence of the voice signal and outputting a mask signal of the direction with the highest probability of the presence of the voice signal; and estimating the directional information of the voice signal based at least on the direction with the highest probability of the presence of the voice signal and the corresponding mask signal.

[0007] Secondly, embodiments of the present invention provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described human voice localization methods of the present invention.

[0008] Thirdly, embodiments of the present invention provide a storage medium storing one or more programs including execution instructions, the execution instructions being readable and executed by electronic devices (including but not limited to computers, servers, or network devices, etc.) to perform any of the above-described human voice localization methods of the present invention.

[0009] Fourthly, embodiments of the present invention also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-described human voice localization methods.

[0010] This invention utilizes a neural network to output a mask signal for the direction with the highest probability. Based on the mask signal for the direction with the highest probability, the directional information of the human voice signal is estimated. Even when there is a lot of external noise, the sound source localization result can still accurately point to the target human voice source, further improving the accuracy of target human voice localization. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of an embodiment of the human voice localization method of the present invention;

[0013] Figure 2A flowchart of another embodiment of the human voice localization method of the present invention;

[0014] Figure 3 This is a flowchart of another embodiment of the human voice localization method of the present invention;

[0015] Figure 4 A flowchart illustrating the process of human voice localization according to an embodiment of the present invention;

[0016] Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0019] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, elements, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0020] In this invention, terms such as "module," "device," and "system" refer to relevant entities applied to a computer, such as hardware, combinations of hardware and software, software, or software in execution. More specifically, for example, an element can be, but is not limited to, a process running on a processor, a processor, an object, an executable element, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be an element. One or more elements may be in an execution process and / or thread, and elements may be localized on a single computer and / or distributed across two or more computers, and may be run on various computer-readable media. Elements can also communicate via local and / or remote processes based on signals having one or more data packets, for example, signals from data interacting with another element in a local system, a distributed system, and / or interacting with other systems via signals over a network of the Internet.

[0021] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0022] This invention provides a method for human voice localization, which can be applied to electronic devices. The electronic device can be a computer, server, or other electronic product, etc., and this invention does not limit this to any particular device.

[0023] Please refer to Figure 1 This illustrates a human voice localization method provided by an embodiment of the present invention.

[0024] like Figure 1 As shown, in step 101, the acquired audio signal is converted into a spectrogram signal, and the spectrogram signal is separated.

[0025] In step 102, non-human voice signal frequency points in each direction of the separated spectrogram signal are filtered out by a neural network, and the probability of the presence of human voice signal in each direction of the spectrogram signal is determined. The direction with the highest probability of human voice signal presence is determined and the mask signal of the direction with the highest probability of human voice signal presence is output.

[0026] In step 103, the directional information of the human voice signal is estimated at least based on the direction in which the human voice signal has the highest probability of existence and the corresponding mask signal.

[0027] In this embodiment, for step 101, based on the audio signal acquired by the microphone device, the audio signal is converted into a spectrogram signal. The spectrogram signal is obtained by performing a short-time Fourier transform on the acquired audio signal, wherein the audio signal contains human voice signals and non-human voice signals. After obtaining the spectrogram signal, the spectrogram signal is separated into multiple audio channels according to the alternative directions using a beamforming algorithm, wherein the audio in each channel also contains human voice audio and non-human voice audio.

[0028] Next, for step 102, a neural network is used to filter out non-human voice signals in each channel of the separated spectrogram signal. These non-human voice signals are equivalent to noise. The neural network removes noise from all audio channels in the spectrogram signal and classifies audio frequencies. The energy of frequencies classified as non-human voice signals is set to zero to reduce the interference of non-human voice noise. The probability of human voice signals in each direction in the spectrogram signal is determined frame by frame. The probability of human voice presence in a frame is summed to obtain the probability of human voice presence in that frame. The audio channel with the highest probability of human voice presence is selected based on the probability of human voice presence. The mask signal of the audio channel with the highest probability of human voice presence is determined and output. For example, the neural network is used to determine the probability of speech presence in each direction and at each frequency. Each direction corresponds to each channel after separation in the spectrogram signal. The spectrogram signal in each direction outputs mask information. Then, the speech presence probability judgment module selects the candidate direction with the highest probability of human voice presence as the approximate range for human voice localization and outputs the mask signal of the corresponding direction.

[0029] Finally, for step 103, the directional information of the human voice signal is estimated based at least on the direction with the highest probability of the human voice signal and the corresponding mask signal. For example, the direction with the highest probability of the human voice signal is selected as the approximate direction of the audio target based on the probability of the human voice signal. Then, the directional information of the audio is estimated using a compressed sensing algorithm, and the directional information of non-human voices is masked by speech activation detection. Finally, the directional information of the human voice signal is determined.

[0030] The method in this application embodiment uses a neural network to output a mask signal of the direction with the highest probability. Based on the mask signal of the direction with the highest probability, the directional information of the human voice signal is estimated. When the external noise is large, the sound source localization result can also accurately point to the target human voice source, further improving the accuracy of target human voice localization.

[0031] Please refer to Figure 2 This illustrates another voice localization method provided by an embodiment of the present invention. The flowchart is primarily a summary of the flowchart. Figure 1The flowchart further defines the step 102 in the middle section, which involves "filtering out non-human voice signals in each direction of the separated spectrogram signal using a neural network and determining the probability of the presence of human voice signals in each direction of the spectrogram signal".

[0032] like Figure 2 As shown, in step 201, a neural network is used to classify the frequency points on the audio channel corresponding to each direction, and the energy of the frequency points classified as non-human voice signals is set to zero.

[0033] In step 202, the probability of human voice presence at each frequency point on each audio channel is determined by the probability of voice presence, and a mask signal is output for the direction in which the sum of the probabilities of human voice presence is maximized. The range in which human voice is located is the range in which human voice is maximized at each frequency point on each audio channel.

[0034] In this embodiment, for step 201, non-human voice signals on the corresponding channels of each direction are filtered out by a neural network. The neural network is used to classify audio frequency points, and the energy of frequency points classified as non-human voice signals is set to zero to reduce the interference of non-human voice noise. The neural network can roughly remove non-human voice energy and reduce non-human voice noise interference. The neural network can filter out the noise signals remaining in the previous step and output a mask to distinguish between speech frequency points and noise frequency points.

[0035] For step 202, the speech presence probability judgment module determines the probability of human voice presence at each frequency point on each audio channel and outputs a mask signal for the direction with the highest probability of human voice presence. The speech presence probability judgment module adaptively selects the approximate direction of human voice based on the output of the neural network, so that even if the speaker is walking while speaking, the switching and tracking can be performed smoothly. The range of human voice localization is the one where the sum of the human voice presence probabilities at each frequency point on each audio channel is the largest. The human voice presence probabilities of the frequency points in a frame are added together to obtain the human voice presence probability of that frame. The direction with the highest probability of human voice presence is selected as the approximate range of human voice localization, and the mask signal of the corresponding direction is output. The mask signal of the corresponding direction is the noise-reduced audio.

[0036] The method in this application embodiment achieves noise reduction by filtering out non-human voice signals on the audio channel corresponding to each direction, and then determines the direction with the highest probability of human voice signal existence as the range of human voice localization, which further improves the accuracy of human voice localization.

[0037] Please refer to this again. Figure 3 This illustrates another voice localization method provided by an embodiment of the present invention. The flowchart is primarily a summary of the flowchart. Figure 1The flowchart further defines the step of "estimating the directional information of the human voice signal based at least on the direction with the highest probability of the human voice signal and the corresponding mask signal" in step 103.

[0038] like Figure 3 As shown, in step 301, the fine orientation spectrum of each frame signal on the channel with the highest probability of the human voice is calculated using the single-frame sound source localization algorithm of compressed sensing.

[0039] In step 302, the directional information of the human voice signal is smoothed by voice activation detection.

[0040] In this embodiment, for step 301, the fine azimuth spectrum of each frame signal on the channel with the highest probability of human voice presence is calculated using a single-frame sound source localization algorithm based on compressed sensing. After audio noise reduction processing, the azimuth information of the audio is estimated using the compressed sensing algorithm. The fine azimuth spectrum of each frame signal on the channel with the highest probability of human voice presence in the audio is calculated. The compressed sensing algorithm can estimate the azimuth information of the audio frame by frame in real time. The compressed sensing algorithm requires the use of a human voice mask and the original recording. The mask of each frequency point is multiplied with each frequency point of the original audio. The usable audio frequency points are extracted using the mask information and input into the compressed sensing algorithm module to achieve the purpose of reducing noise interference.

[0041] For step 302, the voice activation detection module uses the fine azimuth spectrum cache information output by the compressed sensing algorithm to smooth the fluctuating single-frame azimuth information. The voice activation detection module can also fill in the azimuth information of the voice during short pauses between discontinuous voices.

[0042] The method in this application embodiment calculates the fine azimuth spectrum of each frame signal on the channel corresponding to the channel with the highest probability of human voice presence, and uses the fine azimuth spectrum to smooth the azimuth information of the human voice signal, which can make the final output result smoother.

[0043] In some optional embodiments, voice activation detection is used to determine whether the current frame is a voice frame, a voice pause frame, or a silence frame after the voice stops. Smoothing processing is added based on the directional information of different types of frames. For example, the voice activation detection module determines the components of each frame in the voice speech and performs smoothing processing on the fine directional spectrum information of the speech for the three types of states: voice frames, pause frames between voices, and silence frames after the voice stops. This can make the fine directional information of the final output conform to the user's intuition.

[0044] In some optional embodiments, the fine azimuth spectrum of each frame of signal is smoothed, and the human voice in the fine azimuth spectrum of each frame of signal is located using a peak search algorithm to confirm the position of the human voice in all frames of signal. If there are brief pause frames in the process of locating the human voice in the fine azimuth spectrum of each frame of signal using the peak search algorithm, the human voice location information is filled in between the brief pause frames based on the human voice location information of the preceding and following frames. If the interval between the human voice in the current frame and the previous human voice frame is less than a set time, and the DOA results of the two human voice frames are the same or differ by one... Within a certain range, the non-human voice frames between two human voice frames are considered as brief pause frames, and the DOA result of the previous human voice frame is used to replace the DOA result of the non-human voice segment. Then, the orientation buffer information output by the compressed sensing algorithm is used to smooth the fluctuating orientation information of the single frame, and the orientation information of the human voice can be filled in during short pauses between discontinuous human voices. The orientation spectrum output by the speech activation detection module is smoothed, and the human voice is located by the peak search algorithm. The human voice positioning information is filled in between brief pause frames, making the result more consistent with the user's experience.

[0045] In some optional embodiments, a microphone device is used to acquire noisy audio signals in real time, and the acquired audio signals are converted into spectrogram signals by short-time Fourier transform. The noisy audio signals include human voice signals and non-human voice signals. The microphone device consists of a microphone array, which is used to receive noisy speech signals in real time and then perform short-time Fourier transform on the noisy speech signals.

[0046] In some optional embodiments, beamforming algorithms are used to project the converted spectrogram signal onto multiple candidate directions, outputting a multi-channel spectrogram signal, where each channel corresponds to a candidate direction. The beamforming algorithm can separate the spectrogram signal according to the candidate directions, separate human voice from background noise according to direction, reduce noise energy, and improve the signal-to-noise ratio of the audio signal.

[0047] It should be noted that this application first uses a beamforming algorithm to separate the audio into multiple channels based on candidate directions. Then, a neural network is used to filter out non-human voice energy in each direction. Next, the approximate direction of the target is selected based on the probability of speech presence. Then, a compressed sensing algorithm is used to estimate the directional information of this audio, and the output is processed through speech activation detection, masking the directional information of non-human voices.

[0048] The neural network can roughly remove non-human voice energy and reduce non-human voice noise interference. The speech presence probability judgment module adaptively selects the approximate direction of the human voice based on the output of the neural network, and can smoothly switch and track even if the speaker is walking while speaking. The output audio at this time is the noise-reduced audio, and the compressed sensing algorithm can estimate the location information of the audio frame by frame in real time. Finally, the speech activation detection module uses the location buffer information output by the compressed sensing algorithm to smooth the fluctuating location information of a single frame, and can fill in the location information of the human voice during short pauses between discontinuous human voices. This makes the final output result smoother and more in line with human intuitive perception.

[0049] It should be noted that this application also provides an alternative solution: using a fixed beam matrix in a generalized sidelobe canceller instead of the steering vector in a classic compressed sensing algorithm. This method can improve the algorithm performance in the low-frequency band and reduce the computational power of algorithm initialization, but it requires pre-calculation of the fixed beam matrix. In the speech presence probability module, low-precision phase-based source location algorithms such as IPD or GCC are used to assist in channel and mask selection. This solution will increase the computational power and memory requirements.

[0050] Please refer to Figure 4 The document presents a flowchart illustrating the implementation of the human voice localization method of the present invention.

[0051] like Figure 4 As shown, step 1: Use a microphone array to receive noisy speech signals in real time and convert them into spectrogram signals through short-time Fourier transform.

[0052] Step 2: Use beamforming algorithm to separate the spectrogram signal according to the alternative directions.

[0053] Step 3: Use a neural network to determine the probability of speech presence in each direction and at each frequency point, and output mask information for the spectrogram signal in each direction.

[0054] Step 4: Using the speech presence probability judgment module, select the candidate direction with the highest probability of human voice presence as the approximate range for human voice localization, and output the mask signal of the corresponding direction.

[0055] Step 5: Use compressed sensing algorithm to calculate the fine orientation spectrum of each frame of signal.

[0056] Step 6: Use the voice activation detection module to smooth the azimuth spectrum output in the previous step and use the peak search algorithm to locate the human voice, filling in the human voice location information between brief pause frames to make the result more in line with the user's experience.

[0057] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of combined actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps can be performed in other orders or simultaneously according to the present invention. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0058] In some embodiments, the present invention provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions, which can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform any of the above-described human voice localization methods of the present invention.

[0059] In some embodiments, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-described human voice localization methods.

[0060] In some embodiments, the present invention also provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a voice localization method.

[0061] Figure 5 This is a schematic diagram of the hardware structure of an electronic device for performing a voice localization method according to another embodiment of this application, as shown below. Figure 5 As shown, the device includes:

[0062] One or more processors 510 and memory 520, Figure 5 Take the 510 processor as an example.

[0063] The device for performing the voice localization method may further include an input device 530 and an output device 540.

[0064] The processor 510, memory 520, input device 530, and output device 540 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0065] The memory 520, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the voice localization method in the embodiments of this application. The processor 510 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 520, thereby implementing the voice localization method in the above-described method embodiments.

[0066] The memory 520 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the voice location device. Furthermore, the memory 520 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 520 may optionally include memory remotely located relative to the processor 510, and this remote memory may be connected to the voice location device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0067] Input device 530 can receive input digital or character information, and generate signals related to user settings and function control of the voice localization device. Output device 540 may include display devices such as a display screen.

[0068] The one or more modules are stored in the memory 520, and when executed by the one or more processors 510, they perform the human voice localization method in any of the above method embodiments.

[0069] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0070] The electronic devices in this application embodiments exist in various forms, including but not limited to:

[0071] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0072] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include: PDAs, MIDs, and UMPCs, etc.

[0073] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0074] (4) Other airborne electronic devices with data interaction capabilities, such as vehicle-mounted systems installed on vehicles.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for human voice localization, comprising: The acquired audio signal is converted into a spectrogram signal, and the spectrogram signal is then separated. The non-human voice signal frequency points in each direction of the separated spectrogram signal are filtered out by a neural network, and the existence probability of human voice signal in each direction of the spectrogram signal is judged frame by frame. The direction with the highest probability of human voice signal existence is determined and the mask signal of the direction with the highest probability of human voice signal existence is output. The directional information of the human voice signal is estimated based at least on the direction in which the human voice signal has the highest probability of existence and the corresponding mask signal. Specifically, the fine directional spectrum of each frame signal on the corresponding channel with the highest probability of existence of the human voice signal is calculated using a single-frame sound source localization algorithm based on compressed sensing. The directional information of the human voice signal is smoothed by voice activation detection.

2. The method according to claim 1, wherein, The step of filtering out non-human voice frequency points in each direction of the separated spectrogram signal using a neural network and determining the probability of the presence of human voice signal in each direction of the spectrogram signal includes: The frequency points on the corresponding audio channel in each direction are classified using a neural network, and the energy of the frequency points classified as non-human voice signals is set to zero. Then, the probability of human voice signal presence at each frequency point on each audio channel is determined by the probability of voice presence, and a mask signal is output for the direction with the highest probability of human voice signal presence. The range with the largest sum of the probabilities of human voice signal presence at each frequency point on each audio channel is the range for voice localization.

3. The method according to claim 1, wherein, The smoothing process of the location information of the human voice signal through voice activation detection includes: Based on the voice activation detection, it is determined whether the current frame is a voice frame, a voice pause frame, or a silent frame after the voice stops, and corresponding smoothing processing is added according to the orientation information of different types of frames.

4. The method according to claim 1, wherein, The algorithm for single-frame sound source localization using compressed sensing to calculate the fine orientation spectrum of each frame signal in the channel corresponding to the channel with the highest probability of the human voice signal includes: The fine orientation spectrum of each frame of signal is smoothed, and the human voice in the fine orientation spectrum of each frame of signal is located by using a peak search algorithm to confirm the position of the human voice in all frames of signal.

5. The method according to claim 4, wherein, The method further includes: If there are brief pause frames when locating human voices in the fine orientation spectrum of each frame of signal using the peak search algorithm, then human voice location information is filled between the brief pause frames.

6. The method according to claim 1, wherein, The process of converting the acquired audio signal into a spectrogram signal includes: The audio signal with noise is acquired in real time by a microphone device, and the audio signal is converted into a spectrogram signal by a short-time Fourier transform. The audio signal with noise includes human voice signal and non-human voice signal. The microphone device is composed of a microphone array.

7. The method according to claim 1, wherein, The separation of the spectrogram signal includes: The spectrogram signal is projected onto multiple candidate directions using a beamforming algorithm, and a multi-channel spectrogram signal is output, wherein each channel corresponds to one candidate direction.

8. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images