Voice activity detection method, system, speech enhancement method and system

Abstract

The voice activity detection method and system and the voice enhancement method and system provided in the specification determine a voice presence probability that a target voice signal exists in a microphone signal by calculating a linear correlation between a signal subspace where the microphone signal is located and a target subspace where the target voice signal is located. The voice enhancement method and system can calculate a filter coefficient based on the voice presence probability, thereby performing voice enhancement on the microphone signal. The calculation accuracy of the voice presence probability can be improved, and the voice enhancement effect is further improved.

Description

Technical Field

[0001] This specification relates to the field of target speech signal processing technology, and in particular to a speech activity detection method, system, speech enhancement method, and system. Background Technology

[0002] In beamforming-based speech enhancement technologies, particularly in adaptive beamforming algorithms using Minimum Variance Distortionless Response (MVDR), determining the noise covariance matrix—the parameter describing the relationship between the noise statistical characteristics of different microphones—is crucial. Existing methods primarily calculate the noise covariance matrix based on speech presence probability, such as estimating the probability of speech presence using Voice Activity Detection (VAD) methods and then calculating the noise covariance matrix. However, the accuracy of speech presence probability estimation in existing technologies is insufficient, leading to low precision in noise covariance matrix estimation and consequently, poor speech enhancement performance of MVDR algorithms. This is especially true when the number of microphones is small, such as less than five, where the performance drops sharply. Therefore, existing MVDR algorithms are mostly used in microphone array devices with a large number of microphones and wide spacing, such as mobile phones and smart speakers, while their speech enhancement effect is poor for devices like headphones with fewer microphones and smaller spacing.

[0003] Therefore, there is a need to provide a more accurate method, system, speech enhancement method, and system for speech activity detection. Summary of the Invention

[0004] This specification provides a more accurate method and system for speech activity detection, as well as a speech enhancement method and system.

[0005] In a first aspect, this specification provides a voice activity detection method for M microphones distributed in a preset array shape, where M is an integer greater than 1, comprising: acquiring microphone signals output by the M microphones; determining a signal subspace formed by the microphone signals based on the microphone signals; determining a target subspace formed by a target speech signal; and determining and outputting the probability of speech presence of the target speech signal in the microphone signals based on the linear correlation between the signal subspace and the target subspace.

[0006] In some embodiments, determining the signal subspace formed by the microphone signal based on the microphone signal includes: determining the sampling covariance matrix of the microphone signal based on the microphone signal; performing eigenvalue decomposition on the sampling covariance matrix to determine multiple eigenvectors of the sampling covariance matrix; and using a matrix composed of at least some of the eigenvectors as the basis matrix of the signal subspace.

[0007] In some embodiments, determining the signal subspace formed by the microphone signal based on the microphone signal includes: determining the azimuth angle of the signal source in the microphone signal using a spatial estimation method based on the microphone signal, thereby determining the signal guidance vector of the microphone signal, wherein the spatial estimation method includes at least one of a DOA estimation method and a spatial spectrum estimation method; and determining the signal guidance vector as the basis matrix of the signal subspace.

[0008] In some embodiments, determining the target subspace constituted by the target speech signal includes: determining a preset target guidance vector corresponding to the target speech signal as the basis matrix of the target subspace.

[0009] In some embodiments, determining and outputting the probability of speech presence of the target speech signal in the microphone signal based on the linear correlation between the signal subspace and the target subspace includes: determining a volume correlation function between the signal subspace and the target subspace; determining a linear correlation coefficient between the signal subspace and the target subspace based on the volume correlation function, wherein the linear correlation coefficient is negatively correlated with the volume correlation function; and outputting the linear correlation coefficient as the probability of speech presence.

[0010] In some embodiments, determining the linear correlation coefficient between the signal subspace and the target subspace based on the volume correlation function includes one of the following: determining that the volume correlation function is greater than a first threshold and determining that the linear correlation coefficient is 0; determining that the volume correlation function is less than a second threshold and determining that the linear correlation coefficient is 1, wherein the second threshold is less than the first threshold; and determining that the volume correlation function is between the first threshold and the second threshold and determining that the linear correlation coefficient is between 0 and 1, and the linear correlation coefficient is a negative correlation function of the volume correlation function.

[0011] Secondly, this specification also provides a voice activity detection system, including at least one storage medium and at least one processor, wherein the at least one storage medium stores at least one instruction set for voice activity detection; and the at least one processor is communicatively connected to the at least one storage medium, wherein when the voice activity detection system is running, the at least one processor reads the at least one instruction set and implements the voice activity detection method described in the first aspect of this specification.

[0012] Thirdly, this specification also provides a speech enhancement method for M microphones distributed in a preset array shape, where M is an integer greater than 1, comprising: acquiring microphone signals output by the M microphones; determining the probability of the presence of the target speech signal in the microphone signals based on the speech activity detection method described in the first aspect of this specification; determining a filter coefficient vector corresponding to the microphone signals based on the speech presence probability; and merging the microphone signals based on the filter coefficient vector to obtain a target audio signal and output it.

[0013] In some embodiments, determining the filter coefficient vector corresponding to the microphone signal based on the speech presence probability includes: determining the noise covariance matrix of the microphone signal based on the speech presence probability; and determining the filter coefficient vector based on the MVDR method and the noise covariance matrix.

[0014] In some embodiments, determining the filter coefficient vector corresponding to the microphone signal based on the speech presence probability includes: using the speech presence probability as the filter coefficient corresponding to a target microphone signal in the microphone signal, the target microphone signal including the microphone signal with the highest signal-to-noise ratio in the microphone signal; and determining that the filter coefficients corresponding to the other microphone signals in the microphone signal besides the target microphone signal are 0, the filter coefficient vector including a vector composed of the filter coefficient corresponding to the target microphone signal and the filter coefficients corresponding to the other microphone signals.

[0015] Fourthly, this specification also provides a voice enhancement system, including at least one storage medium and at least one processor, wherein the at least one storage medium stores at least one instruction set for performing voice enhancement; and the at least one processor is communicatively connected to the at least one storage medium, wherein when the voice enhancement system is running, the at least one processor reads the at least one instruction set and implements the voice enhancement method described in the second aspect of this specification.

[0016] As can be seen from the above technical solutions, the speech activity detection method, system, speech enhancement method, and system provided in this specification are used in a microphone array composed of multiple microphones. The microphone array can acquire both noise signals and target speech signals, and output microphone signals. Target speech signals and noise signals are two unconnected signals. The target subspace containing the target speech signal and the noise subspace containing the noise signal are two unconnected subspaces. When the target speech signal is absent from the microphone signal, the microphone signal only contains noise signals. In this case, the signal subspace containing the microphone signal and the target subspace containing the target speech signal are two unconnected subspaces, and the linear correlation between the signal subspace and the target subspace is low. When the target speech signal is present in the microphone signal, the microphone signal contains both the target speech signal and noise signals. In this case, the signal subspace containing the microphone signal and the target subspace containing the target speech signal are two connected subspaces, and the linear correlation between the signal subspace and the target subspace is high. Therefore, the speech activity detection method and system provided in this specification can determine the probability of the presence of the target speech signal in the microphone signal by calculating the linear correlation between the signal subspace where the microphone signal is located and the target subspace where the target speech signal is located. The speech enhancement method and system can calculate filtering coefficients based on the speech presence probability to enhance the microphone signal. The method and system can improve the accuracy of the speech presence probability calculation, thereby improving the speech enhancement effect.

[0017] The speech activity detection methods, systems, speech enhancement methods, and other functions of the systems provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the speech activity detection methods, systems, speech enhancement methods, and systems provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A hardware schematic diagram of a voice activity detection system provided according to an embodiment of this specification is shown; Figure 2A An exploded view of an electronic device provided according to an embodiment of this specification is shown. Figure 2B A front view of a first housing provided according to an embodiment of this specification is shown; Figure 2C A top view of a first housing provided according to an embodiment of this specification is shown; Figure 2D A front view of a second housing provided according to an embodiment of this specification is shown; Figure 2E A bottom view of a second housing provided according to an embodiment of this specification is shown; Figure 3 A flowchart of a speech activity detection method provided according to an embodiment of this specification is shown; Figure 4 A flowchart illustrating a method for determining the basis matrix of a signal subspace according to an embodiment of this specification is shown; Figure 5 A flowchart illustrating another method for determining the basis matrix of a signal subspace, according to an embodiment of this specification, is shown; Figure 6 A flowchart illustrating a method for calculating the probability of speech presence according to an embodiment of this specification is shown; Figure 7 A schematic diagram of a spatial protagonist provided according to an embodiment of this specification is shown; and Figure 8 A flowchart of a speech enhancement method provided according to an embodiment of this specification is shown. Detailed Implementation

[0020] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0021] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0022] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0023] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0024] For ease of description, the following explanations are provided for the terms that will appear in the instruction manual: Minimum Variance Distortionless Response (MVDR) is an adaptive beamforming algorithm based on the maximum signal-to-interference-plus-noise ratio (SINR) criterion. The MVDR algorithm adaptively minimizes the array output power in the desired direction while maximizing the SINR. Its goal is to minimize the variance of the recorded signal. If the noise signal and the desired signal are uncorrelated, then the variance of the recorded signal is the sum of the variances of the desired signal and the noise signal. Therefore, the MVDR solution seeks to minimize this sum, thereby mitigating the impact of the noise signal. Its principle is to select appropriate filter coefficients under the constraint of distortion-free desired signal, so as to minimize the average power of the array output.

[0025] Speech activity detection: The process of segmenting speaking and non-speaking periods in a target speech signal.

[0026] Gaussian distribution: The normal distribution, also known as the Gaussian distribution, has a bell-shaped curve, low at both ends and high in the middle, and is symmetrical. Because of its bell-shaped curve, it is often called a bell curve. If a random variable X follows a normal distribution with a mathematical expectation of... variance is The normal distribution is denoted as N. Its probability density function is the expected value of a normal distribution. Its position is determined by its standard deviation. This determines the magnitude of the distribution. When , The normal distribution at time t is the standard normal distribution.

[0027] Figure 1 A hardware schematic diagram of a voice activity detection system according to an embodiment of this specification is shown. The voice activity detection system can be applied to an electronic device 200.

[0028] In some embodiments, the electronic device 200 may be a wireless headset, a wired headset, or a smart wearable device, such as smart glasses, a smart helmet, or a smartwatch, which are devices with audio processing capabilities. The electronic device 200 may also be a mobile device, a tablet computer, a laptop computer, a vehicle-mounted device, or similar content, or any combination thereof. In some embodiments, the mobile device may include smart home devices, smart mobile devices, or similar devices, or any combination thereof. For example, the smart mobile device may include a mobile phone, a personal digital assistant, a gaming device, a navigation device, an Ultra-mobile Personal Computer (UMPC), or any combination thereof. In some embodiments, the smart home device may include a smart TV, a desktop computer, or any combination thereof. In some embodiments, the vehicle-mounted device may include an in-vehicle computer, an in-vehicle TV, or similar content.

[0029] In this specification, we use an electronic device 200 as an example to describe it. The earphone can be a wireless earphone or a wired earphone. Figure 1 As shown, the electronic device 200 may include a microphone array 220 and a computing device 240.

[0030] Microphone array 220 can be an audio acquisition device of electronic device 200. Microphone array 220 can be configured to acquire local audio and output microphone signals, i.e., electronic signals carrying audio information. Microphone array 220 can include M microphones 222 distributed in a preset array shape. M is an integer greater than 1. The M microphones 222 can be uniformly distributed or non-uniformly distributed. The M microphones 222 can output microphone signals. The M microphones 222 can output M microphone signals. Each microphone 222 corresponds to one microphone signal. The M microphone signals are collectively referred to as the microphone signals. In some embodiments, the M microphones 222 can be linearly distributed. In some embodiments, the M microphones 222 can also be arranged in other array shapes, such as a circular array, a rectangular array, etc. For ease of description, the following description will use a linear distribution of M microphones 222 as an example. In some embodiments, M can be any integer greater than 1, such as 2, 3, 4, 5, or even more, etc. In some embodiments, due to space limitations, M can be an integer greater than 1 and not greater than 5, such as in products like headphones. When the electronic device 200 is an earphone, the spacing between adjacent microphones 222 among the M microphones 222 can be between 20mm and 40mm. In some embodiments, the spacing between adjacent microphones 222 can be smaller, for example, between 10mm and 20mm.

[0031] In some embodiments, microphone 222 may be a bone conduction microphone that directly acquires vibration signals from the human body. The bone conduction microphone may include a vibration sensor, such as an optical vibration sensor or an accelerometer. The vibration sensor can acquire mechanical vibration signals (e.g., signals generated by vibrations produced by the skin or bones when a user speaks) and convert these mechanical vibration signals into electrical signals. The mechanical vibration signals referred to here primarily refer to vibrations propagating through solids. The bone conduction microphone contacts the user's skin or bones through the vibration sensor or a vibrating component connected to the vibration sensor, thereby acquiring vibration signals generated by the bones or skin when the user speaks and converting these vibration signals into electrical signals. In some embodiments, the vibration sensor may be a device that is sensitive to mechanical vibrations but insensitive to air vibrations (i.e., the vibration sensor's response to mechanical vibrations exceeds its response to air vibrations). Because the bone conduction microphone can directly pick up vibration signals from the sound-producing part, it can reduce the impact of environmental noise.

[0032] In some embodiments, microphone 222 may also be an air conduction microphone that directly acquires air vibration signals. An air conduction microphone acquires air vibration signals caused by the user emitting sound and converts these signals into electrical signals.

[0033] In some embodiments, the M microphones 222 may be M bone conduction microphones. In some embodiments, the M microphones 222 may also be M air conduction microphones. In some embodiments, the M microphones 222 may include both bone conduction microphones and air conduction microphones. Of course, the microphones 222 may also be other types of microphones, such as optical microphones, microphones for receiving electromyographic signals, etc.

[0034] The computing device 240 can be communicatively connected to the microphone array 220. This communicative connection refers to any form of connection capable of directly or indirectly receiving information. In some embodiments, the computing device 240 and the microphone array 220 can exchange data via a wireless communication connection; in some embodiments, the computing device 240 can also exchange data with the microphone array 220 via a direct wire connection; in some embodiments, the computing device 240 can also establish an indirect connection with the microphone array 220 by directly connecting it to other circuits via a wire, thereby enabling data exchange. This specification will describe the direct wired connection between the computing device 240 and the microphone array 220 as an example.

[0035] The computing device 240 may be a hardware device with data processing capabilities. In some embodiments, the voice activity detection system may include the computing device 240. In some embodiments, the voice activity detection system may be applied to the computing device 240. That is, the voice activity detection system may run on the computing device 240. The voice activity detection system may include a hardware device with data processing capabilities and the necessary programs to drive the hardware device to work. Of course, the voice activity detection system may also be simply a hardware device with data processing capabilities, or simply a program running on the hardware device.

[0036] The voice activity detection system may store data or instructions for executing the voice activity detection method described herein, and may execute the data and / or instructions. When the voice activity detection system is running on the computing device 240, the voice activity detection system may acquire the microphone signal from the microphone array 220 based on the communication connection, and execute the data or instructions of the voice activity detection method described herein to calculate the probability of the presence of the target voice signal in the microphone signal. The voice activity detection method is described in other parts of this specification. For example, in Figures 3 to 8 The description of the speech activity detection method is presented in the document.

[0037] like Figure 1 As shown, the computing device 240 may include at least one storage medium 243 and at least one processor 242. In some embodiments, the electronic device 200 may also include a communication port 245 and an internal communication bus 241.

[0038] The internal communication bus 241 can connect different system components, including storage medium 243, processor 242 and communication port 245.

[0039] The communication port 245 can be used for data communication between the computing device 240 and the outside world. For example, the computing device 240 can obtain the microphone signal from the microphone array 220 through the communication port 245.

[0040] At least one storage medium 243 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk, a read-only storage medium (ROM), or a random access storage medium (RAM). When the voice activity detection system can run on the computing device 240, the storage medium 243 may also include at least one set of instructions stored in the data storage device for performing voice activity detection on the microphone signal. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the voice activity detection method provided in this specification.

[0041] At least one processor 242 may be communicatively connected to at least one storage medium 243 via an internal communication bus 241. The communication connection refers to any form of connection capable of directly or indirectly receiving information. At least one processor 242 is used to execute the at least one instruction set described above. When the voice activity detection system is running on the computing device 240, at least one processor 242 reads the at least one instruction set and executes the voice activity detection method provided in this specification according to the instructions of the at least one instruction set. Processor 242 may execute all steps included in the voice activity detection method. Processor 242 may be in the form of one or more processors. In some embodiments, processor 242 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 242 is described in this specification for the computing device 240. However, it should be noted that the computing device 240 may also include multiple processors 242. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described herein, or they may be executed jointly by multiple processors. For example, if the processor 242 of the computing device 240 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 242 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0042] Figure 2A An exploded view of an electronic device 200 according to an embodiment of this specification is shown. Figure 2A As shown, the electronic device 200 may include a microphone array 220, a computing device 240, a first housing 260, and a second housing 280.

[0043] The first housing 260 can be a mounting base for the microphone array 220. The microphone array 220 can be installed inside the first housing 260. The shape of the first housing 260 can be adapted to the distribution shape of the microphone array 220, and this specification does not impose any restrictions on this. The second housing 280 can be a mounting base for the computing device 240. The computing device 240 can be installed inside the second housing 280. The shape of the second housing 280 can be adapted to the shape of the computing device 240, and this specification does not impose any restrictions on this. When the electronic device 200 is an earphone, the second housing 280 can be connected to the wearable part. The second housing 280 can be connected to the first housing 260. As mentioned above, the microphone array 220 can be electrically connected to the computing device 240. Specifically, the microphone array 220 and the computing device 240 can be electrically connected through the connection of the first housing 260 and the second housing 280.

[0044] In some embodiments, the first housing 260 may be fixedly connected to the second housing 280, for example, by integral molding, welding, riveting, bonding, etc. In some embodiments, the first housing 260 may be detachably connected to the second housing 280. The computing device 240 may be communicatively connected to different microphone arrays 220. Specifically, different microphone arrays 220 may have different numbers of microphones 222, different array shapes, different microphone spacing, different mounting angles of the microphone arrays 220 in the first housing 260, different mounting positions of the microphone arrays 220 in the first housing 260, etc. Users can replace the corresponding microphone arrays 220 according to different application scenarios to make the electronic device 200 suitable for a wider range of scenarios. For example, when the user is close to the electronic device 200 in the application scenario, the user can replace it with a microphone array 220 with a smaller spacing. Or, when the user is close to the electronic device 200 in the application scenario, the user can replace it with a microphone array 220 with a larger spacing and more microphones, etc.

[0045] The detachable connection can be any form of physical connection, such as a threaded connection, a snap-fit ​​connection, a magnetic connection, etc. In some embodiments, the first housing 260 and the second housing 280 can be magnetically connected. That is, the first housing 260 and the second housing 280 are detachably connected by the attraction force of a magnetic device.

[0046] Figure 2B A front view of a first housing 260 provided according to an embodiment of this specification is shown; Figure 2C A top view of a first housing 260 provided according to an embodiment of this specification is shown. Figure 2B and Figure 2CAs shown, the first housing 260 may include a first interface 262. In some embodiments, the first housing 260 may further include a contact 266. In some embodiments, the first housing 260 may further include an angle sensor ( Figure 2B and Figure 2C (Not shown in the image).

[0047] The first interface 262 can be a mounting interface between the first housing 260 and the second housing 280. In some embodiments, the first interface 262 can be circular. The first interface 262 can be rotatably connected to the second housing 280. When the first housing 260 is mounted on the second housing 280, the first housing 260 can rotate relative to the second housing 280 to adjust the angle of the first housing 260 relative to the second housing 280, thereby adjusting the angle of the microphone array 220.

[0048] A first magnetic device 263 may be provided on the first interface 262. The first magnetic device 263 may be located near the second housing 280. The first magnetic device 263 can generate a magnetic attraction force, thereby achieving a detachable connection with the second housing 280. When the first housing 260 is close to the second housing 280, the attraction force enables the first housing 260 to quickly connect with the second housing 280. In some embodiments, after the first housing 260 and the second housing 280 are connected, the first housing 260 can also rotate relative to the second housing 280 to adjust the angle of the microphone array 220. Under the action of the attraction force, the connection between the first housing 260 and the second housing 280 can still be maintained when the first housing 260 rotates relative to the second housing 280.

[0049] In some embodiments, a first positioning device may also be provided on the first interface 262. Figure 2B and Figure 2C (Not shown in the image). The first positioning device can be an outwardly protruding positioning step or an inwardly extending positioning hole. The first positioning device can cooperate with the second housing 280 to achieve quick installation of the first housing 260 and the second housing 280.

[0050] like Figure 2B and Figure 2CAs shown, in some embodiments, the first housing 260 may further include contacts 266. Contacts 266 may be mounted at the first interface 262. Contacts 266 may protrude outward from the first interface 262. Contacts 266 may be elastically connected to the first interface 262. Contacts 266 may be communicatively connected to M microphones 222 in the microphone array 220. Contacts 266 may be made of a flexible metal to enable data transmission. When the first housing 260 is connected to the second housing 280, the microphone array 220 may communicate with the computing device 240 via contacts 266. In some embodiments, contacts 266 may be circularly distributed. After the first housing 260 is connected to the second housing 280, when the first housing 260 rotates relative to the second housing 280, contacts 266 may also rotate relative to the second housing 280, maintaining communication with the computing device 240.

[0051] In some embodiments, an angle sensor may also be provided on the first housing 260. Figure 2B and Figure 2C (Not shown in the image). The angle sensor can be communicatively connected to contact 266, thereby enabling communication with computing device 240. The angle sensor can collect angle data of the first housing 260 to determine the angle of microphone array 220, providing reference data for subsequent calculation of the probability of voice presence.

[0052] Figure 2D A front view of a second housing 280 provided according to an embodiment of this specification is shown; Figure 2E A bottom view of a second housing 280 provided according to an embodiment of this specification is shown. Figure 2D and Figure 2E As shown, the second housing 280 may include a second interface 282. In some embodiments, the second housing 280 may also include a guide rail 286.

[0053] The second interface 282 can be a mounting interface between the second housing 280 and the first housing 260. In some embodiments, the second interface 282 can be circular. The second interface 282 can be rotatably connected to the first interface 262 of the first housing 260. When the first housing 260 is mounted on the second housing 280, the first housing 260 can rotate relative to the second housing 280 to adjust the angle of the first housing 260 relative to the second housing 280, thereby adjusting the angle of the microphone array 220.

[0054] A second magnetic device 283 may be provided on the second interface 282. The second magnetic device 283 may be located near the first housing 260. The second magnetic device 283 can generate a magnetic attraction force, thereby achieving a detachable connection with the first interface 262. The second magnetic device 283 can be used in conjunction with the first magnetic device 263. When the first housing 260 is close to the second housing 280, the attraction force between the second magnetic device 283 and the first magnetic device 263 allows the first housing 260 to be quickly installed on the second housing 280. When the first housing 260 is installed on the second housing 280, the second magnetic device 283 and the first magnetic device 263 are positioned opposite each other. In some embodiments, after the first housing 260 and the second housing 280 are connected, the first housing 260 can also rotate relative to the second housing 280 to adjust the angle of the microphone array 220. Under the action of the attraction force, the connection between the first housing 260 and the second housing 280 can still be maintained when the first housing 260 rotates relative to the second housing 280.

[0055] In some embodiments, a second positioning device may also be provided on the second interface 282. Figure 2D and Figure 2E (Not shown in the image). The second positioning device can be an outwardly protruding positioning step or an inwardly extending positioning hole. The second positioning device can cooperate with the first positioning device of the first housing 260 to achieve quick installation of the first housing 260 and the second housing 280. When the first positioning device is the positioning step, the second positioning device can be the positioning hole. When the first positioning device is the positioning hole, the second positioning device can be the positioning step.

[0056] like Figure 2D and Figure 2EAs shown, in some embodiments, the second housing 280 may further include a guide rail 286. The guide rail 286 may be mounted at the second interface 282. The guide rail 286 may be communicatively connected to the computing device 240. The guide rail 286 may be made of metal to enable data transmission. When the first housing 260 is connected to the second housing 280, the contact 266 may contact the guide rail 286 to form a communication connection, thereby enabling the microphone array 220 to communicate with the computing device 240 for data transmission. As mentioned above, the contact 266 may be elastically connected to the first interface 262. Therefore, after the first housing 260 and the second housing 280 are connected, under the elastic force of the elastic connection, the contact 266 may fully contact the guide rail 286 to achieve a reliable communication connection. In some embodiments, the guide rail 286 may be circularly distributed. After the first housing 260 is connected to the second housing 280, when the first housing 260 rotates relative to the second housing 280, the contact 266 can also rotate relative to the guide rail 286 and maintain communication connection with the guide rail 286.

[0057] Figure 3 A flowchart of a speech activity detection method P100 provided according to an embodiment of this specification is shown. Method P100 can calculate the probability of speech presence in the microphone signal containing a target speech signal. Specifically, processor 242 can execute method P100. Figure 3 As shown, method P100 may include: S120: Acquire the microphone signals output by M microphones 222.

[0058] As previously mentioned, each microphone 222 can output a corresponding microphone signal. M microphones 222 correspond to M microphone signals. When calculating the probability of the presence of a target speech signal in the microphone signals, method P100 can perform the calculation based on all microphone signals among the M microphone signals, or it can perform the calculation based on a portion of the microphone signals. Therefore, the microphone signals can include either the M microphone signals corresponding to the M microphones 222 or a portion of the microphone signals. The following description in this specification will use the example of the microphone signals including the M microphone signals corresponding to the M microphones 222 as an example.

[0059] In some embodiments, the microphone signal may be a time-domain signal. For ease of description, we will label the microphone signal at time t as... Microphone signal It can be a signal vector composed of M microphone signals. In this case, the microphone signals... This can be expressed as the following formula: Formula (1) microphone signal This is a time-domain signal. In some embodiments, in step S120, the computing device 240 can also process the microphone signal. Spectrum analysis is performed. Specifically, the computing device 240 can perform spectrum analysis based on the time-domain signal of the microphone signal. Perform a Fourier transform to obtain the frequency domain signal of the microphone signal. In the following description, microphone signals in the frequency domain will be used. Describe the microphone signal. It can be an M-dimensional signal vector composed of M microphone signals. In this case, the microphone signals... It can be expressed as the following formula: Formula (2) As previously described, microphone 222 can collect ambient noise and output a noise signal, or it can collect the target user's speech and output the target speech signal. When the target user does not speak, the microphone signal only contains the noise signal. When the target user speaks, the microphone signal contains both the target speech signal and the noise signal. This can be expressed as the following formula: Formula (3) in, The complex amplitude of the target speech signal. The target guidance vector for the target speech signal. microphone signal Noise signals in the data. is the noise guiding vector of the noise signal.

[0060] The target speech signal has a complex amplitude. In some embodiments, there is one target speech signal source surrounding the microphone 222. In some embodiments, there are L target speech signal sources surrounding the microphone 222. It can be an L×1 dimensional vector. This can be expressed as the following formula: Formula (4) Target guidance vector It is an M×L dimensional matrix. Target guidance vector. This can be expressed as the following formula: Formula (5) in, For carrier frequency. d 222 is the distance between adjacent microphones 222. c is the speed of sound. ... These are the incident angles between the L target speech signal sources and the microphone 222. In some embodiments, the target speech signal sources... The angles are usually distributed within a specific range of angles. Therefore, ... The relative positions of the M microphones 222 are known. The computing device 240 pre-stores the relative positions of these microphones, such as relative distances or relative coordinates. That is, the computing device 240 pre-stores the distances between adjacent microphones 222. d In other words, the target guidance vector can be pre-stored in the computing device 240. .

[0061] noise signal It can be an M-dimensional signal vector collected by M microphones 222. (Noise signal) This can be expressed as the following formula: Let N be the complex amplitude of the noise signal. In some embodiments, N noise signal sources exist around the microphone 222. In this case, It can be an N×1 dimensional vector. This can be expressed as the following formula: Formula (5) The noise guidance vector It is an M×N dimensional matrix. Due to the irregularity of the noise signal, therefore, the noise signal... and noise guidance vector It is unknown.

[0062] In some embodiments, formula (3) can also be expressed as the following formula: Formula (6) in, It's a microphone signal. The signal guidance vector.

[0063] As mentioned earlier, microphone 222 can collect target speech signals. It can also collect noise signals. Target speech signal With noise signal These are two types of signals that are not interconnected. Therefore, the target speech signal The target subspace and noise signal The noise subspace it resides in belongs to two unconnected subspaces. Therefore, when the microphone signal... The target speech signal is not present. At that time, microphone signal It contains only noise signals At this moment, the microphone signal The signal subspace where it is located and the target speech signal The target subspace belongs to two unconnected subspaces, and the linear correlation between the signal subspace and the target subspace is low or zero. When the microphone signal... The target speech signal exists. At that time, microphone signal It contains the target speech signal It also contains noise signals. At this moment, the microphone signal The signal subspace where it is located and the target speech signal The target subspace belongs to two interconnected subspaces, and the signal subspace and the target subspace have a high linear correlation. Therefore, the computing device 240 can calculate the microphone signal. The signal subspace where it is located and the target speech signal Linear correlation of the target subspace to determine the microphone signal The target speech signal exists. The probability of speech presence. For ease of description, we define the signal subspace as... We define the target subspace as .

[0064] like Figure 3 As shown, the method P100 may further include: S140: Based on the microphone signal Determine the microphone signal The signal subspace formed .

[0065] In some embodiments, the signal subspace is determined. This refers to determining the signal subspace. basis matrix In some embodiments, the computing device 240 may base its operation on microphone signals. The sampling covariance matrix is ​​used to determine the signal subspace. basis matrix For ease of description, we define the sampling covariance matrix as follows: . Figure 4 An embodiment of the present specification is shown, providing a method for determining a signal subspace. basis matrix The flowchart. Figure 4 The steps shown correspond to the computing device 240 based on the microphone signal. Sampling covariance matrix Determine the signal subspace basis matrix .like Figure 4 As shown, step S140 may include: S142: Based on microphone signal Determine the microphone signal Sampling covariance matrix .

[0066] Sampling covariance matrix This can be expressed as the following formula: Formula (7) S144: For the sampling covariance matrix Perform eigenvalue decomposition to determine the sampling covariance matrix. Multiple feature vectors.

[0067] Furthermore, regarding the sampling covariance matrix Perform eigenvalue decomposition to obtain the sampling covariance matrix. Multiple eigenvalues. Sampling covariance matrix. It is an M×M dimensional matrix. The sampling covariance matrix. The number of eigenvalues ​​is M, and the M eigenvalues ​​can be expressed by the following formula: Formula (8) in, .

[0068] M eigenvalues ​​correspond to M eigenvectors. For ease of description, we define the M eigenvectors as follows: Among them, there is a one-to-one correspondence between the M eigenvalues ​​and the M eigenvectors.

[0069] S146: Use the matrix formed by at least some of the eigenvectors among the plurality of eigenvectors as the signal subspace. basis matrix .

[0070] In some embodiments, the computing device 240 may use a matrix composed of M feature vectors as the signal subspace. basis matrix In some embodiments, the computing device 240 may use a matrix composed of a subset of the M feature vectors as the signal subspace. basis matrix .because Therefore, the computing device 240 can process q different eigenvalues. The corresponding q feature vectors The matrix formed serves as the signal subspace. basis matrix The signal subspace basis matrix Let M be a column full-rank matrix with a length of q.

[0071] In some embodiments, the computing device 240 may be based on microphone signals. Signal guidance vector Determine the signal subspace basis matrix . Figure 5 Another method for determining the signal subspace is shown according to an embodiment of this specification. basis matrix The flowchart. Figure 5 The steps shown correspond to the computing device 240 based on the microphone signal. Signal guidance vector Determine the signal subspace basis matrix .like Figure 5 As shown, step S140 may include: S148: Based on microphone signal Microphone signals are determined using spatial estimation methods. The azimuth angle of the signal source is used to determine the microphone signal. Signal guidance vector .

[0072] As mentioned earlier, the target guidance vector The noise guidance vector can be pre-stored in the computing device 240. It is unknown. The computing device 240 can use a spatial estimation method to determine the noise-guided vector. Estimation is performed to determine the signal guidance vector. The spatial estimation method may include at least one of the following: Direction of Arrival (DOA) estimation method and spatial spectrum estimation method. The DOA estimation method may include, but is not limited to, linear spectrum estimation (such as the periodogram method), maximum likelihood spectrum estimation, maximum entropy method, MUSIC algorithm (Multiple Signal Classification algorithm), ESPRIT algorithm (Estimating signal parameters via rotational invariance techniques), etc.

[0073] S149: Determine the signal guidance vector For signal subspace basis matrix .

[0074] At this time, the signal subspace basis matrix It is an M×(L+N) dimensional column full-rank matrix.

[0075] like Figure 3 As shown, the method P100 may further include: S160: Determine the target subspace constituted by the target speech signal .

[0076] In some embodiments, the target subspace is determined. This refers to determining the target subspace. basis matrix As mentioned earlier, the target guidance vector can be pre-stored in the computing device. Step S160 may be: the computing device 240 determines a preset target guidance vector corresponding to the target speech signal. For the target subspace basis matrix The target subspace basis matrix It is an M×L dimensional full-rank matrix.

[0077] like Figure 3 As shown, the method P100 may further include: S180: Based on signal subspace With the target subspace Linear correlation to determine microphone signal The probability of the presence of the target speech signal in the speech signal. And output it.

[0078] In some embodiments, the computing device 240 can compute the signal subspace. With the target subspace The volume correlation function is used to calculate the signal subspace. With the target subspace The linear correlation. Figure 6 An embodiment of this specification is shown for calculating the probability of speech presence. The flowchart. Figure 6 This shows step S180. For example... Figure 6 As shown, step S180 may include: S182: Determine the signal subspace With the target subspace The volume-related function.

[0079] signal subspace With the target subspace The volume correlation function can be the signal subspace basis matrix With the target subspace basis matrix Volume correlation function .

[0080] When signal subspace basis matrix Based on microphone signal Sampling covariance matrix For a given M×q-dimensional full-rank column matrix, the volume-related function This can be expressed as the following formula: Formula (9) in, Represents the basis matrix of dimension M×q The volume function in q dimensions. Represents an M×L dimensional basis matrix The volume function in L dimensions. This can be expressed as the following formula: Formula (10) in, for singular values, . . It can be regarded as being caused by The volume of the polyhedron spanned by the column vectors.

[0081] This can be expressed as the following formula: Formula (11) in, for singular values, . . It can be regarded as being caused by The volume of the polyhedron spanned by the column vectors.

[0082] This can be expressed as the following formula: Formula (12) in, . Indicates by Zhang Cheng's target subspace and by Zhang Cheng's signal subspace The main character in the space.

[0083] From formulas (9) and (12), it can be seen that the volume correlation function In reality, there are two subspaces (target subspace) and signal subspace The product of the spatial main sine. Figure 7 A schematic diagram of a spatial protagonist provided according to an embodiment of this specification is shown. Figure 7 As shown, two intersecting planes (plane 1 and plane 2) in three-dimensional space share a common basis vector 3. Therefore, its primary protagonist... The value is 0. The second main character. The angle between their respective basis vectors (basis vector 4 and basis vector 5) is equal to the angle between their other basis vectors. Therefore, the volume correlation function between plane 1 and plane 2 is 0.

[0084] When signal subspace basis matrix Signal guidance vector When, volume correlation function The calculation is similar to the method described above, and will not be repeated here.

[0085] Based on the above analysis, the volume correlation function was found. It has the following property: when the basis matrix and basis matrix The two subspaces spanned separately (signal subspaces) and target subspace When linearly correlated, i.e., in the signal subspace and target subspace If there are other shared basis vectors besides the zero vector, then in There exists at least one spatial protagonist with a sine value of 0, at which point the volume-related function... The value is 0. This occurs when the two subspaces are linearly independent, i.e., the signal subspace... and target subspace When the only common basis vector is the zero vector, All spatial protagonists have sine values ​​greater than 0, at which point the volume-related function... Greater than 0. In some embodiments, when the two subspaces are orthogonal, All spatial protagonists have a sine value of 1, at which point the volume-related function... It reached its maximum value of 1.

[0086] Based on the above analysis, the volume correlation function Provides a pair of two basis matrices and Separately spanned signal subspaces and target subspace A measure of the linear correlation between them. Based on this, the target subspace can be used. Signal subspace The inherent geometric structure between them will affect the microphone signal. The problem of speech activity detection is transformed into a signal subspace. and target subspace Two basis matrices and Volume correlation function question.

[0087] S184: Based on volume correlation function Determine the signal subspace With the target subspace linear correlation coefficient .

[0088] From volume correlation function From the definition and properties, we know that the volume-related function The closer the value is to 1, the larger the signal subspace. With the target subspace The lower the overlap and the closer to orthogonality, the better the signal subspace. With the target subspace The lower the linear correlation coefficient, the stronger the microphone signal. The more likely the signal in the volume correlation function originates from noise, the more these components should be suppressed. Conversely, when the volume correlation function... The closer the value is to 0, the larger the signal subspace. With the target subspace The higher the overlap, the larger the signal subspace. With the target subspace The higher the linear correlation coefficient, the stronger the microphone signal. The more likely the target speech signal is to dominate, the more likely these components are to be preserved. Therefore, the linear correlation coefficient... Volume-related functions Negative correlation. Specifically, step S184 may include one of the following: Determine the volume correlation function If the value is greater than the first threshold 'a', determine the linear correlation coefficient. Determine the volume correlation function If the value is less than the second threshold b, determine the linear correlation coefficient. Determine the volume correlation function Determine the linear correlation coefficient between the first threshold a and the second threshold b. The correlation coefficient is between 0 and 1, and is linearly related. Volume correlation function The negative correlation function. Where the second threshold b is less than the first threshold a. Specifically, the linear correlation coefficient... This can be expressed as the following formula: Formula (13) in, It can be with A linearly negatively correlated function. In some embodiments, This can be expressed as the following formula: Formula (14) S186: The linear correlation coefficient As the probability of the existence of the speech And output it.

[0089] In summary, in the speech activity detection system and method P100 provided in this specification, the computing device 240 can calculate the basis matrix. and Volume correlation function To calculate the signal subspace With the target subspace The linear correlation. Volume correlation function. The higher the value, the more it indicates the signal subspace. With the target subspace The lower the linear correlation, the smaller the signal subspace. With the target subspace The lower the overlap, the weaker the microphone signal. The lower the overlap with the target speech signal, the stronger the microphone signal. The probability of the presence of the target speech signal in the speech. The lower the value, the better. Volume-related function. The lower the value, the more it indicates the signal subspace. With the target subspace The higher the linear correlation, the higher the signal subspace. With the target subspace The higher the overlap, the better, meaning the microphone signal... The higher the overlap with the target speech signal, the stronger the microphone signal. The probability of the presence of the target speech signal in the speech. The higher the accuracy, the better. The speech activity detection system and method P100 provided in this manual can improve the accuracy of speech activity detection and the speech enhancement effect.

[0090] This specification also provides a voice enhancement system. The voice enhancement system can also be applied to electronic device 200. In some embodiments, the voice enhancement system may include a computing device 240. In some embodiments, the voice enhancement system can be applied to computing device 240. That is, the voice enhancement system can run on computing device 240. The voice enhancement system may include hardware devices with data processing capabilities and the necessary programs required to drive the hardware devices. Of course, the voice enhancement system may also be simply a hardware device with data processing capabilities, or simply a program running on the hardware device.

[0091] The speech enhancement system may store data or instructions for performing the speech enhancement methods described herein, and may execute said data and / or instructions. When the speech enhancement system is running on the computing device 240, the speech enhancement system may acquire microphone signals from the microphone array 220 based on the communication connection, and execute the data or instructions for the speech enhancement methods described herein. The speech enhancement methods are described in other parts of this specification. For example, in Figure 8 The speech enhancement method is described in the description.

[0092] When the voice enhancement system is running on the computing device 240, the voice enhancement system is communicatively connected to the microphone array 220. The storage medium 243 may further include at least one instruction set stored in the data storage device for performing voice enhancement calculations on the microphone signals. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the voice enhancement methods provided in this specification. The processor 242 can read the at least one instruction set and execute the voice enhancement methods provided in this specification according to the instructions of the at least one instruction set. The processor 242 can execute all the steps included in the voice enhancement method.

[0093] Figure 8 A flowchart of a speech enhancement method P200 provided according to an embodiment of this specification is shown. The method P200 can enhance the speech of the microphone signal. Specifically, the processor 242 can execute the method P200. Figure 8 As shown, method P200 may include: S220: Acquire the microphone signals output by the M microphones. .

[0094] As described in step S120, it will not be repeated here.

[0095] S240: Based on the aforementioned voice activity detection method P100, determine the microphone signal. The probability of the presence of the target speech signal in the speech. .

[0096] S260: Based on the probability of speech presence Determine microphone signal The corresponding filter coefficient vector .

[0097] Filter coefficient vector It can be an M×1 dimensional vector. (Filter coefficient vector) This can be expressed as the following formula: Formula (15) Wherein, the filter coefficient corresponding to the m-th microphone 222 is . .

[0098] In some embodiments, the computing device 240 can determine the filter coefficient vector based on the MVDR method. Specifically, step S260 may include: based on the probability of speech presence. Determine microphone signal noise covariance matrix ; and based on the MVDR method and noise covariance matrix Determine the filter coefficient vector Noise covariance matrix This can be expressed as the following formula: Formula (16) Filter coefficient vector This can be expressed as the following formula: Formula (17) in, This is the preset target guidance vector.

[0099] In some embodiments, the computing device 240 can determine the filter coefficient vector based on a fixed-beam method. Specifically, step S260 may include: determining the probability of speech presence. As a microphone signal Target microphone signal Corresponding filter coefficients Determine the microphone signal Mid-target microphone signal The filter coefficients for all other microphone signals are 0. Filter coefficient vector Including target microphone signal Corresponding filter coefficients And a vector consisting of the filter coefficients corresponding to the remaining microphone signals. Filter coefficient vector This can be expressed as the following formula: Formula (18) In some embodiments, the target microphone signal This can include microphone signals. The microphone signal with the highest signal-to-noise ratio. In some embodiments, the target microphone signal. This can be the microphone signal closest to the target speech signal source (such as the mouth). In some embodiments, It can be pre-stored in the computing device 240.

[0100] S280: Based on the filter coefficient vector microphone signal The signals are merged to obtain the target audio signal. And output it.

[0101] Target audio signal This can be expressed as the following formula: Formula (21) In summary, the speech activity detection system and method P100 and the speech enhancement system and method P200 provided in this specification are used in a microphone array 220 composed of multiple microphones 222. The speech activity detection system and method P100 and the speech enhancement system and method P200 are capable of acquiring microphone signals collected by the microphone array 220. Microphone signal It can include both noise signals and target speech signals. Target speech signals and noise signals are two unconnected types of signals. The target subspace contains the target speech signal. The noise subspace containing the noise signal and the noise signal are two unconnected subspaces. When the microphone signal... When the target speech signal is not present, the microphone signal It contains only noise signals. At this time, the microphone signal... The signal subspace The target subspace where the target speech signal is located Two unconnected subspaces, signal subspace With the target subspace The linear correlation is low. When the microphone signal When the target speech signal is present, the microphone signal The signal contains both the target speech signal and the noise signal. At this time, the microphone signal... The signal subspace The target subspace where the target speech signal is located Belonging to two interconnected subspaces, the signal subspace With the target subspace The linear correlation is high. Therefore, the speech activity detection method P100 and system provided in this specification can calculate the microphone signal. The signal subspace The target subspace where the target speech signal is located Linear correlation to determine microphone signal The probability of the presence of the target speech signal in the speech. The speech enhancement method P200 and system described above can be based on the probability of speech presence. Calculate the filter coefficient vector Thus, the microphone signal Speech enhancement is performed. In summary, the speech activity detection system and method P100 and the speech enhancement system and method P200 provided in this specification can increase the probability of speech presence. This improves the accuracy of speech calculations, thereby enhancing the speech enhancement effect.

[0102] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for voice activity detection. When executed by a processor, the executable instructions instruct the processor to perform the steps of the voice activity detection method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device (e.g., computing device 240), the program code causes the computing device to perform the voice activity detection steps described herein. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on a computing device. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system (e.g., processor 242). The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a computing device, partially on a computing device, as a standalone software package, partially on a computing device and partially on a remote computing device, or entirely on a remote computing device.

[0103] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0104] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0105] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0106] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and to aid in understanding a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may extract some features as individual embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple sub-embodiments. It is also valid when each sub-embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0107] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. The entire contents used for all purposes, except for any history of prosecution documents associated with it, that may be inconsistent with or conflict with this document, or that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0108] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A method for detecting speech activity, characterized in that, M microphones arranged in a preset array shape for use in headphone products, wherein M is an integer greater than 1 and not greater than 5, including: Obtain the microphone signals output by the M microphones; Based on the microphone signals, determine the signal subspace formed by the microphone signals; Determine the target subspace formed by the target speech signal; Determine the volume correlation function between the signal subspace and the target subspace; Based on the volume correlation function, the linear correlation coefficient between the signal subspace and the target subspace is determined; and The linear correlation coefficient is used as the probability of the presence of the target speech signal in the microphone signal, and the probability of the presence of the speech is output.

2. The speech activity detection method as described in claim 1, characterized in that, Determining the signal subspace formed by the microphone signals based on the microphone signals includes: Based on the microphone signal, determine the sampling covariance matrix of the microphone signal; Perform eigenvalue decomposition on the sampling covariance matrix to determine multiple eigenvectors of the sampling covariance matrix; and The matrix formed by at least some of the eigenvectors among the plurality of eigenvectors shall be used as the basis matrix of the signal subspace.

3. The speech activity detection method as described in claim 1, characterized in that, Determining the signal subspace formed by the microphone signals based on the microphone signals includes: Based on the microphone signal, the azimuth angle of the signal source in the microphone signal is determined by a spatial estimation method, thereby determining the signal guidance vector of the microphone signal. The spatial estimation method includes at least one of a DOA estimation method and a spatial spectrum estimation method. The signal guiding vector is determined to be the basis matrix of the signal subspace.

4. The speech activity detection method as described in claim 1, characterized in that, The determination of the target subspace constituted by the target speech signal includes: The preset target guidance vector corresponding to the target speech signal is determined as the basis matrix of the target subspace.

5. The speech activity detection method as described in claim 1, wherein, The linear correlation coefficient is negatively correlated with the volume correlation function.

6. The speech activity detection method as described in claim 1, characterized in that, The determination of the linear correlation coefficient between the signal subspace and the target subspace based on the volume correlation function includes one of the following cases: If the volume correlation function is determined to be greater than a first threshold, the linear correlation coefficient is determined to be 0. The volume correlation function is determined to be less than a second threshold, and the linear correlation coefficient is determined to be 1, wherein the second threshold is less than the first threshold; as well as The volume correlation function is determined to be between the first threshold and the second threshold, the linear correlation coefficient is determined to be between 0 and 1, and the linear correlation coefficient is a negative correlation function of the volume correlation function.

7. A voice activity detection system, characterized in that, include: At least one storage medium storing at least one instruction set for voice activity detection; as well as At least one processor is communicatively connected to the at least one storage medium. When the voice activity detection system is running, the at least one processor reads the at least one instruction set and implements the voice activity detection method according to any one of claims 1-6.

8. A speech enhancement method, characterized in that, M microphones arranged in a preset array shape for use in headphone products, wherein M is an integer greater than 1 and not greater than 5, including: Obtain the microphone signals output by the M microphones; Based on the voice activity detection method according to any one of claims 1-6, determine the probability of the presence of the target voice signal in the microphone signal; The filter coefficient vector corresponding to the microphone signal is determined based on the probability of the presence of the speech; and The microphone signals are combined based on the filter coefficient vector to obtain the target audio signal and output it.

9. The speech enhancement method as described in claim 8, characterized in that, Determining the filter coefficient vector corresponding to the microphone signal based on the probability of speech presence includes: The noise covariance matrix of the microphone signal is determined based on the probability of the speech presence; and The filter coefficient vector is determined based on the MVDR method and the noise covariance matrix.

10. The speech enhancement method as described in claim 8, characterized in that, Determining the filter coefficient vector corresponding to the microphone signal based on the probability of speech presence includes: The probability of speech presence is used as the filtering coefficient corresponding to the target microphone signal in the microphone signal, wherein the target microphone signal includes the microphone signal with the highest signal-to-noise ratio among the microphone signals; and The filter coefficients for all microphone signals other than the target microphone signal in the microphone signal are determined to be 0. The filter coefficient vector includes a vector consisting of the filter coefficients corresponding to the target microphone signal and the filter coefficients corresponding to the other microphone signals.

11. A speech enhancement system, characterized in that, include: At least one storage medium storing at least one instruction set for speech enhancement; as well as At least one processor is communicatively connected to the at least one storage medium. When the speech enhancement system is running, the at least one processor reads the at least one instruction set and implements the speech enhancement method according to any one of claims 8-10.

Images