Speech recognition method and device, electronic equipment, medium and program product

Abstract

The invention discloses a voice recognition method and device, electronic equipment, a medium and a program product, and belongs to the technical field of audio. The method comprises the following steps: acquiring an identification error corresponding to a secondary path of the audio equipment, wherein the identification error is used for representing whether signals acquired by at least two microphones in the audio equipment are human voice signals or not; under the condition that the identification error is greater than or equal to a first threshold value, a first frequency spectrum ratio is obtained, and the first frequency spectrum ratio is the ratio of the frequency spectrums of the signals collected by the at least two microphones; and when the first spectrum ratio is greater than or equal to a second threshold value, determining that the signals collected by the at least two microphones are sent by the user wearing the audio equipment.

Description

Technical Field

[0001] The present application belongs to the field of audio technology, and specifically relates to a speech recognition method, device, electronic equipment, medium and program product. Background Art

[0002] Currently, electronic devices can perform voice detection in real time. When a user's voice is detected, the device can perform operations corresponding to the semantic information of the voice, such as pausing music or adjusting the volume, thereby realizing control of the electronic device by voice.

[0003] In the related art, an electronic device can collect sound signals through a microphone in the electronic device, and then use a voice activity detection (VAD) algorithm to determine that the sound signal collected by the microphone is a human voice signal; then, based on the semantic information corresponding to the human voice signal, an operation corresponding to the semantic information is performed.

[0004] However, in the above method, when there are other users around the user, the microphone in the electronic device may collect interfering human voice signals emitted by other users, so that the electronic device may determine the interfering human voice signals as the above-mentioned human voice signals, and perform corresponding operations based on the interfering human voice signals, resulting in misrecognition of the electronic device; in this way, the accuracy of the electronic device in recognizing the user's voice is poor. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a speech recognition method, device, electronic device, medium and program product, which can improve the accuracy of electronic equipment in recognizing user voice in the presence of interfering human voice signals around.

[0006] In a first aspect, an embodiment of the present application provides a speech recognition method, which is executed by an audio device, and the speech recognition method includes: obtaining an identification error corresponding to a secondary path of the audio device, and the identification error is used to characterize whether a signal collected by at least two microphones in the audio device is a human voice signal; when the identification error is greater than or equal to a first threshold, obtaining a first spectrum ratio, and the first spectrum ratio is a ratio of the spectrum of the signals collected by the at least two microphones; when the first spectrum ratio is greater than or equal to a second threshold, determining that the signal collected by the at least two microphones is emitted by a user wearing the audio device.

[0007] In a second aspect, an embodiment of the present application provides a speech recognition device, which includes: an acquisition module and an identification module. The acquisition module is used to acquire an identification error corresponding to a secondary path of an audio device, and the identification error is used to characterize whether a signal collected by at least two microphones in the audio device is a human voice signal; and when the identification error is greater than or equal to a first threshold, a first spectrum ratio is acquired, and the first spectrum ratio is the ratio of the spectrum of the signals collected by at least two microphones. The identification module is used to determine that the signals collected by at least two microphones are emitted by a user wearing the audio device when the first spectrum ratio acquired by the acquisition module is greater than or equal to a second threshold.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, an identification error corresponding to a secondary path of an audio device can be obtained, and the identification error is used to characterize whether a signal collected by at least two microphones in the audio device is a human voice signal; then, when the identification error is greater than or equal to a first threshold, a first spectrum ratio is obtained, and the first spectrum ratio is the ratio of the spectrums of the signals collected by the at least two microphones; finally, when the first spectrum ratio is greater than or equal to a second threshold, it is determined that the signals collected by the at least two microphones are emitted by a user wearing the audio device. In the present solution, since it is possible to determine whether the signals collected by at least two microphones are human voice signals through the recognition errors corresponding to the secondary paths, when determining that the signals collected by at least two microphones are human voice signals, it is possible to determine whether the signals collected by at least two microphones are emitted by the user wearing the audio device through the ratio of the frequency spectra of the signals collected by the at least two microphones. It can be understood that when the user wearing the audio device speaks, since the in-ear microphone among the at least two microphones collects the sound signal emitted through bone conduction, the energy of the signal output by the in-ear microphone will be enhanced. At this time, the frequency spectra of the signals output by the at least two microphones will change significantly, and then the ratio of the frequency spectra will also change. Therefore, when the audio device determines that the first frequency spectrum ratio is greater than or equal to the second threshold value, it can determine that the collected sound is emitted by the user wearing the audio device. In this way, misidentification is avoided and the accuracy of recognizing user voice is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic diagram of a speech recognition system provided in an embodiment of the present application;

[0014] Figure 2 This is one of the flow charts of a speech recognition method provided in an embodiment of the present application;

[0015] Figure 3 This is the second flowchart of a speech recognition method provided in an embodiment of the present application;

[0016] Figure 4 It is one of the schematic diagrams of the signal model of the active noise reduction system provided in the embodiment of the present application;

[0017] Figure 5 This is the second schematic diagram of the active noise reduction system signal model provided in the embodiment of the present application;

[0018] Figure 6 This is the third flowchart of a speech recognition method provided in an embodiment of the present application;

[0019] Figure 7 This is the fourth flowchart of a speech recognition method provided in an embodiment of the present application;

[0020] Figure 8is a schematic diagram of frequency spectrum distribution of microphones inside and outside the ear canal when no self-talking occurs, provided in an embodiment of the present application;

[0021] Fig. 9 is a schematic diagram of frequency spectrum distribution of microphones inside and outside the ear canal when self-speaking occurs, provided in an embodiment of the present application;

[0022] Fig.10 is a structural schematic diagram of a speech recognition device provided in an embodiment of the present application;

[0023] Fig.11 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0024] Fig.12 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The terms "at least one (item)", "at least one of" and the like in the specification and claims of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0028] The following is an explanation of the professional terms involved in the speech recognition method, device, electronic device, medium and program product provided in the embodiments of the present application.

[0029] Own voice detection: also known as self-talk detection, the self-talk detection solution is a technical solution for identifying whether the user wearing the smart audio device is in a speaking state. It can also be regarded as a sub-direction of Voice Activity Detection (VAD).

[0030] VAD: also known as voice endpoint detection, voice boundary detection; the purpose of VAD is to identify and eliminate long periods of silence from the sound signal stream, so as to save voice channel resources without reducing service quality.

[0031] Active Noise Cancellation (ANC): refers to the use of a noise reduction system to generate reverse sound waves equal to the external noise, neutralizing the noise and thus achieving the effect of noise reduction.

[0032] Main Path: The acoustic response from the reference microphone to the error microphone, where the reference microphone is the microphone outside the ear in the audio device and the error microphone is the microphone inside the ear in the audio device.

[0033] Secondary Path: Noise cancellation in ANC is acoustic cancellation performed in the ear canal, requiring a speaker to play the reverse noise for cancellation and an error microphone to collect the residual noise. This path is called the secondary path.

[0034] Fast Fourier Transform (FFT): A general term for efficient and fast calculation methods of discrete Fourier transform using computers. The basic idea of ​​FFT is to decompose the original N-point sequence into a series of short sequences in sequence. By making full use of the symmetric and periodic properties of the exponential factors in the discrete Fourier transform calculation formula, the corresponding DFTs of these short sequences are calculated and appropriately combined to achieve the purpose of deleting repeated calculations, reducing multiplication operations and simplifying the structure.

[0035] The speech recognition method, device, electronic device, medium and program product provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0036] The speech recognition method, device, electronic device, medium and program product provided in the embodiments of the present application can be applied in the following scenarios 1 and 2.

[0037] Scenario 1: Users control electronic devices, such as mobile phones, through audio devices to perform voice command interaction scenarios, such as voice wake-up, pausing or playing music, adjusting volume, and other commands and language large model questions and answers.

[0038] Scenario 2: Controls whether the active noise reduction system of the audio device updates parameters.

[0039] In the above scenario 1, taking the audio device as headphones and the electronic device as a mobile phone as an example, when the two microphones of the headphones collect sound signals, the headphones can obtain the recognition error corresponding to the secondary path, and thus determine whether the signal detected by the headphones is a human voice signal based on the recognition error. Then, when the recognition error is greater than or equal to a first threshold, the headphones can determine that the signal detected by the headphones is a human voice signal. Then, when the recognition error is greater than or equal to a first threshold, the headphones can obtain the ratio of the frequency spectra of the signals output by the two microphones of the headphones, and when the first frequency spectrum ratio is greater than or equal to a second threshold, determine that the signal detected by the headphones is emitted by a user wearing the headphones. Finally, the semantic information of the signal detected by the headphones is identified to determine the user's intention, for example, to pause the music, and send a control instruction containing the pause music instruction to the mobile phone to control the mobile phone to execute the pause music operation.

[0040] In the above scenario 2, the audio device is a headset. When the two microphones of the headset collect sound signals, the headset can obtain the recognition error corresponding to the secondary path, and thus determine whether the signal detected by the headset is a human voice signal based on the recognition error. Then, when the recognition error is greater than or equal to a first threshold, the headset can determine that the signal detected by the headset is a human voice signal. Then, the headset can obtain the ratio of the frequency spectra of the signals output by the two microphones of the headset, and when the first frequency spectrum ratio is greater than or equal to a second threshold, determine that the signal detected by the headset is emitted by the user wearing the headset. At this time, the headset can update the ANC noise reduction parameters to suspend ANC noise reduction.

[0041] In the speech recognition method, device, electronic device, medium and program product provided in the embodiments of the present application, since it is possible to judge whether the signals collected by the two microphones are human voice signals through the recognition error corresponding to the secondary path, when determining that the signals collected by at least two microphones are human voice signals, it is possible to judge whether the signals collected by at least two microphones are emitted by the user wearing the audio device through the ratio of the spectra of the signals collected by at least two microphones; it can be understood that when the user wearing the audio device speaks, since the in-ear microphone of at least two microphones collects the sound signal emitted through bone conduction, the energy of the signal output by the in-ear microphone will be enhanced, and at this time, the spectra of the signals output by at least two microphones will change significantly, and then the ratio of the spectra will also change; thus, when the audio device determines that the first spectrum ratio is greater than or equal to the second threshold, it can determine that the collected sound is emitted by the user wearing the audio device. In this way, the phenomenon of misrecognition is avoided and the accuracy of recognizing the user's voice is improved.

[0042] The present application embodiment provides a speech recognition system, such as Figure 1As shown, the speech recognition system 10 includes: a system identification module 11, a VAD detection module 12 and an ear canal internal and external spectrum information analysis module 13.

[0043] Among them, the system can calculate the recognition error through the external ear microphone signal, the internal ear microphone signal and the speaker playback signal, and then determine whether the system recognition relative error is greater than or equal to the first threshold. If it is less than the first threshold, it is determined that self-talk does not occur, and the active noise reduction algorithm is updated in real time.

[0044] If the recognition error is greater than or equal to the first threshold, VAD is further used to detect whether the signal collected by the two microphones is a speech signal. If the output result of the VAD detection is not a speech signal, the signal currently collected by the two microphones is not a speech signal, and the active noise reduction algorithm is updated in real time. If it is greater than the set threshold, it is judged that the current signal is a speech signal with stronger energy, and it is further judged whether the current speech signal is the user speaking himself or the person next to the user is speaking loudly.

[0045] The output result of VAD detection is a voice signal. The system can further analyze the ratio of the signal spectrum inside and outside the ear canal. If the ratio is greater than the set second threshold, it is determined that the current stronger voice signal energy comes from the user's self-speaking, triggering the system's self-speaking state. At this time, the active noise reduction algorithm is stopped, and the signals collected by the microphone outside the ear and the microphone inside the ear are identified, and the semantic information of the signal is obtained, and the electronic device is controlled to perform operations corresponding to the semantic information based on the semantic information.

[0046] The execution subject of the speech recognition method provided in the embodiment of the present application may be a speech recognition device, and the speech recognition device may be an electronic device, or a functional module in an electronic device. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0047] The present application provides a speech recognition method. Figure 2 FIG. 1 is a flow chart of a speech recognition method provided by an embodiment of the present application, which can be applied to an audio device. Figure 2 As shown, the speech recognition method provided in the embodiment of the present application may include the following steps 201 to 203.

[0048] Step 201: The audio device obtains an identification error corresponding to a secondary path of the audio device.

[0049] In the embodiment of the present application, the above-mentioned identification error is used to characterize whether the signals collected by at least two microphones in the audio device are human voice signals.

[0050] In the embodiment of the present application, the above-mentioned audio device may be a headset with an active noise reduction function.

[0051] Optionally, in the embodiment of the present application, the above-mentioned earphones can be any one of the following: in-ear earphones, headphones, ear-hook earphones or bone conduction earphones.

[0052] Exemplarily, the in-ear headphones may be true wireless stereo (TWS) headphones.

[0053] In the embodiment of the present application, the at least two microphones may be an external ear microphone and an internal ear microphone, respectively.

[0054] In an example, when the at least two microphones are 2 microphones, the 2 microphones may be an external-ear microphone and an internal-ear microphone, respectively.

[0055] In another example, when the at least two microphones are three microphones, the three microphones may be two external microphones and one internal microphone; or the three microphones may be one external microphone and two internal microphones. The specific number may be determined according to the requirements and is not limited in the embodiments of the present application.

[0056] In another example, when the at least two microphones are four microphones, the four microphones may be three external ear microphones and one internal ear microphone; or the four microphones may be two external ear microphones and two internal ear microphones. The specific configuration may be determined according to the requirements and is not limited in the embodiments of the present application.

[0057] Optionally, in the embodiment of the present application, the above-mentioned microphone can be any one of the following: an electric microphone, a capacitive microphone, a piezoelectric microphone or a semiconductor microphone, etc., which can be determined according to actual usage requirements and is not limited in the embodiment of the present application.

[0058] In the embodiment of the present application, the secondary path is the sound propagation path between the player and the in-ear microphone in the audio device.

[0059] Exemplarily, the player may be a speaker.

[0060] For example, the speaker may be any of the following: a super tweeter, a tweeter, a midrange speaker, a mid-bass speaker, a woofer, or a subwoofer, etc. The specific speaker may be determined according to actual use requirements, and the present application embodiment does not limit the speaker.

[0061] Step 202: When the recognition error is greater than or equal to a first threshold, the audio device obtains a first frequency spectrum ratio.

[0062] In the embodiment of the present application, the first spectrum ratio is a ratio of spectrums of signals collected by the at least two microphones.

[0063] In the embodiment of the present application, the first spectrum ratio can effectively distinguish whether the signal is emitted by the user wearing the audio device or by other users around the user.

[0064] Optionally, in the embodiment of the present application, the first threshold may be a user-defined threshold. For example, the first threshold may be any one of the following: 0.01, 0.02, 0.04, 0.08, 0.1 or 0.2, etc. The specific value may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0065] In the embodiment of the present application, after obtaining the recognition error, the audio device may compare the recognition error with the first threshold value, so as to determine whether the recognition error is greater than or equal to the first threshold value.

[0066] Exemplarily, when the recognition error obtained by the audio device is 0.5 and the first threshold is 0.01, the audio device compares the recognition error 0.5 with the first threshold 0.01 and can determine that the recognition error is greater than the first threshold, so that the audio device can continue to obtain the first spectrum ratio.

[0067] Optionally, in the embodiment of the present application, when the recognition error obtained by the audio device is less than the first threshold, the audio device may continue to perform the active noise reduction function.

[0068] Step 203: When the first spectrum ratio is greater than or equal to the second threshold, the audio device determines that the signals collected by the at least two microphones are emitted by the user wearing the audio device.

[0069] Optionally, in the embodiment of the present application, the second threshold value may be a user-defined threshold value. For example, the second threshold value may be any one of the following: -6, -4, -2, -1, 0, 1, 2, etc. The specific value may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0070] In the embodiment of the present application, after obtaining the first spectrum ratio, the audio device may compare the first spectrum ratio with the second threshold value, so as to determine whether the first spectrum ratio is greater than or equal to the first threshold value.

[0071] Exemplarily, when the first spectrum ratio obtained by the audio device is 1 and the second threshold is 0, the audio device compares the first spectrum ratio 1 with the second threshold 0, and can determine that the above-mentioned identification error is greater than the first threshold. At this time, the audio device can determine that the signals collected by at least two microphones are emitted by the user wearing the audio device.

[0072] Optionally, in an embodiment of the present application, after determining that the signal collected by at least two microphones is emitted by a user wearing the audio device, the audio device can generate a control instruction based on the signal to control an electronic device connected to the audio device to perform an operation corresponding to the control instruction.

[0073] Optionally, in an embodiment of the present application, the audio device can identify the signals collected by two microphones through a first algorithm to obtain semantic information corresponding to the signal, and then generate a control instruction based on the semantic information, and control the electronic device to perform the operation corresponding to the control instruction based on the control instruction.

[0074] Optionally, in an embodiment of the present application, the audio device may be connected to the electronic device via Bluetooth; or the audio device may be connected to the electronic device via a Wireless Fidelity (WiFi) network.

[0075] Optionally, in the embodiment of the present application, the first algorithm may be any one of the following: an artificial intelligence (AI) algorithm, a neural network algorithm, an automatic speech recognition algorithm (ASR), etc. The specific algorithm may be determined according to actual use requirements, and the embodiment of the present application does not limit it.

[0076] Exemplarily, the audio device can input the above signal into the ASR algorithm, which splits the signal according to audio frames, and converts the split audio frames into multi-dimensional vector information according to the characteristics of the human ear, and then combines the multi-dimensional vector information into phonemes, and finally combines the phonemes into words and concatenates them into sentences, thereby obtaining the semantic information corresponding to the signal.

[0077] Optionally, in an embodiment of the present application, after the audio device generates a control instruction according to the semantic information, the audio device may send the control instruction to the electronic device to control the electronic device to perform an operation corresponding to the control instruction.

[0078] For example, taking the audio device as headphones and the electronic device as a mobile phone, and taking the voice corresponding to the above signal as "pause music" as an example, after the headphones receive the "pause music" voice, they can obtain semantic information through the above ASR algorithm, that is, perform the "pause" operation on the "music application"; then, the headphones can directly generate a control instruction "pause the song played in the music application" based on the semantic information, and send the control instruction to the mobile phone via Bluetooth, so that the mobile phone can pause the song played in the music application after receiving the control instruction.

[0079] As another example, taking the voice corresponding to the above signal as "Find the registration time for certificate A", after the headset receives the voice "Find the registration time for certificate A", it can obtain semantic information through the above ASR algorithm, that is, execute the "Find the registration time for certificate A" operation; then, the headset can directly generate a control instruction "Find the registration time for certificate A" based on the semantic information, and send the control instruction to the mobile phone via Bluetooth, so that after receiving the control instruction, the mobile phone can find the registration time for certificate A through the browser.

[0080] Optionally, in an embodiment of the present application, after determining that the signals collected by at least two microphones are emitted by a user wearing the audio device, the audio device can update the ANC noise reduction parameters to pause the ANC noise reduction so that the user can clearly hear his or her own voice.

[0081] In the speech recognition method provided in the embodiment of the present application, an identification error corresponding to the secondary path of the audio device can be obtained, and the identification error is used to characterize whether the signals collected by at least two microphones in the audio device are human voice signals; then, when the identification error is greater than or equal to a first threshold, a first spectrum ratio is obtained, and the first spectrum ratio is the ratio of the spectrums of the signals collected by the at least two microphones; finally, when the first spectrum ratio is greater than or equal to a second threshold, it is determined that the signals collected by the at least two microphones are emitted by a user wearing the audio device. In this solution, since it is possible to judge whether the signals collected by at least two microphones are human voice signals through the recognition error corresponding to the secondary path, when determining that the signals collected by at least two microphones are human voice signals, it is possible to judge whether the signals collected by at least two microphones are emitted by the user wearing the audio device through the ratio of the spectra of the signals collected by at least two microphones; it can be understood that when the user wearing the audio device speaks, since the in-ear microphone among the at least two microphones collects the sound signal emitted through bone conduction, the energy of the signal output by the in-ear microphone will be enhanced, and at this time, the spectra of the signals output by at least two microphones will change significantly, and then the ratio of the spectra will also change; thus, when the audio device determines that the first spectrum ratio is greater than or equal to the second threshold, it can determine that the collected sound is emitted by the user wearing the audio device. In this way, the phenomenon of misidentification is avoided and the accuracy of recognizing the user's voice is improved.

[0082] Optionally, in the embodiment of the present application, combined with Figure 2 ,like Figure 3 As shown, before the above step 201, the speech recognition method provided in the embodiment of the present application further includes the following step 301, and the above step 201 can be specifically implemented by the following steps 201a and 201b.

[0083] Step 301: An audio device acquires at least two first signals, and performs fusion processing on the at least two first signals to obtain a second signal.

[0084] In the embodiment of the present application, the at least two first signals are signals collected by at least two microphones, and the at least two first signals correspond one-to-one to the at least two microphones.

[0085] In the embodiment of the present application, the audio device may collect the first signal through each of the at least two microphones, thereby obtaining the at least two first signals mentioned above.

[0086] Optionally, in the embodiment of the present application, one of the at least two microphones may be an external ear microphone or an internal ear microphone.

[0087] Optionally, in the embodiment of the present application, another microphone among the at least two microphones may be an external ear microphone or an internal ear microphone.

[0088] Exemplarily, when the at least two microphones are two microphones, when one of the two microphones is an outside-ear microphone, the other of the two microphones is an inside-ear microphone; or, when one of the two microphones is an inside-ear microphone, the other of the two microphones is an outside-ear microphone.

[0089] Optionally, in the embodiment of the present application, after obtaining at least two signals respectively collected by at least two microphones, the audio device may filter each of the at least two signals respectively, thereby obtaining at least two first signals respectively. In other words, each of the at least two first signals may be a signal that has been filtered.

[0090] Exemplarily, after obtaining at least two signals, the audio device may respectively perform filtering processing on each of the at least two signals through a filter, so as to obtain at least two first signals.

[0091] Optionally, in an embodiment of the present application, the audio device may directly perform superposition processing on at least two first signals to obtain a second signal; or, the audio device may perform convolution processing on at least two first signals to obtain the second signal.

[0092] Step 201a: The audio device inputs the second signal into the secondary path, processes the second signal through the secondary path, and outputs a third signal.

[0093] In the embodiment of the present application, after the audio device inputs the second signal into the secondary path, the secondary path may perform noise reduction processing on the second signal to obtain the second signal after noise reduction processing, that is, the third signal mentioned above.

[0094] It should be noted that the above-mentioned secondary path may include an unconstrained frequency domain adaptive algorithm.

[0095] Exemplarily, after the audio device inputs the second signal into the unconstrained frequency domain adaptive algorithm, the unconstrained frequency domain adaptive algorithm can obtain the inverse wave signal corresponding to the second signal, and then the second signal is fused with the inverse wave signal to obtain the above-mentioned third signal.

[0096] It should be noted that the above-mentioned reverse wave signal is a signal with a phase opposite to that of the second signal.

[0097] Step 201b: The audio device determines an identification error based on the frequency spectrum of the second signal and the frequency spectrum of the third signal.

[0098] In an embodiment of the present application, the audio device can perform FFT processing on the second signal and the third signal respectively, so as to obtain the second signal and the third signal in the frequency domain respectively, and then obtain the spectrum of the second signal and the spectrum of the third signal respectively according to the signal energy sizes of the second signal and the third signal in the frequency domain.

[0099] In the embodiment of the present application, the audio device may determine the recognition error according to the ratio between the signal energy included in the frequency spectrum of the second signal and the signal energy included in the frequency spectrum of the third signal.

[0100] In the embodiment of the present application, the audio device can determine whether the self-talking state occurs through the recognition error of the secondary path. When self-talking occurs, the bone conduction sound is picked up by the in-ear microphone through the ear canal. The principle that the recognition error of the secondary path increases can effectively distinguish whether the audio device has collected the sound signal.

[0101] It should be noted that the above-mentioned self-speaking state is a state in which a microphone in the audio device collects a sound signal.

[0102] Exemplarily, the following explains that when self-talk occurs, the principle that the active noise reduction system signal model increases the recognition error of the secondary path can effectively distinguish whether the audio device collects the sound signal.

[0103] like Figure 4 As shown, when the user wearing the audio device speaks without speaking, the external ear microphone collects a first sound signal. Figure 4 denoted by x(n), and the first sound signal is input into the primary transmission path respectively, Figure 4 In the figure, P(Z) represents the secondary path system identification module. Figure 4 denoted by sys_iden, and the feedforward ANC control filter, Figure 4where Kf(Z) is used; then the feedforward ANC control filter can filter the first sound signal to obtain a filtered first sound signal; then, the second sound signal collected by the in-ear microphone is Figure 4 denoted by e(n) in the figure, which are input into the secondary path system identification module and fed back into the ANC control filter. Figure 4 In the example, Kb(Z) is used to represent the second sound signal. The second sound signal is filtered by the feedback ANC control filter to obtain a filtered second sound signal. The filtered first sound signal and the filtered second sound signal are then fused and input into the secondary path system identification module and the secondary transfer path, respectively. Figure 4 In the figure, S(Z) is used to represent the secondary path system identification module; the secondary path system identification module is used to calculate the above-mentioned identification error. Finally, the audio device can fuse the first sound signal output by the primary transmission path and the sound signal output by the secondary transmission path to obtain a fused signal.

[0104] According to the above Figure 4 It can be seen that the secondary path output can be expressed by the following formula 1.

[0105] o(n)=e(n)-x(n)*P(n)+v(n) (1)

[0106] Where 0(n) is the secondary path output, e(n) is the sound signal collected by the in-ear microphone, P(n) is the signal output through the primary transfer path P(Z), and v(n) is the system measurement noise.

[0107] Combination Figure 4 ,like Figure 5 As shown in FIG. 1 , when a user wearing an audio device speaks to himself, the external microphone will collect the sound signal emitted by the user. Figure 5 In the figure, x(n) is used to represent the sound signal emitted by the user's ear bones. Figure 5 Then, the sound from the bones in the ear will travel through the air. Figure 5 In the figure, it is represented by Air_tf(z), which is collected by the microphone outside the ear. At this time, the signal collected by the audio device is the superposition of the x(n) signal and the s(n) signal. Then, the audio device inputs the superimposed signal into the primary transmission path respectively. Figure 5 In the figure, P(Z) represents the secondary path system identification module. Figure 5 denoted by sys_iden, and the feed-forward ANC control filter, Figure 5In the figure, Kf(Z) is used to represent the superimposed signal, which is referred to as the first sound signal below, and then the feedforward ANC control filter outputs the filtered first sound signal after filtering. Then, the audio device inputs the sound signal emitted by the user collected by the in-ear microphone, which is referred to as the second sound signal below, into the feedback ANC control filter, respectively. Figure 5 In the secondary path system identification module, Kb(Z) is used to represent the filtered second sound signal by feeding back the ANC control filter; then, the audio device can fuse the filtered first sound signal and the filtered second sound signal, and input them into the secondary path system identification module and the secondary transmission path respectively. Figure 5 In the figure, S(Z) is used to represent the secondary path. The identification error corresponding to the secondary path can be obtained through the secondary path system identification module. Then, the sound signal output by the secondary transfer path, the sound signal output by the primary transfer path and the bone conduction module in the audio device are combined. Figure 5 The in-ear microphone, represented by Bone_tf(Z), collects the sound signal emitted by the user's in-ear bones for fusion.

[0108] According to the above Figure 5 It can be understood that when the user wearing the earphones speaks to himself, the signal transmission path of the entire recognition system changes, and the occurrence of self-speaking leads to the introduction of two acoustic propagation paths in the system: a. The self-speaking sound source is transmitted to the ear canal through the bone conduction and picked up by the internal microphone; b. The self-speaking sound source is transmitted to the outside of the ear canal through the air and picked up by the external microphone of the earphone. The output signal through the secondary path can be expressed by the following formula 2.

[0109] o(n)=e(n)-x(n)*P(n)+s(n)*Bone_tf(n)+v(n) (2)

[0110] Among them, 0(n) is the secondary path output, e(n) is the sound signal collected by the in-ear microphone, P(n) is the signal output through the primary transfer path P(Z), v(n) is the system measurement noise, s(n) is the signal output through the secondary transfer path S(Z), and Bone_tf(n) is the signal output through Bone_tf(Z).

[0111] By comparing the above formula 1 and formula 2, it can be seen that before and after the self-talk occurs, the measurement error of the system identification caused by the self-talk becomes larger, and then by setting a suitable threshold, it is possible to effectively distinguish whether the current state has self-talk through the system identification error.

[0112] Optionally, in the embodiment of the present application, the above step 201b can be specifically implemented through the following steps 401 to 405.

[0113] Step 401: The audio device performs fast Fourier transform on the second signal to obtain a transformed second signal.

[0114] In the embodiment of the present application, the audio device may convert the second signal in the time domain into a second signal in the frequency domain through FFT.

[0115] Exemplarily, the audio device may convert the waveform of the second signal in the time domain into a sine wave through FFT. The sine wave is a description of the frequency domain, thereby obtaining the second signal in the frequency domain.

[0116] Step 402: The audio device obtains a fourth signal in the first frequency band from the transformed second signal.

[0117] Optionally, in an embodiment of the present application, the first frequency band is a frequency band preset by a user.

[0118] It should be noted that the first frequency band is the frequency band that can best show the difference between the fourth signal and the fifth signal.

[0119] For example, the frequency band included in the above-mentioned assumed spectrum is [0, 1000], and the first frequency band may be [200, 300], [150, 400] or [300, 500], etc. The specific frequency band may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0120] In the embodiment of the present application, the audio device may obtain the fourth signal by sampling from the spectrum of the transformed second signal according to the first frequency band.

[0121] Optionally, in the embodiment of the present application, after obtaining the fourth signal, the audio device may perform conjugate processing on the fourth signal to obtain a conjugated fourth signal.

[0122] In the embodiment of the present application, the audio device performs a phase inversion process on the fourth signal, namely the above-mentioned conjugation process, to obtain a conjugated fourth signal. In other words, the conjugated fourth signal is in opposite phase to the fourth signal.

[0123] It should be noted that, since the fourth signal after FFT transformation is in complex form, the audio device can obtain the signal energy value of the fourth signal corresponding to the first frequency band after conjugating the fourth signal, that is, the above-mentioned conjugated fourth signal.

[0124] Step 403: The audio device performs fast Fourier transform on the third signal to obtain a transformed third signal.

[0125] In the embodiment of the present application, the audio device may convert the third signal in the time domain into a third signal in the frequency domain through FFT.

[0126] Exemplarily, the audio device may convert the waveform of the third signal in the time domain into a sine wave through FFT. The sine wave is a description of the frequency domain, thereby obtaining the third signal in the frequency domain.

[0127] Step 404: The audio device obtains a fifth signal in the first frequency band from the transformed third signal.

[0128] In the embodiment of the present application, the audio device can obtain the above-mentioned fifth signal by sampling from the spectrum of the transformed third signal according to the first frequency band.

[0129] Optionally, in the embodiment of the present application, after obtaining the fifth signal, the audio device may perform conjugate processing on the fifth signal to obtain a conjugated fifth signal.

[0130] In the embodiment of the present application, the audio device performs a phase inversion process on the fifth signal, namely the above-mentioned conjugation process, to obtain a conjugated fifth signal. In other words, the conjugated fifth signal is in opposite phase to the fifth signal.

[0131] It should be noted that the execution timing of the above steps 403 and 404 can be performed in parallel; or, step 403 is performed first, and then step 404; or, step 404 is performed first, and then step 403. The specific timing can be determined according to the use requirements, and the embodiment of the present application does not limit it. For example, Figure 4 As shown, the audio device may first execute the above step 403 and then execute the above step 404.

[0132] Step 405: The audio device calculates an identification error based on the fourth signal, the fifth signal, and the number of frequency points included in the first frequency band.

[0133] Exemplarily, the audio device may perform a division operation on the magnitude of the spectrum of the fourth signal and the magnitude of the spectrum of the fifth signal, and then divide the result by the number of frequency points included in the first frequency band, thereby calculating the recognition error.

[0134] Optionally, in an embodiment of the present application, the audio device may calculate an identification error based on the fourth signal, the conjugated fourth signal, the fifth signal, the conjugated fifth signal, and the number of frequency points included in the first frequency band.

[0135] In the embodiment of the present application, the audio device may multiply the fourth signal by the conjugated fourth signal to obtain a first energy value, and multiply the fifth signal by the conjugated fifth signal to obtain a second energy value; then divide the first energy value by the second energy value to obtain a third energy value, and finally divide the third energy value by the number of frequency points included in the first frequency band to obtain the above-mentioned identification error. This can be specifically achieved by the following formula 3.

[0136]

[0137] Among them, re_err is the identification error, l1, l m is the first frequency band, E(k) is the fourth signal, E * (k) is the fourth signal after conjugation, X(k) is the fifth signal, X * (k) is the fifth signal after conjugation, and m is the number of frequency points included in the first frequency band.

[0138] In the embodiment of the present application, the audio device calculates the recognition error through the fourth signal, the conjugated fourth signal, the fifth signal, the conjugated fifth signal, and the number of frequency points included in the first frequency band. It can be understood that a larger recognition error indicates that the microphone has collected the sound signal, so the audio device can determine whether the microphone has collected the sound signal by the size of the recognition error, thereby improving the accuracy of the audio device in identifying the sound signal.

[0139] Optionally, in the embodiment of the present application, combined with Figure 2 ,like Figure 6 As shown, before "obtaining the first spectrum ratio" in the above step 202, the speech recognition method provided in the embodiment of the present application also includes the following step 501, and the "obtaining the first spectrum ratio" in the above step 202 can be specifically implemented by the following step 202a.

[0140] Step 501: The audio device performs signal recognition on at least two first signals respectively collected by at least two microphones to obtain at least two pieces of feature information.

[0141] In the embodiment of the present application, the at least two characteristic information mentioned above correspond one-to-one to the at least two first signals.

[0142] In the embodiment of the present application, the audio device may input at least two first signals into a VAD algorithm respectively to obtain at least two pieces of feature information.

[0143] Exemplarily, taking at least two first signals as an example, the audio device can input the first first signal of the at least two first signals into the VAD algorithm, and the VAD algorithm can extract feature information corresponding to the first first signal from the first first signal through convolution, and input the second first signal of the at least two first signals into the VAD algorithm, and extract feature information corresponding to the second first signal from the second first signal through convolution.

[0144] Optionally, in an embodiment of the present application, the above-mentioned characteristic information may include at least one of the following: signal energy, spectrum, cepstrum and harmonic wave, etc.

[0145] Step 202a: When at least two pieces of characteristic information match preset characteristic information, obtain a first spectrum ratio.

[0146] Optionally, in the embodiment of the present application, when at least two pieces of characteristic information are both signal energy, when the signal energy matches a preset energy threshold, the audio device may further determine that the first signal is a human voice signal.

[0147] Exemplarily, the signal energy matches the preset energy threshold value, which means that the signal energy is the same as or greater than the preset energy threshold value.

[0148] Optionally, in the embodiment of the present application, when at least two pieces of characteristic information are both frequency spectra, and when the frequency spectrum of the first signal matches the frequency spectrum of a preset human voice signal, the audio device may further determine that the first signal is a human voice signal.

[0149] Exemplarily, the spectrum of the first signal matches the spectrum of the preset human voice signal when the difference between the spectrum of the first signal and the spectrum of the preset human voice signal is less than 5%. In this case, it can be considered that the spectrum of the first signal matches the spectrum of the preset human voice signal.

[0150] In an embodiment of the present application, the audio device can further detect at least two first signals respectively collected by at least two microphones through a VAD algorithm to increase the probability that the collected signal is a human voice signal, thereby improving the probability that the at least two first signals are human voice signals and improving the accuracy of the audio device in determining the human voice signal.

[0151] Optionally, in the embodiment of the present application, combined with Figure 2 ,like Figure 7 As shown, the above step 202 can be specifically implemented through the following steps 202b to 202f.

[0152] Step 202b: When the recognition error is greater than or equal to the first threshold, the audio device performs a fast Fourier transform on the sixth signal collected by the first microphone of the at least two microphones to obtain a transformed sixth signal.

[0153] Optionally, in the embodiment of the present application, the first microphone may be one or more, which may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0154] It can be understood that, in the case where there are multiple first microphones, the sixth signal may be the sum of signals collected by the multiple first microphones.

[0155] In the embodiment of the present application, the audio device may convert the sixth signal in the time domain into a sixth signal in the frequency domain through FFT.

[0156] Exemplarily, the audio device may convert the waveform of the sixth signal in the time domain into a sine wave through FFT. The sine wave is a description of the frequency domain, thereby obtaining the sixth signal in the frequency domain.

[0157] Step 202c: The audio device obtains a seventh signal in the first frequency band from the transformed sixth signal.

[0158] In the embodiment of the present application, the audio device can obtain the above-mentioned seventh signal by sampling from the frequency spectrum of the transformed sixth signal according to the first frequency band.

[0159] Optionally, in the embodiment of the present application, after obtaining the seventh signal, the audio device may perform conjugate processing on the seventh signal to obtain a conjugated seventh signal.

[0160] In the embodiment of the present application, the audio device performs a phase inversion process on the seventh signal, namely the above-mentioned conjugation process, to obtain a conjugated seventh signal. In other words, the conjugated seventh signal is in opposite phase to the seventh signal.

[0161] Step 202d: The audio device performs a fast Fourier transform on the eighth signal collected by the second microphone of the at least two microphones to obtain a transformed eighth signal.

[0162] Optionally, in the embodiment of the present application, the second microphone may be one or more, which may be determined according to actual use requirements, and the embodiment of the present application does not limit this.

[0163] It can be understood that, in the case where there are multiple second microphones, the eighth signal may be the sum of signals collected by the multiple second microphones.

[0164] In the embodiment of the present application, the audio device can convert the eighth signal in the time domain into the eighth signal in the frequency domain through FFT.

[0165] Exemplarily, the audio device may convert the waveform of the eighth signal in the time domain into a sine wave through FFT. The sine wave is a description of the frequency domain, thereby obtaining the eighth signal in the frequency domain.

[0166] Step 202e: The audio device obtains a ninth signal in the first frequency band from the transformed eighth signal.

[0167] In the embodiment of the present application, the audio device can obtain the above-mentioned ninth signal by sampling from the frequency spectrum of the transformed eighth signal according to the first frequency band.

[0168] Optionally, in the embodiment of the present application, after obtaining the ninth signal, the audio device may perform conjugate processing on the ninth signal to obtain a conjugated ninth signal.

[0169] In the embodiment of the present application, the audio device can perform a phase inversion process on the ninth signal, that is, the above-mentioned conjugation process, to obtain a conjugated ninth signal. In other words, the conjugated ninth signal is in opposite phase to the ninth signal.

[0170] It should be noted that the execution timing of the above step 202d and step 202e can be performed in parallel; or, step 202d is performed first, and then step 202e; or, step 202e is performed first, and then step 202d is performed. The specific timing can be determined according to the use requirements, and the embodiment of the present application does not limit it. For example, Figure 6 As shown, the audio device may first execute the above step 202d, and then execute the above step 202e.

[0171] Step 202f: The audio device calculates a first spectrum ratio based on the seventh signal, the ninth signal, and the number of frequency points included in the first frequency band.

[0172] Exemplarily, the audio device may perform a division operation on the amplitude of the seventh signal and the amplitude of the ninth signal, and then divide the result by the number of frequency points included in the first frequency band, thereby calculating the first frequency spectrum ratio.

[0173] Optionally, in an embodiment of the present application, the audio device may calculate a first spectrum ratio based on the seventh signal, the conjugated seventh signal, the ninth signal, the conjugated ninth signal, and the number of frequency points included in the first frequency band.

[0174] In the embodiment of the present application, the audio device may multiply the seventh signal by the conjugated seventh signal to obtain a fourth energy value, and multiply the ninth signal by the conjugated ninth signal to obtain a fifth energy value; then divide the fourth energy value by the fifth energy value to obtain a sixth energy value, and finally perform logarithmic processing on the sixth value to obtain a seventh energy value, and divide the seventh energy value by the number of frequency points included in the first frequency band to obtain the above-mentioned first spectrum ratio. Specifically, the following formula can be used

[0175] Formula 4 is used to implement it.

[0176]

[0177] Among them, spec_ratio is the first spectrum ratio, FB(K) is the seventh signal, FB * (K) is the seventh signal after conjugation, FF(K) is the ninth signal, FF * (K) is the ninth signal after conjugation, m is the number of frequency points included in the first frequency band, i1 to i m The first frequency band.

[0178] It can be understood that when self-talk does not occur, due to the physical isolation of the audio equipment and the cancellation effect of the active noise reduction field, the intensity of the microphone outside the ear canal in the first frequency band is much greater than the signal of the microphone inside the ear canal. After self-talk occurs, due to the change in the signal path, the energy of the microphone inside the ear canal is enhanced in the first frequency band, and the energy ratio of the two changes significantly. Figure 8 The frequency spectrum distribution of microphones inside and outside the ear canal when no self-talk occurs is shown. Fig. 9 The spectrum distribution of microphones inside and outside the ear canal when self-talk occurs is shown.

[0179] It should be noted that Figure 8 and Fig. 9 The fb-mic is an external microphone and the ff-mic is an internal microphone.

[0180] In the embodiment of the present application, since the in-ear microphone of the two microphones collects the sound signal emitted through bone conduction, the energy of the signal output by the in-ear microphone will be enhanced, and the spectrum of the signal output by the two microphones will change significantly, and then the ratio of the spectrum will also increase; thus, when the audio device determines that the first spectrum ratio is greater than or equal to the second threshold, it can determine that the collected sound is emitted by the user wearing the audio device. In this way, the phenomenon of misrecognition is avoided and the accuracy of recognizing the user's voice is improved.

[0181] The above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0182] It should be noted that the speech recognition method provided in the embodiment of the present application can be executed by a speech recognition device. In the embodiment of the present application, the speech recognition device provided in the embodiment of the present application is described by taking the speech recognition method executed by the speech recognition device as an example.

[0183] Fig.10 FIG. 1 shows a possible structural diagram of a speech recognition device involved in an embodiment of the present application. Fig.10 As shown, the speech recognition device 70 may include: an acquisition module 71 and a recognition module 72 .

[0184] The acquisition module 71 is used to acquire the recognition error corresponding to the secondary path of the audio device, and the recognition error is used to characterize whether the signal collected by at least two microphones in the audio device is a human voice signal; and when the recognition error is greater than or equal to the first threshold, the first spectrum ratio is acquired, and the first spectrum ratio is the ratio of the spectrum of the signal collected by the at least two microphones. The identification module 72 is used to determine that the signal collected by the at least two microphones is emitted by the user wearing the audio device when the first spectrum ratio acquired by the acquisition module is greater than or equal to the second threshold.

[0185] In a possible implementation, the speech recognition device further includes: an input module and an output module. The acquisition module 71 is also used to acquire at least two first signals before acquiring the recognition error corresponding to the secondary path of the audio device, and to fuse the at least two first signals to obtain a second signal, wherein the at least two first signals are signals collected by the at least two microphones, and the at least two first signals correspond one-to-one to the at least two microphones. The input module is used to input the second signal acquired by the acquisition module into the secondary path. The output module is used to process the second signal through the secondary path and output a third signal. The acquisition module 71 is specifically used to determine the recognition error based on the spectrum of the second signal and the spectrum of the third signal.

[0186] In a possible implementation, the acquisition module 71 is specifically used to perform a fast Fourier transform on the second signal to obtain a transformed second signal; obtain a fourth signal in the first frequency band from the transformed second signal; perform a fast Fourier transform on the third signal to obtain a transformed third signal; obtain a fifth signal in the first frequency band from the transformed third signal; and calculate an identification error based on the fourth signal, the fifth signal, and the number of frequency points contained in the first frequency band.

[0187] In a possible implementation, the acquisition module 71 is specifically used to perform a fast Fourier transform on a sixth signal collected by a first microphone among at least two microphones to obtain a transformed sixth signal; obtain a seventh signal in a first frequency band from the transformed sixth signal; perform a fast Fourier transform on an eighth signal collected by a second microphone among at least two microphones to obtain a transformed eighth signal; obtain a ninth signal in the first frequency band from the transformed eighth signal; and calculate a first spectrum ratio based on the seventh signal, the ninth signal, and the number of frequency points included in the first frequency band.

[0188] In a possible implementation, the identification module 72 is further configured to perform signal identification on at least two first signals respectively collected by at least two microphones to obtain at least two feature information before the acquisition module 71 acquires the first spectrum ratio, and the at least two feature information correspond to the at least two first signals one by one. The acquisition module 71 is specifically configured to acquire the first spectrum ratio when both of the at least two feature information match the preset feature information.

[0189] The embodiment of the present application provides a speech recognition device, which can judge whether the signals collected by at least two microphones are human voice signals through the recognition error corresponding to the secondary path. Therefore, when it is determined that the signals collected by at least two microphones are human voice signals, it can judge whether the signals collected by at least two microphones are emitted by the user wearing the audio device through the ratio of the spectrum of the signals collected by at least two microphones. It can be understood that when the user wearing the speech recognition device speaks, since the in-ear microphone of at least two microphones collects the sound signal emitted through bone conduction, the energy of the signal output by the in-ear microphone will be enhanced, and at this time, the spectrum of the signal output by at least two microphones will change significantly, and then the ratio of the spectrum will also change. Therefore, when the speech recognition device determines that the first spectrum ratio is greater than or equal to the second threshold, it can determine that the collected sound is emitted by the user wearing the audio device. In this way, the phenomenon of misrecognition is avoided and the accuracy of recognizing the user's voice is improved.

[0190] The speech recognition device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0191] The speech recognition device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0192] The speech recognition device provided in the embodiment of the present application can implement each process implemented in the above embodiment, and to avoid repetition, it will not be repeated here.

[0193] Alternatively, if Fig.11 As shown, an embodiment of the present application also provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned speech recognition method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0194] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0195] Fig.12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0196] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0197] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.12 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0198] Among them, the processor 110 is used to obtain an identification error corresponding to a secondary path of the audio device, and the identification error is used to characterize whether the signals collected by at least two microphones in the audio device are human voice signals; when the identification error is greater than or equal to a first threshold, a first spectrum ratio is obtained, and the first spectrum ratio is the ratio of the spectrums of the signals collected by the at least two microphones; when the first spectrum ratio is greater than or equal to a second threshold, it is determined that the signals collected by the at least two microphones are emitted by a user wearing the audio device.

[0199] Optionally, in an embodiment of the present application, the processor 110 is further used to obtain at least two first signals before obtaining the recognition error corresponding to the secondary path of the audio device, and to perform fusion processing on the at least two first signals to obtain a second signal, wherein the at least two first signals are signals collected by at least two microphones, and the at least two first signals correspond one to one to the at least two microphones. The processor 110 is specifically used to input the second signal into the secondary path, process the second signal through the secondary path to output a third signal, and determine the recognition error based on the spectrum of the second signal and the spectrum of the third signal.

[0200] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform a fast Fourier transform on the second signal to obtain a transformed second signal; obtain a fourth signal in the first frequency band from the transformed second signal; perform a fast Fourier transform on the third signal to obtain a transformed third signal; obtain a fifth signal in the first frequency band from the transformed third signal; and calculate an identification error based on the fourth signal, the fifth signal, and the number of frequency points contained in the first frequency band.

[0201] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform a fast Fourier transform on a sixth signal collected by a first microphone among at least two microphones to obtain a transformed sixth signal; obtain a seventh signal in the first frequency band from the transformed sixth signal; perform a fast Fourier transform on an eighth signal collected by a second microphone among at least two microphones to obtain a transformed eighth signal; obtain a ninth signal in the first frequency band from the transformed eighth signal; and calculate a first spectrum ratio based on the seventh signal, the ninth signal, and the number of frequency points included in the first frequency band.

[0202] Optionally, in an embodiment of the present application, the processor 110 is further configured to perform signal recognition on at least two first signals respectively collected by at least two microphones before obtaining the first spectrum ratio, to obtain at least two feature information, wherein the at least two feature information correspond one-to-one to the at least two first signals. The processor 110 is specifically configured to obtain the first spectrum ratio when both of the at least two feature information match the preset feature information.

[0203] The embodiment of the present application provides an electronic device, which can judge whether the signals collected by at least two microphones are human voice signals through the recognition error corresponding to the secondary path. Therefore, when it is determined that the signals collected by at least two microphones are human voice signals, it can judge whether the signals collected by at least two microphones are emitted by the user wearing the audio device through the ratio of the spectrum of the signals collected by at least two microphones. It can be understood that when the user wearing the audio device speaks, since the in-ear microphone among the at least two microphones collects the sound signal emitted through bone conduction, the energy of the signal output by the in-ear microphone will be enhanced, and at this time, the spectrum of the signal output by at least two microphones will change significantly, and then the ratio of the spectrum will also change. Therefore, when the audio device determines that the first spectrum ratio is greater than or equal to the second threshold, it can determine that the collected sound is emitted by the user wearing the audio device. In this way, the phenomenon of misrecognition is avoided and the accuracy of recognizing the user's voice is improved.

[0204] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0205] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0206] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0207] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0208] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0209] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0210] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0211] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0212] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0213] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned speech recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0214] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0215] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0216] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A speech recognition method, characterized in that: Executed by an audio device, the method includes: Obtaining an identification error corresponding to a secondary path of the audio device, wherein the identification error is used to indicate whether a signal collected by at least two microphones in the audio device is a human voice signal; When the identification error is greater than or equal to a first threshold, obtaining a first spectrum ratio, where the first spectrum ratio is a ratio of spectrums of signals collected by the at least two microphones; When the first spectrum ratio is greater than or equal to a second threshold, it is determined that the signals collected by the at least two microphones are emitted by a user wearing the audio device.

2. The method according to claim 1, characterized in that: Before obtaining the identification error corresponding to the secondary path of the audio device, the method further includes: Acquire at least two first signals, and perform fusion processing on the at least two first signals to obtain a second signal, wherein the at least two first signals are signals collected by the at least two microphones, and the at least two first signals correspond one-to-one to the at least two microphones; The obtaining of the identification error corresponding to the secondary path of the audio device includes: inputting the second signal into the secondary path, processing the second signal through the secondary path to output a third signal; The identification error is determined based on a frequency spectrum of the second signal and a frequency spectrum of the third signal.

3. The method according to claim 2, characterized in that The determining the identification error based on the spectrum of the second signal and the spectrum of the third signal includes: Performing a fast Fourier transform on the second signal to obtain a transformed second signal; Acquire a fourth signal in the first frequency band from the transformed second signal; Performing a fast Fourier transform on the third signal to obtain the transformed third signal; Acquire a fifth signal in the first frequency band from the transformed third signal; The identification error is obtained by calculation based on the fourth signal, the fifth signal, and the number of frequency points included in the first frequency band.

4. The method according to claim 1, characterized in that The obtaining of the first spectrum ratio includes: Performing a fast Fourier transform on a sixth signal collected by a first microphone among the at least two microphones to obtain the transformed sixth signal; Acquire a seventh signal in the first frequency band from the transformed sixth signal; Performing a fast Fourier transform on an eighth signal collected by a second microphone among the at least two microphones to obtain the transformed eighth signal; Acquire a ninth signal in the first frequency band from the transformed eighth signal; The first spectrum ratio is calculated based on the seventh signal, the ninth signal, and the number of frequency points included in the first frequency band.

5. The method according to claim 1 or 4, characterized in that: Before obtaining the first spectrum ratio, the method further includes: Performing signal recognition on at least two first signals respectively collected by the at least two microphones to obtain at least two pieces of feature information, wherein the at least two pieces of feature information correspond one-to-one to the at least two first signals; The obtaining of the first spectrum ratio includes: In a case where both of the at least two pieces of characteristic information match the preset characteristic information, the first spectrum ratio is acquired.

6. A speech recognition device, characterized in that: The speech recognition device comprises: an acquisition module and a recognition module; The acquisition module is used to acquire an identification error corresponding to a secondary path of the audio device, wherein the identification error is used to indicate whether the signals collected by at least two microphones in the audio device are human voice signals; and when the identification error is greater than or equal to a first threshold, acquire a first spectrum ratio, wherein the first spectrum ratio is a ratio of the spectrums of the signals collected by the at least two microphones; The identification module is used to determine that the signals collected by the at least two microphones are emitted by the user wearing the audio device when the first spectrum ratio acquired by the acquisition module is greater than or equal to a second threshold.

7. The device according to claim 6, characterized in that The speech recognition device further comprises: an input module and an output module; The acquisition module is further used to acquire at least two first signals and perform fusion processing on the at least two first signals to obtain a second signal, wherein the at least two first signals are signals collected by the at least two microphones, and the at least two first signals correspond one-to-one to the at least two microphones; The input module is used to input the second signal acquired by the acquisition module into the secondary path; The output module is used for processing the second signal through the secondary path to output a third signal; The acquisition module is specifically configured to determine the identification error based on the frequency spectrum of the second signal and the frequency spectrum of the third signal.

8. The device according to claim 6, characterized in that The acquisition module is specifically used for: Performing a fast Fourier transform on a sixth signal collected by a first microphone among the at least two microphones to obtain the transformed sixth signal; Acquire a seventh signal in the first frequency band from the transformed sixth signal; Performing a fast Fourier transform on an eighth signal collected by a second microphone among the at least two microphones to obtain the transformed eighth signal; Acquire a ninth signal in the first frequency band from the transformed eighth signal; The first spectrum ratio is calculated based on the seventh signal, the ninth signal, and the number of frequency points included in the first frequency band.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the speech recognition method as claimed in any one of claims 1 to 5.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the speech recognition method according to any one of claims 1 to 5 are implemented.

11. A computer program product, wherein the computer program product is stored in a storage medium and is executed by at least one processor to implement the speech recognition method according to any one of claims 1 to 5.

Images