Speech recognition system and related speech recognition method

Abstract

The invention discloses a speech recognition system, which comprises a processing circuit and a memory. The processing circuit is configured to: receive a plurality of sound signals, respectively; respectively performing a time domain-to-frequency domain conversion operation on the plurality of sound signals to generate a plurality of frequency domain signals; using a morphological filter to filter the plurality of frequency domain signals, respectively, to generate a plurality of filtered background images; respectively generating a plurality of initial voice fingerprints according to the plurality of filtered background images, wherein each initial voice fingerprint in the plurality of initial voice fingerprints comprises time and frequency points corresponding to a plurality of peak values in the corresponding filtered background image; generating at least one voice fingerprint according to the plurality of initial voice fingerprints; and storing the at least one voice fingerprint in the memory for voice recognition.

Description

Technical Field

[0001] This invention relates to a speech recognition system. Background Technology

[0002] Due to the uniqueness of each individual's voice, many electronic devices have utilized voiceprints to identify users in recent years. However, since the uniqueness of voice is not limited to differences in the user's vocal structure, but also includes factors such as age and health status, traditional voice recognition devices first perform volume normalization and voice activity detection (VAD) on the received audio signal. They then detect speech-containing segments in the audio signal and perform feature extraction to improve the accuracy of voice recognition. However, these volume normalization and voice activity detection operations increase the design and manufacturing costs of voice recognition devices. Summary of the Invention

[0003] Therefore, one of the objectives of this invention is to provide a speech recognition system that can accurately recognize speech without requiring volume normalization and / or speech activity detection operations, thereby solving the problems described in the prior art.

[0004] In one embodiment of the present invention, a speech recognition system is disclosed, comprising a processing circuit and a memory. The processing circuit is configured to perform the following operations: receiving multiple sound signals respectively; performing time-domain to frequency-domain conversion operations on the multiple sound signals respectively to generate multiple frequency-domain signals; using a morphological filter to filter the multiple frequency-domain signals respectively to generate multiple filtered background images; generating multiple initial speech fingerprints based on the multiple filtered background images respectively, wherein each of the multiple initial speech fingerprints includes the time and frequency points corresponding to multiple peaks in the corresponding filtered background image; generating at least one speech fingerprint based on the multiple initial speech fingerprints; and storing the at least one speech fingerprint in the memory for use in speech recognition.

[0005] In one embodiment of the present invention, a speech recognition method is disclosed, comprising the following steps: receiving multiple sound signals respectively; performing time-domain to frequency-domain conversion operations on the multiple sound signals respectively to generate multiple frequency-domain signals; using morphological filters to filter the multiple frequency-domain signals respectively to generate multiple filtered background images; generating multiple initial speech fingerprints based on the multiple filtered background images respectively, wherein each of the multiple initial speech fingerprints includes the time and frequency points corresponding to multiple peaks in the corresponding filtered background image; generating at least one speech fingerprint based on the multiple initial speech fingerprints; and storing the at least one speech fingerprint in a memory for use in speech recognition. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of a speech recognition system according to an embodiment of the present invention. Figure 2 A flowchart for establishing a voice fingerprint in a voice recognition system according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating how to use morphological filters to filter processed frequency domain signals to generate a filtered background image. Figure 4 A schematic diagram of generating an inflated background image for a morphological filter. Figure 5 A schematic diagram of generating an erosion background map for a morphological filter. Figure 6 This is a schematic diagram of how a morphological filter generates a filtered background image based on a dilated background image and an eroded background image. Figure 7 This is a schematic diagram of generating multiple voice fingerprints based on multiple initial voice fingerprints. Figure 8 This is a flowchart illustrating speech recognition in a speech recognition system according to an embodiment of the present invention. Figure 9 This is a schematic diagram illustrating the determination of whether the voice fingerprint to be identified matches the voice fingerprint in the memory according to an embodiment of the present invention. Detailed Implementation

[0007] Figure 1 This is a schematic diagram of a speech recognition system 100 according to an embodiment of the present invention. Figure 1As shown, the voice recognition system 100 includes a microphone 110, an audio interface 120, a processing circuit 130, and a memory 140. In this embodiment, the voice recognition system 100 can be located in an electronic device, such as a smartphone, tablet computer, laptop computer, desktop computer, or any other electronic device with voice recognition functionality. The microphone 110 can be a microphone used to receive external sounds. The memory 140 can be any type of non-volatile memory (NVM).

[0008] In this embodiment, the user will first establish their own voice fingerprint through the voice recognition system 100. That is, the user will first say a keyword to the voice recognition system 100 a few times. For example, the user will say the keyword three times, such as saying 'Realtek Semiconductor' three times, so that the voice recognition system 100 can use it to identify the user's voice fingerprint after establishing it. Figure 2 A flowchart illustrating the process of establishing a voice fingerprint in a voice recognition system 100 according to an embodiment of the present invention. Figure 2 As shown, the process begins with the voice recognition system 100 being enabled and having completed initialization. The user controls the voice recognition system 100 to begin the voice fingerprint establishment process. In step 202, the user speaks a keyword, and the processing circuit 130 receives the user's voice signal through the audio interface 120 and the microphone 110. In step 204, the processing circuit 130 performs a time-domain to frequency-domain conversion on the user's voice signal, for example, a Fast Fourier Transform, to generate a frequency-domain signal. In step 206, the processing circuit 130 performs noise reduction processing on the frequency-domain signal to generate a processed frequency-domain signal.

[0009] It should be noted that since the time-domain to frequency-domain conversion operation and noise reduction processing in steps 204 and 206 are well known to those skilled in the art, and their content is not the focus of this invention, the relevant details will not be elaborated here.

[0010] At step 208, the processing circuit 130 uses a morphological filter to filter the processed frequency domain signal to generate a filtered background image. Specifically, refer to... Figure 3The flowchart shown illustrates the use of morphological filters to filter the processed frequency domain signal to generate the filtered background image. At step 300, the process begins. At step 302, the processing circuit 130 obtains a spectrogram from the processed frequency domain signal, where the spectrogram, also known as the time spectrum, describes the intensity of each frequency component of the signal as a function of time. In a three-dimensional coordinate system, the x-axis represents time, with each unit being a frame, the duration of which can be determined by the engineer's design, for example, 34.83 milliseconds; the y-axis represents frequency, with each unit being a frequency point, the distance between adjacent frequency points determined by the engineer's design, for example, 21.53 Hz; and the z-axis represents intensity, with the unit being decibels (dB).

[0011] At step 304, the processing circuit 130 sets the core, wherein the size of the core can be 3*3, 5*5 or other suitable size.

[0012] At step 306, the processing circuit 130 performs a dilation process and an erosion process on the spectrogram to generate a dilation background and an erosion background, respectively. Specifically, refer to... Figure 4 The diagram shown illustrates the generation of an inflated background image. (See diagram for example.) Figure 4As shown, assuming the core size set in step 304 is 3*3, and the content of the 5*5 matrix above is a part of the spectrogram, where each cell on the x-axis is a sound frame, each cell on the y-axis is a frequency point, and the value inside each cell is the intensity (decibels). The processing circuit 130 moves the center point of the core to each cell on the spectrogram and selects the maximum value inside the core as the value in the corresponding cell of the dilated background image. Taking (x,y)=(1,5) in the spectrogram as an example, the values ​​inside the 3*3 core centered at (x,y)=(1,5) are "-4", "-3", "-12", and "-10", respectively. Since the maximum value among the above four values ​​is "-3", the processing circuit 130 sets the value of (x,y)=(1,5) in the dilated background image to "-3". Furthermore, taking (x,y)=(3,3) in the spectrogram as an example, the values ​​within the 3*3 core centered at (x,y)=(3,3) are "-10", "7", "11", "-29", "-24", "-22", "-20", "-24", and "-23". Since the maximum value among the above nine values ​​is "11", the processing circuit 130 sets the value of (x,y)=(3,3) in the dilated background diagram to "11".

[0013] Additionally, refer to Figure 5 The diagram shown illustrates the formation of the erosion background image. (See diagram for example.) Figure 5 As shown, assuming the core size set in step 304 is 3*3, and the content of the 5*5 matrix above is a part of the spectrogram, where each cell on the x-axis is a sound frame, each cell on the y-axis is a frequency point, and the value inside each cell is the intensity (decibels). The processing circuit 130 moves the center point of the core to each cell on the spectrogram and selects the minimum value inside the core as the value in the corresponding cell of the erosion background image. Taking (x,y)=(1,5) in the spectrogram as an example, the values ​​inside the 3*3 core centered at (x,y)=(1,5) are "-4", "-3", "-12", and "-10" respectively. Since the minimum value among the above four values ​​is "-12", the processing circuit 130 sets the value of (x,y)=(1,5) in the erosion background image to "-12". Furthermore, taking (x,y)=(3,3) in the spectrogram as an example, the values ​​within the 3*3 core centered at (x,y)=(3,3) are "-10", "7", "11", "-29", "-24", "-22", "-20", "-24", and "-23". Since the minimum value among the above nine values ​​is "-29", the processing circuit 130 sets the value of (x,y)=(3,3) in the erosion background diagram to "-29".

[0014] At step 308, refer to Figure 6The processing circuit 130 subtracts the erosion background image from the dilated background image to obtain the filtered background image. In this embodiment, the value (z-axis) of each cell in the filtered background image represents the difference between the intensity value of each cell (x, y) in the spectrogram and the intensity value of its neighboring cells. In other words, for cells with larger values ​​in the filtered background image, it means that the difference between their intensity and that of their neighboring cells is large; while for cells with smaller values ​​in the filtered background image, it means that the difference between their intensity and that of their neighboring cells is small.

[0015] Next, return to Figure 2 In step 210, the processing circuit 130 generates a corresponding initial speech fingerprint based on the filtered background image. In this embodiment, the processing circuit 130 can determine multiple peaks in the spectrogram based on the filtered background image and the spectrogram, and determine the corresponding initial speech fingerprint accordingly. In this embodiment, the processing circuit 130 can determine multiple peaks in the spectrogram based on the following three conditions: (1) the frequency point is between a preset minimum frequency and a preset maximum frequency, wherein the preset minimum frequency and the preset maximum frequency can be 80 Hz and 3000 Hz respectively, or any other suitable sound frequency range; (2) the intensity of the filtered background image is greater than or equal to a first threshold value, for example, 40 dB; (3) the intensity of the spectrogram is greater than or equal to a second threshold value, for example, 5 dB. Figures 4-6 To illustrate, suppose only frequencies between y = 3 and 5 fall between the preset minimum and maximum frequencies. Then, only the cell (x,y) = (3,4) will have an intensity greater than or equal to the first threshold (40 dB) in the filtered background image and a intensity greater than or equal to the second threshold (5 dB) in the spectrogram. Therefore, the cell (x,y) = (3,4) in the spectrogram is determined to be a peak.

[0016] In this embodiment, when a user utters a keyword once, the above process is performed to generate multiple peaks, and the (x, y) corresponding to these multiple peaks is the initial voice fingerprint. In this embodiment, the user will utter the keyword multiple times in sequence; therefore, the processing circuit 130 will generate multiple initial voice fingerprints. (See reference...) Figure 3 It describes three initial voice fingerprints 710, 720, and 730, where each initial voice fingerprint 710, 720, and 730 contains multiple peaks corresponding to (x, y).

[0017] At step 212, the processing circuit 130 generates at least one voice fingerprint based on a plurality of initial voice fingerprints, and stores the at least one voice fingerprint in the memory 140. (See reference) Figure 7The processing circuit 130 determines the voice fingerprint 780 based on the initial voice fingerprints 710 and 720. For example, the processing circuit 130 can find the peak numbers with matching time intervals (i.e., similar time intervals) in the initial voice fingerprints 710 and 720, and perform a weighted calculation (e.g., an average operation) on the two frequency points corresponding to the peak numbers in the initial voice fingerprints 710 and 720 to obtain the frequency point of the corresponding peak in the voice fingerprint 780. Figure 7 For example, the time intervals between the peak numbers "1", "2", "4", "5", and "6" of the initial voice fingerprint 710 are "3", "7", "11", and "8" respectively, and the time intervals between the peak numbers "1", "2", "3", "5", and "6" of the initial voice fingerprint 720 are "4", "6", "11", and "9" respectively. Therefore, since the peak numbers "1", "2", "4", "5", and "6" of the initial voice fingerprint 710 and the peak numbers "1", "2", "3", "5", and "6" of the initial voice fingerprint 720 have similar time intervals (e.g., the difference in intervals is less than the critical value), the average of the time intervals of the peak numbers "1", "2", "4", "5", and "6" of the initial voice fingerprint 710 and the time intervals of the peak numbers "1", "2", "3", "5", and "6" of the initial voice fingerprint 720 can be used as the time interval of the voice fingerprint 780. That is, the x-axis (time) of the peak numbers "1" to "5" of the voice fingerprint 780 can be set to "0", "3.5", "10", "21", and "29.5". Furthermore, the frequency points corresponding to the peak numbers "1", "2", "4", "5", and "6" of the initial voice fingerprint 710 and the frequency points corresponding to the peak numbers "1", "2", "3", "5", and "6" of the initial voice fingerprint 720 can also be weighted and added together (for example, by averaging) to obtain the y-axis (frequency point) values ​​of the peak numbers "1" to "5" of the voice fingerprint 780. For example, the y-axis (frequency points) of the peak numbers "1" to "5" of the voice fingerprint 780 can be set to "93.5", "29.5", "89", "68", and "20".

[0018] In the embodiments, the peak numbers with matching time intervals in the initial voice fingerprints 710 and 720 described above can be obtained using a recursive exhaustive search method or any other suitable mathematical method.

[0019] Similarly, processing circuit 130 can find peak numbers with similar time intervals in the initial voice fingerprints 720 and 730, and perform weighted calculations (e.g., averaging) on ​​the two frequency points corresponding to the peak numbers in the initial voice fingerprints 720 and 730 to obtain the frequency point of the corresponding peak in the voice fingerprint 790. Figure 7For example, since the peak numbers "2", "3", "4", "5", "6" of the initial voice fingerprint 720 and the peak numbers "2", "4", "5", "6", "7" of the initial voice fingerprint 730 have similar time intervals (e.g., the interval difference is less than the threshold value), the average of the time intervals of the peak numbers "2", "3", "4", "5", "6" of the initial voice fingerprint 720 and the time intervals of the peak numbers "2", "4", "5", "6", "7" of the initial voice fingerprint 730 can be used as the time interval of the voice fingerprint 790. That is, the x-axis (time) of the peak numbers "1" to "5" of the voice fingerprint 790 can be set to "0", "8", "14", "19", "27.5". In addition, the frequency points corresponding to the peak numbers "2", "3", "4", "5", and "6" of the initial voice fingerprint 720 and the frequency points corresponding to the peak numbers "2", "4", "5", "6", and "7" of the initial voice fingerprint 730 can also be weighted and added together (for example, by averaging) to obtain the y-axis (frequency point) values ​​of the peak numbers "1" to "5" of the voice fingerprint 790. For example, the y-axis (frequency points) of the peak numbers "1" to "5" of the voice fingerprint 790 can be set to "31", "88", "32", "67.5", and "22".

[0020] It should be noted that the detailed operations described in the above embodiments for determining two voice fingerprints 780 and 790 based on multiple initial voice fingerprints 710, 720, and 730 are merely illustrative examples and not limitations of the present invention. In other embodiments, as long as the processing circuit 130 can perform similarity matching on two or more initial voice fingerprints to determine the x-axis (time) and y-axis (frequency) values ​​corresponding to multiple peak numbers of multiple voice fingerprints, the designer can use any suitable algorithm, and these design variations should fall within the scope of the present invention.

[0021] In one embodiment, if the number of matching peak numbers found between two initial voice fingerprints is less than a threshold, for example, less than half the number of peaks in the initial voice fingerprint 710, the processing circuit 710 determines that the similarity matching between the two initial voice fingerprints has failed, and therefore cannot determine a voice fingerprint. In another embodiment, if the frequency difference between any pair of matching peak numbers found between two initial voice fingerprints is greater than a threshold, for example, greater than 150 Hz, the processing circuit 710 also determines that the similarity matching between the two initial voice fingerprints has failed, and therefore cannot determine a voice fingerprint.

[0022] Finally, the processing circuit 130 will process at least one voice fingerprint, for example... Figure 7After the voice fingerprints 780 and 790 are stored in the memory 140 for subsequent voice recognition, the process of establishing voice fingerprints in the voice recognition system 100 is completed.

[0023] Figure 8 A flowchart illustrating speech recognition in a speech recognition system 800 according to an embodiment of the present invention. Figure 8 As shown, the process begins; the speech recognition system 100 is enabled and has completed initialization, and the memory 140 has stored the speech fingerprint used for speech recognition, for example... Figure 7 The voice fingerprints 780 and 790 are shown. In step 802, the user speaks a keyword, and the processing circuit 130 receives the user's specific voice signal through the audio interface 120 and the microphone 110. In step 804, the processing circuit 130 performs a time-domain to frequency-domain conversion operation on the specific voice signal, for example, a Fast Fourier Transform operation, to generate a specific frequency-domain signal. In step 806, the processing circuit 130 performs noise reduction processing on the specific frequency-domain signal to generate a processed specific frequency-domain signal.

[0024] At step 808, the processing circuit 130 uses a morphological filter to filter the specific processed frequency domain signal to generate a specific filtered background image, wherein the operation of step 808 is similar to... Figure 2 The steps shown in step 208 are the same.

[0025] In step 810, the processing circuit 130 generates the voice fingerprint to be identified based on a specific filtered background image, wherein the operation of step 810 is similar to... Figure 2 The process of generating the voice fingerprint to be identified in step 810 is the same as the process of generating the initial voice fingerprint in step 210.

[0026] In step 812, the processing circuit 130 determines whether the voice fingerprint to be identified matches one of the multiple voice fingerprints 780, 790 stored in the memory 140, in order to determine whether the voice fingerprint to be identified and the multiple voice fingerprints 780, 790 belong to the same user. Figure 9To illustrate, the processing circuit 130 can search for whether the voice fingerprint to be identified has multiple peak numbers whose time (x) and frequency (y) differ from the time (x) and frequency (y) corresponding to the peak numbers "1" to "5" of the voice fingerprint 780 within a range (i.e., below the threshold). If so, the processing circuit 130 determines that the voice fingerprint to be identified matches the voice fingerprint 780, and that the voice fingerprint to be identified and the voice fingerprint 780 come from the same user; if not, the processing circuit 130 determines that the voice fingerprint to be identified does not match the voice fingerprint 780, and that the voice fingerprint to be identified and the voice fingerprint 780 do not come from the same user. Figure 9 In the example, since the time intervals of the multiple times (x) corresponding to the peak numbers "1", "2", "4", "5", and "6" of the voice fingerprint to be identified are all less than the threshold value, and the differences between the time intervals of the multiple times (x) corresponding to the peak numbers "1" to "5" of the initial voice fingerprint 730 are also less than the threshold value, the processing circuit 130 determines that the voice fingerprint to be identified matches the voice fingerprint 780.

[0027] After determining that the voice fingerprint to be identified matches the voice fingerprint 780, the processing circuit 130 can perform certain specific operations, such as unlocking certain functions of the electronic device, or waking up the electronic device, etc.

[0028] In an embodiment of the present invention, by Figure 2 , 8 In the illustrated embodiment, since the voice fingerprint is established and the voice recognition process is performed by matching the time (x-axis) and frequency (y-axis) corresponding to the multiple peak numbers of the voice fingerprint, voice activity detection (VAD) and volume normalization are not required, thereby reducing the design and manufacturing costs of the voice recognition device. The above description is only a preferred embodiment of the present invention. Any equivalent changes and modifications made to the scope of the present invention should be included in the scope of the present invention. Symbol Explanation

[0029] 100: Voice Recognition System 110: Radio device 120: Audio Interface 130: Processing circuit 140: Memory 200-212: Steps 300-308: Steps 710, 720, 730: Initial voice fingerprint 780, 790: Voice fingerprint 800~812: Steps

Claims

1. A speech recognition system, comprising: Processing circuitry; and Memory; The processing circuit described therein is used to perform the following operations: It can receive multiple sound signals respectively; The multiple sound signals are converted from the time domain to the frequency domain to generate multiple frequency domain signals. Morphological filters are used to filter multiple frequency domain signals separately to generate multiple filtered background images; Multiple initial speech fingerprints are generated based on the multiple filtered background images, wherein each of the multiple initial speech fingerprints contains the time and frequency points corresponding to multiple peaks in the corresponding filtered background image; At least one voice fingerprint is generated based on the plurality of initial voice fingerprints; as well as The at least one voice fingerprint is stored in the memory for use in voice recognition.

2. The speech recognition system as described in claim 1, wherein, The step of generating the at least one voice fingerprint based on the plurality of initial voice fingerprints includes: Similarity matching is performed on at least two of the plurality of initial voice fingerprints to determine the time and frequency points corresponding to the plurality of peaks of the at least one voice fingerprint.

3. The speech recognition system as described in claim 2, wherein, The step of performing similarity matching on at least two of the plurality of initial speech fingerprints to determine the time and frequency points corresponding to the plurality of peaks of the at least one speech fingerprint includes: Find multiple peaks with matching time intervals in the at least two initial voice fingerprints, and determine the time and frequency corresponding to the multiple peaks of the at least one voice fingerprint based on the time and frequency corresponding to the multiple peaks with matching time intervals in the at least two initial voice fingerprints.

4. The speech recognition system as described in claim 1, wherein, The multiple sound signals are generated by the same keywords spoken by the user at different times.

5. The speech recognition system as described in claim 1, wherein, The processing circuit also performs the following operations: Receive specific sound signals; A time-domain to frequency-domain conversion operation is performed on the specific sound signal to generate a specific frequency domain signal; The morphological filter is used to filter the specific frequency domain signal to generate a specific filtered background image; Based on the specific filtered background image, a voice fingerprint to be identified is generated; and Determine whether the voice fingerprint to be identified matches at least one voice fingerprint.

6. A speech recognition method, comprising: It can receive multiple sound signals respectively; The multiple sound signals are converted from the time domain to the frequency domain to generate multiple frequency domain signals. Morphological filters are used to filter the multiple frequency domain signals to generate multiple filtered background images. Multiple initial speech fingerprints are generated based on the multiple filtered background images, wherein each of the multiple initial speech fingerprints contains the time and frequency points corresponding to multiple peaks in the corresponding filtered background image; At least one voice fingerprint is generated based on the plurality of initial voice fingerprints; as well as The at least one voice fingerprint is stored in a memory for use in voice recognition.

7. The speech recognition method as described in claim 6, wherein, The step of generating the at least one voice fingerprint based on the plurality of initial voice fingerprints includes: Similarity matching is performed on at least two of the plurality of initial voice fingerprints to determine the time and frequency points corresponding to the plurality of peaks of the at least one voice fingerprint.

8. The speech recognition method as described in claim 7, wherein, The step of performing similarity matching on at least two of the plurality of initial speech fingerprints to determine the time and frequency points corresponding to the plurality of peaks of the at least one speech fingerprint includes: Find multiple peaks with matching time intervals in the at least two initial voice fingerprints, and determine the time and frequency corresponding to the multiple peaks of the at least one voice fingerprint based on the time and frequency corresponding to the multiple peaks with matching time intervals in the at least two initial voice fingerprints.

9. The speech recognition method as described in claim 6, wherein, The multiple sound signals are generated by the same keywords spoken by the user at different times.

10. The speech recognition method as described in claim 6, further comprising: Receive specific sound signals; A time-domain to frequency-domain conversion operation is performed on the specific sound signal to generate a specific frequency domain signal; The morphological filter is used to filter the specific frequency domain signal to generate a specific filtered background image; Based on the specific filtered background image, a voice fingerprint to be identified is generated; and Determine whether the voice fingerprint to be identified matches at least one voice fingerprint.

Images