Speech recognition method and device, equipment and storage medium

Abstract

The invention discloses a speech recognition method and device, equipment and a storage medium. The method comprises the following steps: acquiring speech feature information of at least one speech chunk in speech to be recognized; time sequence classification processing is carried out on the speech feature information of a first chunk located at the initial position in the at least one chunk to obtain a first probability matrix of the first chunk, the first probability matrix comprises at least one probability vector, and the probability vector comprises probability values of multiple pinyin with tones; determining the phonetic alphabets with the probability values meeting a preset condition in the first probability matrix as target phonetic alphabets; and displaying a target single character corresponding to the target pinyin in a preset mapping table, wherein the preset mapping table comprises a plurality of pinyin with tones and a single character corresponding to each pinyin with tones. According to the embodiment of the invention, the first word on the screen is quickly determined, so that the perception of the user on the first word on the screen is improved, and the user experience of voice recognition is improved.

Description

Technical Field

[0001] The present application belongs to the field of speech recognition technology, and in particular, relates to a speech recognition method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, speech recognition technology has been widely used in various fields. For example, in human-computer interaction scenarios, three main processing contents are required: speech recognition, natural language understanding, and speech response. Among them, as the first function to interact with users, the speed of speech recognition has a significant impact on the user experience. However, in the existing speech recognition, users have a more obvious perception of the delay in the first word appearing on the screen, which makes the user experience of speech recognition poor. Summary of the invention

[0003] The embodiments of the present application provide a speech recognition method, apparatus, device and computer storage medium. By determining the single word corresponding to the target pinyin in the preset mapping table as the target single word, the first word on the screen can be quickly determined, thereby improving the user's perception of the first word on the screen, and further improving the user's speech recognition experience.

[0004] In a first aspect, an embodiment of the present application provides a speech recognition method, the method comprising:

[0005] Acquiring speech feature information of at least one language block in the speech to be recognized;

[0006] Performing time-series classification processing on the speech feature information of a first language block located at a starting position in the at least one language block to obtain a first probability matrix of the first language block, wherein the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple pinyins with tones;

[0007] Determine the toned pinyin whose probability value in the first probability matrix meets the preset condition as the target pinyin of the first language block;

[0008] The target single word corresponding to the target pinyin in the preset mapping table is displayed, and the preset mapping table includes a plurality of the pinyins with tones and the single words corresponding to the pinyins with tones.

[0009] In a second aspect, an embodiment of the present application provides a speech recognition device, the device comprising:

[0010] An acquisition module, used to acquire speech feature information of at least one language block in the speech to be recognized;

[0011] A first time-series classification module is used to perform time-series classification processing on the speech feature information of a first language block located at a starting position in the at least one language block to obtain a first probability matrix of the first language block, wherein the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple pinyins with tones;

[0012] A determination module, configured to determine the tone-bearing pinyin whose probability value in the first probability matrix meets a preset condition as the target pinyin of the first language block;

[0013] The display module is used to display the target single word corresponding to the target pinyin in the preset mapping table, and the preset mapping table includes multiple pinyins with tones and the single words corresponding to each pinyin with tones.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the speech recognition method as described in any one of the above is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the speech recognition method as described in any one of the above is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes any one of the speech recognition methods described above.

[0017] In a sixth aspect, an embodiment of the present application provides a vehicle, comprising: an electronic device, wherein the electronic device is used to implement the speech recognition method as described in any one of the above.

[0018] The speech recognition method, device, equipment and storage medium of the embodiment of the present application can perform time-series classification processing on the speech feature information of the first language block in the speech to be recognized, and obtain the first probability matrix of the first language block, the first language block is the language block located at the starting position in at least one language block, the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple tone pinyins; then the tone pinyins whose probability values ​​in the first probability matrix meet the preset conditions are determined as the target pinyins of the first language block; finally, the target single word corresponding to the target pinyin in the preset mapping table is displayed, and the preset mapping table includes multiple tone pinyins and the single words corresponding to each tone pinyin. In this way, in this embodiment, by determining the single word corresponding to the target pinyin in the preset mapping table as the target single word, the first word on the screen can be quickly determined, thereby improving the user's perception of the first word on the screen, and then improving the user's speech recognition experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 It is a flowchart of the speech recognition method provided in the embodiment of the present application;

[0021] Figure 2 It is a structural diagram of the speech recognition model provided in the embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of a peak in the first probability matrix provided in an embodiment of the present application.

[0023] Figure 4 is a structural diagram of a speech recognition device provided in an embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0027] With the rapid development of artificial intelligence technology, speech recognition technology has been widely used in various fields. For example, in human-computer interaction scenarios, three main processing contents are required: speech recognition, natural language understanding, and voice response. Among them, as the first function to interact with users, the speed of speech recognition has a significant impact on the user experience. However, in existing speech recognition, words are decoded by decoding images, so users' perception of the delay of the first word on the screen is more obvious, which makes the user experience of speech recognition poor.

[0028] In order to solve the problems in the prior art, the embodiments of the present application provide a speech recognition method, device, equipment and computer storage medium. The speech recognition method provided by the embodiments of the present application is first introduced below.

[0029] Figure 1FIG. 1 is a flow chart of a speech recognition method provided by an embodiment of the present application. Figure 1 As shown, a speech recognition method may include the following steps S101 to S104.

[0030] S101: Acquire speech feature information of at least one language block in the speech to be recognized.

[0031] S102, performing time-series classification processing on the speech feature information of a first language block located at a starting position in at least one language block to obtain a first probability matrix of the first language block, wherein the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple pinyins with tones.

[0032] S103, determining the tone-bearing pinyin whose probability value in the first probability matrix meets the preset conditions as the target pinyin of the first language block.

[0033] S104, displaying the target word corresponding to the target pinyin in the preset mapping table, wherein the preset mapping table includes a plurality of pinyins with tones and the words corresponding to the pinyins with tones.

[0034] The speech recognition method of the embodiment of the present application can perform time-series classification processing on the speech feature information of the first language block in the speech to be recognized, and obtain the first probability matrix of the first language block, the first language block is the language block located at the starting position in at least one language block, the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple tone pinyins; then the tone pinyins whose probability values ​​in the first probability matrix meet the preset conditions are determined as target pinyins; finally, the target single words corresponding to the target pinyins in the preset mapping table are displayed, and the preset mapping table includes multiple tone pinyins and the single words corresponding to each tone pinyin. In this way, in this embodiment, by determining the single words corresponding to the target pinyins in the preset mapping table as the target single words, the first word on the screen can be quickly determined, thereby improving the user's perception of the first word on the screen, and then improving the user's speech recognition experience.

[0035] The above-mentioned speech recognition method is mainly applied to vehicle-mounted terminals. The present application is not limited thereto, and can also be a terminal device with data processing capability such as a mobile terminal.

[0036] The specific implementation methods of the above steps are introduced below.

[0037] In S101, since the speech to be recognized is not stable in the macroscopic sense, but stable in the microscopic sense, and has short-term stability (the speech can be considered to be approximately unchanged within 10-30ms), the speech can be divided into several short segments according to a preset time length (for example, 400ms) for processing, and each short segment is called a language block. For example, the speech to be recognized is "Cyndi Wang boy", and it is divided into the language blocks of "Cyndi Wang" and "boy" according to the preset time length.

[0038] The above-mentioned speech feature information may be speech features used to characterize a speech block.

[0039] In some embodiments, the above S101 may specifically include:

[0040] When obtaining a speech to be recognized, segmenting the speech to be recognized into at least one language block;

[0041] Processing each language block in at least one language block to extract physical feature information of each language block, where the physical feature information is information representing acoustic features in the language block;

[0042] The physical feature information of each language block is processed to extract the speech feature information of each language block.

[0043] The above segmentation of the speech to be recognized into at least one block may be, for example, segmenting the speech to be recognized into at least one block according to a preset duration of 400ms. It should be noted that in the embodiment of the present application, the preset duration for segmentation is not limited to 400ms, but may be other values, which are not specifically limited here.

[0044] The above-mentioned physical feature information is information that characterizes acoustic features in a speech block, and illustratively may include filter bank features (Filter Bank, Fbank) and Mel-Frequency Cepstral Coefficeitns (MFCCs).

[0045] The above processing is performed on each language block to extract the physical feature information of each language block. Exemplarily, each language block can be pre-emphasized, framed and windowed, discrete Fourier transformed (DFT) and Mel filter group are performed to extract the Fbank features of each language block, and then the inverse discrete Fourier transform (IDFT) is continued to be processed to extract the MFCCs features of each language block.

[0046] The above-mentioned physical feature information of each language block is processed to extract the speech feature information of each language block. Exemplarily, the physical feature information of each language block can be processed by at least one encoder to extract the speech feature information of each language block. The encoder can perform neural network feature extraction on the physical feature information, so that the feature density of the speech feature information is higher than the feature density of the physical feature information, that is, the speech feature information is more abstract than the physical feature information.

[0047] In this embodiment, the speech to be recognized is divided into at least one language block, and each language block is processed to extract physical feature information of each language block, thereby accurately extracting language feature information.

[0048] As an implementation of the present application, in order to further improve the speed of extracting speech feature information, before the physical feature information of each language block is processed to extract the speech feature information of each language block, the following may also be included:

[0049] Downsampling the physical feature information of each language block to obtain third feature information of each language block, where the number of frames of the third feature information is lower than the number of frames of the physical feature information;

[0050] The above processing of the physical feature information of each language block to extract the speech feature information of each language block may specifically include:

[0051] The third feature information of each language block is processed to extract the speech feature information of each language block.

[0052] The above-mentioned downsampling is a multi-rate digital signal processing technology or a process of reducing the signal sampling rate, which is usually used to reduce the data transmission rate or data size. In this embodiment, if the downsampling is 4, it means that the number of frames of the third feature information is one quarter of the number of frames of the physical feature information. For example, the number of frames of the physical feature information is 40, and the number of frames of the third feature information is 10.

[0053] In this embodiment, by downsampling the physical feature information of each language block to obtain the third feature information of each language block, the amount of data for feature extraction can be reduced, thereby further improving the speed of processing the third feature information of each language block, and extracting the speech feature information of each language block more quickly.

[0054] In S102, the first language block may be a language block located at the starting position of at least one language block of the speech to be recognized.

[0055] The above-mentioned pinyin with tone can be exemplarily gan4, where letters represent pinyin and numbers represent tone. Tone refers to the tone in Mandarin, which is usually called four tones, namely, yinping (first tone), represented by "ˉ", such as lā; yangping (second tone), represented by "ˊ", such as lá; shangsheng (third tone), represented by "ˇ", such as lǎ; qusheng (fourth tone), represented by "ˋ", such as là. In addition, there is a special tone in Mandarin, called light tone, and specifically 0 / 1 / 2 / 3 / 4 represent light tone, first tone, second tone, third tone and fourth tone respectively.

[0056] The above-mentioned connectionist temporal classification (CTC) can calculate the probability distribution of all possible outputs y for a given input x, which is used to solve the problem that it is difficult to correspond one-to-one between input sequences and output sequences.

[0057] The above-mentioned time-series classification processing is performed on the speech feature information of the first language block located at the starting position in at least one language block to obtain the first probability matrix of the first language block. Exemplarily, it can be that the time-series classification processing is performed on the speech feature information of the first language block located at the starting position in at least one language block, and the probability of its output on 2000 toned pinyin is predicted to obtain the first probability matrix of the first language block.

[0058] The first probability matrix may include at least one probability vector. For example, when the duration of the first language block is 400 ms, the frame shift is 10 ms, and the number of toned pinyin is 2000, the first probability matrix has 10 rows and 2000 columns.

[0059] The above probability vector is a row vector in the first probability matrix, and may include probability values ​​of multiple pinyins with tones. The probability values ​​of multiple pinyins with tones included in a probability vector are normally distributed, and the sum of the probability values ​​of the multiple pinyins with tones is 1.

[0060] In S103, the target pinyin, for example, can be the pinyin with the highest probability value in the first vector, where the first vector is the probability vector located at the starting position in the first probability matrix; or, it can also be the first pinyin with a probability value greater than a preset threshold in the first probability matrix.

[0061] In some embodiments, determining the tone-matched pinyin whose probability value in the first probability matrix meets a preset condition as the target pinyin may specifically include:

[0062] The tone-matched pinyin with the highest probability value in the first vector is determined as the target pinyin, and the first vector is a probability vector located at the starting position in the first probability matrix.

[0063] In this embodiment, only the tone-accented pinyin with the highest probability value in the first vector located at the starting position of the first probability matrix is ​​determined as the target pinyin, and there is no need to further explore whether there are probability values ​​that meet the preset conditions in other probability vectors in the first probability matrix, thereby improving the speed of determining the target pinyin, achieving rapid display of the first character on the screen, improving the user's perception of the first character on the screen, and thus improving the user experience of speech recognition.

[0064] In S104, the preset mapping table includes a plurality of pinyins with tones and the corresponding single words of each pinyin with tones. The preset mapping table is as follows:

[0065]

[0066]

[0067] Among them, letters represent pinyin, and numbers represent tones, which refer to the tones in Mandarin, usually called four tones, namely, Yinping (first tone), represented by "ˉ", such as lā; Yangping (second tone), represented by "ˊ", such as lá; Shangsheng (third tone), represented by "ˇ", such as lǎ; Qusheng (fourth tone), represented by "ˋ", such as là. In addition, there is a special tone in Mandarin, called light tone. Specifically, 0 / 1 / 2 / 3 / 4 represent light tone, first tone, second tone, third tone and fourth tone respectively.

[0068] In some embodiments, in order to balance the speed of first word appearing on the screen and the accuracy of first word appearing on the screen, the above S104 may specifically include:

[0069] Determine the word corresponding to the target pinyin in the preset mapping table as the target word;

[0070] When the probability value of the target pinyin is greater than a preset threshold, the target word is displayed.

[0071] In the above case, when the probability value of the target pinyin is greater than the preset threshold, the target word is displayed. For example, when the probability value of the target pinyin is greater than 0.8, the target word is displayed. In this embodiment, the preset threshold is not limited to 0.8, but can also be other values, which are not specifically limited here.

[0072] In this embodiment, by comparing the probability value of the target pinyin with a preset threshold, the target word is displayed only when the probability value of the target pinyin is greater than the preset threshold, thereby balancing the speed of first word display on the screen and the accuracy of first word display on the screen.

[0073] In some embodiments, the above method may further include:

[0074] When the probability value of the target pinyin is less than or equal to the preset threshold, the first pinyin with a tone whose probability value is greater than the preset threshold in the first probability matrix is ​​determined as the first pinyin;

[0075] The single word corresponding to the first pinyin in the preset mapping table is displayed.

[0076] In this embodiment, when the probability value of the target pinyin is less than or equal to a preset threshold, the first tone-matched pinyin with a probability value greater than the preset threshold in the first probability matrix is ​​determined as the first pinyin, and the single word corresponding to the first pinyin in the preset mapping table is displayed. When the target single word may be incorrect, the first word on the screen can be quickly determined, thereby improving the accuracy rate of the first word on the screen.

[0077] As another implementation of the present application, in order to improve the speed of performing time-series classification processing on the speech feature information, before the above S102, the following may also be included:

[0078] Performing a linear transformation process on the speech feature information of a first language block located at a starting position in the at least one language block to obtain first feature information of the first language block, wherein the dimension of the first feature information is lower than the dimension of the speech feature information;

[0079] The above S102 may specifically include:

[0080] A time series classification process is performed on the first feature information of the first language block to obtain a first probability matrix of the first language block.

[0081] The linear transformation processing is performed on the speech feature information of the first language block located at the starting position in the at least one language block to obtain the first feature information of the first language block. Exemplarily, the linear transformation processing is performed on the speech feature information of the first language block located at the starting position in the at least one language block by using a deep neural network (DNN) to obtain the first feature information of the first language block.

[0082] The dimension of the first feature information is lower than that of the speech feature information. For example, the dimension of the speech feature information is 10*512, and after linear transformation through a deep neural network, it becomes the first feature information of 10*128 dimension.

[0083] In this embodiment, by performing linear transformation processing on the speech feature information of the first language block, the dimension of the first feature information is made lower than the dimension of the speech feature information, thereby reducing the amount of data for time series classification processing and further increasing the speed of time series classification processing of the speech feature information.

[0084] As another implementation of the present application, in order to correct an incorrect first character, after the above S101, the following may also be included:

[0085] Processing the speech feature information of each language block in at least one language block to extract second feature information of each language block, wherein the feature density of the second feature information is higher than the feature density of the speech feature information;

[0086] Performing time series classification processing on the second feature information of the first language block to obtain a second probability matrix of the first language block;

[0087] The second probability matrix is ​​parsed through a decoding graph to obtain a first word of the first language block, wherein the decoding graph is constructed by a plurality of tone pinyins and corresponding words;

[0088] After displaying the target word corresponding to the target pinyin in the preset mapping table, the method further includes:

[0089] When the target word does not match the first word, the first word is displayed.

[0090] The above-mentioned processing of the speech feature information of each language block to extract the second feature information of each language block can be exemplarily performed by processing the speech feature information of each language block by at least one encoder to extract the second feature information of each language block.

[0091] The feature density of the second feature information is higher than that of the voice feature information, which means that the second feature information is more abstract and has more features than the voice feature information. The feature density can be used to characterize the density of features in the second feature information or the voice feature information.

[0092] The above decoding graph is constructed by a plurality of tone pinyins and corresponding single characters. The decoding graph can be exemplarily a WFST decoder.

[0093] In this embodiment, the speech feature information of each language block is processed to extract second feature information with more features, and is analyzed by a decoding graph with high accuracy determined by single words. When the target word is inconsistent with the first word, the first word is displayed to correct the incorrect first word and improve the accuracy of speech recognition.

[0094] In order to facilitate the understanding of the speech recognition method in the embodiment of the present application, the application process of the speech recognition method is described here, as follows:

[0095] The present application provides a speech recognition method, such as Figure 2 As shown, a bypass branch is connected to the lower layer of the backbone model for CTC training. During decoding, the bypass branch decodes the first chunk (i.e. the above-mentioned language block) and uses the first peak result in the chunk (i.e. the single word corresponding to the pinyin with the highest probability value in the above-mentioned first vector) to display on the screen, which can greatly reduce the delay of the first word displaying on the screen.

[0096] Figure 2 The decoding process of chunkbychunk is described in . The first chunk is special in that it extracts a branch from the low-level encoder to form a bypass. As shown in the branch part in the figure, it includes: linear transformation layer + classification layer.

[0097] After the data of the first chunk is sent in, the feature extraction operation (i.e. the extraction of the above-mentioned physical feature information) is first performed, and the features then enter the backbone network for downsampling and forward calculation. After reaching the 4th layer (i.e. the low-level encoder), it starts to bifurcate, and the output of the 4th layer continues to be sent to the 5th layer of the backbone (i.e. the high-level encoder), while the output of the 4th layer is sent to the bypass branch. After the linear transformation of the deep neural network of the bypass branch is completed, the CTC classification layer calculation is immediately performed to predict the first probability matrix of the first chunk of the bypass.

[0098] For example, assuming that the duration of the first chunk is 400ms (frame shift is 10ms), and the model downsampling is 4, then the first chunk is calculated by the bypass branch to obtain a 10xN first probability matrix, where 10 represents the number of frames, N represents the dimension, and N is the total number of modeling units. In this case, the modeling unit uses pinyin with tone, and the total number of modeling units is, for example, 2000.

[0099] Due to the particularity of the CTC classification layer, the probability value of the probability vector in the first probability matrix of the bypass branch is peaked, as shown in Figure 3 As shown, the first peak is found in the first probability matrix of the bypass branch, the first peak is taken, and the result of the first word being on the screen is determined by querying the mapping table of "token pinyin tokenId-single word".

[0100] The mapping table example is as follows:

[0101]

[0102]

[0103] Finally, the first word is put on the screen by looking up a table, which is faster than completing the main trunk and then entering the wfst decoder for searching.

[0104] In order to balance the accuracy and speed of the first word on the screen, the probability and threshold of the bypassed first peak need to be judged. Given a threshold (i.e. the above preset threshold), th_score, when the probability value is greater than the threshold, the word corresponding to the first peak of the first chunk is put on the screen.

[0105] In this embodiment, a bypass branch is pulled out at the lower layer of the network for training, and the word corresponding to the first peak of the first chunk is displayed on the screen, thereby significantly reducing the first word response delay and improving the user experience.

[0106] Based on the speech recognition method provided in the above embodiment, the present application also provides a specific implementation of a speech recognition device. Please refer to the following embodiment.

[0107] like Figure 4As shown, the speech recognition device 400 provided in the embodiment of the present application may include the following modules: an acquisition module 401 , a first time series classification module 402 , a first determination module 403 and a first display module 404 .

[0108] The acquisition module 401 is used to acquire speech feature information of at least one language block in the speech to be recognized.

[0109] The first time series classification module 402 is used to perform time series classification processing on the speech feature information of the first language block located at the starting position in at least one language block to obtain a first probability matrix of the first language block. The first probability matrix includes at least one probability vector, and the probability vector includes multiple probability values ​​of toned pinyin.

[0110] The first determination module 403 is used to determine the tone-bearing pinyin whose probability value meets the preset condition in the first probability matrix as the target pinyin of the first language block.

[0111] The first display module 404 is used to display the target word corresponding to the target pinyin in the preset mapping table. The preset mapping table includes multiple pinyins with tones and the words corresponding to the pinyins with tones.

[0112] The speech recognition device of the embodiment of the present application can perform time-series classification processing on the speech feature information of the first language block in the speech to be recognized, and obtain the first probability matrix of the first language block, the first language block is the language block located at the starting position in at least one language block, the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple tone pinyins; then the tone pinyins whose probability values ​​in the first probability matrix meet the preset conditions are determined as target pinyins; finally, the target single words corresponding to the target pinyins in the preset mapping table are displayed, and the preset mapping table includes multiple tone pinyins and the single words corresponding to each tone pinyin. In this way, in this embodiment, by determining the single words corresponding to the target pinyins in the preset mapping table as the target single words, the first word on the screen can be quickly determined, thereby improving the user's perception of the first word on the screen, and then improving the user's speech recognition experience.

[0113] As another implementation of the present application, in order to improve the speed of performing time-series classification processing on speech feature information, the apparatus 400 may further include:

[0114] A linear transformation module, configured to perform a linear transformation process on the speech feature information of a first language block located at a starting position in at least one language block to obtain first feature information of the first language block, wherein the dimension of the first feature information is lower than the dimension of the speech feature information;

[0115] The first time series classification module 402 is specifically configured to perform time series classification processing on the first feature information of the first language block to obtain a first probability matrix of the first language block.

[0116] As another implementation of the present application, in order to correct an incorrect first character, the above-mentioned device 400 may further include:

[0117] A feature extraction module, used for processing the speech feature information of each language block, extracting the second feature information of each language block, wherein the feature density of the second feature information is higher than the feature density of the speech feature information;

[0118] A second time series classification module, used for performing time series classification processing on the second feature information of the first language block to obtain a second probability matrix of the first language block;

[0119] A parsing module, used for parsing the second probability matrix through a decoding graph to obtain a first word of the first language block, wherein the decoding graph is constructed by a plurality of tone pinyins and corresponding words;

[0120] The first display module 404 is further configured to display the first word when the target word is inconsistent with the first word.

[0121] In some embodiments, the acquisition module 401 may specifically include:

[0122] A segmentation unit, configured to segment the speech to be recognized into at least one language block when the speech to be recognized is acquired;

[0123] A first feature extraction unit, configured to process each language block in at least one language block to extract physical feature information of each language block, where the physical feature information is information representing acoustic features in the language block;

[0124] The second feature extraction unit is used to process the physical feature information of each language block to extract the speech feature information of each language block.

[0125] As an implementation of the present application, in order to further improve the extraction speed of speech feature information, the acquisition module 401 may further include:

[0126] A downsampling unit, configured to perform downsampling processing on the physical feature information of each language block to obtain third feature information of each language block, wherein the frame number of the third feature information is lower than the frame number of the physical feature information;

[0127] The second feature extraction unit may also be used to process the third feature information of each language block to extract the speech feature information of each language block.

[0128] In some embodiments, in order to balance the speed of first word appearing on the screen and the accuracy of first word appearing on the screen, the first display module 404 may specifically include:

[0129] A first determining unit, used for determining a single word corresponding to the target pinyin in a preset mapping table as a target single word;

[0130] The first display unit is used to display the target word when the probability value of the target pinyin is greater than a preset threshold.

[0131] In some embodiments, the above device may further include:

[0132] A second determination module is used to determine the first tone-bearing pinyin with a probability value greater than the preset threshold in the first probability matrix as the first pinyin when the probability value of the target pinyin is less than or equal to the preset threshold;

[0133] The second display module is used to display the single word corresponding to the first pinyin in the preset mapping table.

[0134] Based on the speech recognition method provided in the above embodiment, the present application also provides a specific implementation method of the electronic device accordingly. Please refer to the following embodiment.

[0135] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0136] The electronic device may include a processor 501 and a memory 502 storing computer program instructions.

[0137] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0138] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0139] In certain embodiments, the memory 502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0140] The processor 501 implements any one of the speech recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory 502 .

[0141] In one example, the electronic device may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0142] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0143] Bus 510 includes hardware, software or both, and the parts of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0144] The electronic device can execute the speech recognition method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 4 The described speech recognition method and device.

[0145] In addition, in combination with the speech recognition method in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any speech recognition method in the above embodiment is implemented.

[0146] In combination with the speech recognition method in the above embodiment, the embodiment of the present application may provide a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes any of the above speech recognition methods.

[0147] In combination with the speech recognition method in the above embodiment, the embodiment of the present application can provide a vehicle for implementation. The vehicle includes: an electronic device for implementing any of the above speech recognition methods.

[0148] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0149] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0150] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0151] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0152] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

[0153] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A speech recognition method, characterized in that: The method comprises: Acquiring speech feature information of at least one language block in the speech to be recognized; Performing time-series classification processing on the speech feature information of a first language block located at a starting position in the at least one language block to obtain a first probability matrix of the first language block, wherein the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple pinyins with tones; Determine the toned pinyin whose probability value in the first probability matrix meets the preset condition as the target pinyin of the first language block; The target single word corresponding to the target pinyin in the preset mapping table is displayed, and the preset mapping table includes a plurality of the pinyins with tones and the single words corresponding to the pinyins with tones.

2. The method according to claim 1, characterized in that Before performing time-series classification processing on the speech feature information of the first language block located at the starting position of the at least one language block to obtain the first probability matrix of the first language block, the method further includes: performing a linear transformation process on the speech feature information of a first language block located at a starting position in the at least one language block to obtain first feature information of the first language block, wherein the dimension of the first feature information is lower than the dimension of the speech feature information; The performing time-series classification processing on the speech feature information of the first language block located at the starting position of the at least one language block to obtain a first probability matrix of the first language block includes: Performing time series classification processing on the first feature information of the first language block to obtain a first probability matrix of the first language block.

3. The method according to claim 2, characterized in that After acquiring the speech feature information of at least one language block in the speech to be recognized, the method further includes: Processing the speech feature information of each language block in the at least one language block to extract second feature information of each language block, wherein the feature density of the second feature information is higher than the feature density of the speech feature information; Performing time series classification processing on the second feature information of the first language block to obtain a second probability matrix of the first language block; The second probability matrix is ​​parsed through a decoding graph to obtain a first word of the first language block, wherein the decoding graph is constructed by a plurality of tone pinyins and corresponding words; After displaying the target word corresponding to the target pinyin in the preset mapping table, the method further comprises: When the target word is inconsistent with the first word, the first word is displayed.

4. The method according to claim 1, characterized in that The step of obtaining speech feature information of at least one language block in the speech to be recognized includes: When obtaining a speech to be recognized, segmenting the speech to be recognized into at least one language block; Processing each language block in the at least one language block to extract physical feature information of each language block, wherein the physical feature information is information representing acoustic features in the language block; The physical feature information of each of the language blocks is processed to extract the speech feature information of each of the language blocks.

5. The method according to claim 4, characterized in that Before the physical feature information of each of the language blocks is processed to extract the speech feature information of each of the language blocks, the method further includes: Downsampling the physical feature information of each of the language blocks to obtain third feature information of each of the language blocks, wherein the number of frames of the third feature information is lower than the number of frames of the physical feature information; The processing of the physical feature information of each of the language blocks to extract the speech feature information of each of the language blocks includes: The third feature information of each of the language blocks is processed to extract the speech feature information of each of the language blocks.

6. The method according to claim 1, characterized in that The displaying of the target word corresponding to the target pinyin in the preset mapping table comprises: Determine the word corresponding to the target pinyin in the preset mapping table as the target word; When the probability value of the target pinyin is greater than a preset threshold, the target word is displayed.

7. The method according to claim 6, characterized in that The method further comprises: When the probability value of the target pinyin is less than or equal to the preset threshold, the first pinyin with a tone whose probability value is greater than the preset threshold in the first probability matrix is ​​determined as the first pinyin; The single word corresponding to the first pinyin in the preset mapping table is displayed.

8. A speech recognition device, characterized in that: The device comprises: An acquisition module, used to acquire speech feature information of at least one language block in the speech to be recognized; A first time-series classification module is used to perform time-series classification processing on the speech feature information of a first language block located at a starting position in the at least one language block to obtain a first probability matrix of the first language block, wherein the first probability matrix includes at least one probability vector, and the probability vector includes probability values ​​of multiple pinyins with tones; A determination module, configured to determine the tone-bearing pinyin whose probability value in the first probability matrix meets a preset condition as the target pinyin of the first language block; The display module is used to display the target single word corresponding to the target pinyin in the preset mapping table, and the preset mapping table includes multiple pinyins with tones and the single words corresponding to each pinyin with tones.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the speech recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the speech recognition method according to any one of claims 1 to 7 is implemented.

Images