Speech recognition method and apparatus

Abstract

The embodiment of the application discloses a speech recognition method and device, relates to the technical field of speech recognition, and mainly aims to solve the problem of poor accuracy of speech recognition in the current speech recognition process. The method is: acquiring speech information, and splitting the speech information into a plurality of first sub-blocks; determining a first sub-block association relationship, wherein the first sub-block association relationship is determined based on an attention parameter calculated by a preset self-attention algorithm on the first sub-block, and the first sub-block association relationship is used to represent the association degree between each first sub-block; grouping and combining each first sub-block into a second sub-block according to the first sub-block association relationship; and identifying the speech information according to the second sub-block to obtain an identification result. The embodiment of the application is used in the speech recognition process.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of speech recognition, and in particular to a speech recognition method and device. BACKGROUND

[0002] With the gradual development of intelligent recognition technology, speech recognition technology has also progressed. At present, in the conventional speech recognition process, it is often implemented based on a translation model, which generally recognizes the collected speech based on speech features, that is, establishes a corresponding text mapping for each segment of speech based on speech features, and performs speech recognition based on the text mapping.

[0003] However, in the actual application, in the process of speech recognition based on a translation model, the recognition process is often simply to establish a text mapping relationship according to speech features, especially in Chinese, there are many homophonic characters and homophonic words, and only through the translation model to recognize according to the audio features often has the problem of poor recognition accuracy. SUMMARY

[0004] In view of the above problems, embodiments of the present application provide a speech recognition method and device, the main purpose of which is to solve the problem of poor accuracy of speech recognition in the current speech recognition process.

[0005] To solve the above technical problems, in a first aspect, the embodiments of the present application provide a speech recognition method, which comprises:

[0006] obtaining speech information and splitting the speech information into a plurality of first sub-blocks;

[0007] determining a first sub-block association relationship, wherein the first sub-block association relationship is determined based on attention parameters calculated by a preset self-attention algorithm on the first sub-blocks, and the first sub-block association relationship is used to represent the association degree between each of the first sub-blocks;

[0008] grouping and combining each of the first sub-blocks into a second sub-block according to the first sub-block association relationship;

[0009] recognizing the speech information according to the second sub-block to obtain a recognition result.

[0010] Optionally, the splitting of the speech information into a plurality of first sub-blocks comprises:

[0011] obtaining corresponding spectrum information according to the speech information, wherein the spectrum information is obtained after the speech information is processed by a Fourier transform algorithm;

[0012] perform a convolution operation according to the spectrum information to obtain a speech feature vector, and determine the first sub-blocks according to the speech feature vector.

[0013] Optionally, the determining the association relationship of the first sub-blocks comprises:

[0014] calculating, by a preset self-attention algorithm, an attention parameter between each two of the first sub-blocks;

[0015] determining, as an association parameter of the corresponding first sub-block, the plurality of attention parameters corresponding to each of the first sub-blocks;

[0016] determining, as associated sub-blocks, a target number of the first sub-blocks according to the association parameter of each of the first sub-blocks, and recording a relationship between the associated sub-blocks as the association relationship of the first sub-blocks, wherein the target number is determined according to a numerical value of the association parameter.

[0017] Optionally, the association parameter comprises a plurality of attention parameters.

[0018] The calculating, by a preset self-attention algorithm, an attention parameter between each two of the first sub-blocks comprises:

[0019] inputting all the first sub-blocks corresponding to the speech information from two input ends of a self-attention algorithm matrix respectively;

[0020] performing inner product calculation according to the self-attention algorithm matrix and the plurality of first sub-blocks input from the two input ends respectively to obtain the attention parameter of each of the remaining first sub-blocks corresponding to the current first sub-block respectively, wherein the remaining first sub-blocks are the remaining sub-blocks of all the first sub-blocks corresponding to the speech information except the current first sub-block.

[0021] obtaining the association parameter of the current first sub-block according to the attention parameter of each of the remaining first sub-blocks corresponding to the current first sub-block respectively.

[0022] Optionally, the determining, as associated sub-blocks, a target number of the first sub-blocks according to the association parameter of each of the first sub-blocks, and recording a relationship between the associated sub-blocks as the association relationship of the first sub-blocks comprises:

[0023] performing a sorting operation on the plurality of attention parameters contained in the association parameter of the first sub-blocks to obtain a sorting sequence, the sorting sequence being arranged in descending order based on the attention parameters;

[0024] obtaining, as target attention parameters, a target number of attention parameters from a starting position of the sorting sequence;

[0025] According to a correspondence between the attention parameter and the remaining first sub-blocks, a target sub-block corresponding to the target attention parameter is determined, the current first sub-block and the target sub-block are determined as the associated sub-blocks, and a relationship of the associated sub-blocks is recorded as the first sub-block association relationship.

[0026] Optionally, the grouping and combining each first sub-block according to the first sub-block association relationship into a second sub-block comprises:

[0027] combining the associated sub-blocks according to the first sub-block association relationship to obtain the second sub-block;

[0028] After the grouping and combining each first sub-block according to the first sub-block association relationship into a second sub-block, the method further comprises:

[0029] accumulating the attention parameter of each first sub-block in the associated sub-blocks to obtain an attention parameter corresponding to the second sub-block;

[0030] determining a second sub-block association relationship according to the attention parameter of each second sub-block, the second sub-block association relationship being positively correlated with the value of the attention parameter.

[0031] Optionally, the identifying the speech information according to the second sub-block to obtain an identification result comprises:

[0032] determining a syntax structure of the speech information according to the correspondence between the second sub-block and the first sub-block, and the correspondence between the speech information and the second sub-block;

[0033] identifying the speech information according to the syntax structure to obtain the identification result.

[0034] In a second aspect, the embodiments of the present application further provide a speech recognition device, comprising:

[0035] a splitting unit configured to obtain speech information and split the speech information into a plurality of first sub-blocks;

[0036] a determining unit configured to determine a first sub-block association relationship, wherein the first sub-block association relationship is determined based on an attention parameter calculated by a preset self-attention algorithm on the first sub-blocks, and the first sub-block association relationship is used to represent an association degree between each first sub-block;

[0037] a combining unit configured to group and combine each first sub-block according to the first sub-block association relationship into a second sub-block;

[0038] an identifying unit configured to identify the speech information according to the second sub-block to obtain an identification result.

[0039] Optionally, the splitting unit is further configured to:

[0040] acquire corresponding spectral information according to the voice information, wherein the spectral information is obtained after the voice information is processed by a Fourier transform algorithm;

[0041] perform a convolution operation according to the spectral information to obtain a voice feature vector, and determine the first sub-block according to the voice feature vector.

[0042] Optionally, the determining unit is further configured to:

[0043] calculate attention parameters between each two of the first sub-blocks by a preset self-attention algorithm;

[0044] determine a plurality of the attention parameters corresponding to each of the first sub-blocks as association parameters of the corresponding first sub-block;

[0045] determine a target number of first sub-blocks as associated sub-blocks according to the association parameters of each of the first sub-blocks, and record the relationship between the associated sub-blocks as the first sub-block association relationship, wherein the target number is determined according to the numerical value of the association parameters.

[0046] Optionally, the association parameters include a plurality of attention parameters.

[0047] The determining unit is further configured to:

[0048] input all the first sub-blocks corresponding to the voice information from two input ends of a self-attention algorithm matrix respectively;

[0049] perform inner product calculation according to the self-attention algorithm matrix and the plurality of first sub-blocks input from the two input ends respectively, to obtain the attention parameters of each of the remaining first sub-blocks corresponding to the current first sub-block respectively, wherein the remaining first sub-blocks are the remaining sub-blocks of all the first sub-blocks corresponding to the voice information except the current first sub-block.

[0050] obtain the association parameters of the current first sub-block according to the attention parameters of each of the remaining first sub-blocks corresponding to the current first sub-block respectively.

[0051] Optionally, the determining unit is further configured to:

[0052] perform sorting operation on the plurality of attention parameters included in the association parameters of the first sub-block to obtain a sorting sequence, wherein the sorting sequence is arranged in descending order based on the attention parameters.

[0053] Obtain a target number of attention parameters from the starting position of the ordered sequence as target attention parameters;

[0054] According to a correspondence between the target attention parameters and the remaining first sub-blocks, determine a target sub-block corresponding to the target attention parameters, and determine the current first sub-block and the target sub-block as the associated sub-blocks, and record the relationship of the associated sub-blocks as the first sub-block association relationship.

[0055] Optionally, the combining unit is further configured to:

[0056] Combine the associated sub-blocks according to the first sub-block association relationship to obtain the second sub-blocks.

[0057] After grouping and combining each of the first sub-blocks into a second sub-block according to the first sub-block association relationship, the method further comprises:

[0058] Accumulate the attention parameters of each of the first sub-blocks in the associated sub-blocks to obtain the attention parameters corresponding to the second sub-blocks.

[0059] Determine a second sub-block association relationship according to the attention parameters of each of the second sub-blocks, and the second sub-block association relationship is positively correlated with the values of the attention parameters.

[0060] Optionally, the identifying unit is further configured to:

[0061] Determine a syntax structure of the speech information according to the correspondence between the second sub-blocks and the first sub-blocks, and the correspondence between the speech information and the second sub-blocks.

[0062] Identify the speech information according to the syntax structure to obtain the identification result.

[0063] In order to achieve the above-mentioned purpose, according to the third aspect of the embodiment of the present application, a storage medium is provided, the storage medium comprises a stored program, wherein when the program runs, the device where the storage medium is located is controlled to execute the speech recognition method in any one of the first aspect.

[0064] In order to achieve the above-mentioned purpose, according to the fourth aspect of the embodiment of the present application, a device is provided, the device comprises at least one processor, and at least one memory connected with the processor, bus; wherein the processor, memory complete mutual communication through the bus; the processor is used to call the program instruction in the memory, and executes the speech recognition method in any one of the first aspect.

[0065] By the technical solution, the voice recognition method and device provided by the embodiment of the present application can solve the problem of poor accuracy of voice recognition in the current voice recognition process. The embodiment of the present application first acquires voice information, and splits the voice information into a plurality of first sub-blocks. Then, the first sub-block association relationship is determined. After that, each first sub-block is grouped and combined into a second sub-block according to the first sub-block association relationship. Finally, the voice information is recognized according to the second sub-block to obtain a recognition result, thereby realizing the voice recognition function. In the above solution, the first sub-block association relationship is determined based on the attention parameters calculated by the preset self-attention algorithm. The first sub-block association relationship is used to represent the association degree between each first sub-block. This ensures that the second sub-block can be constructed based on the association degree between the first sub-blocks in the process of forming the second sub-block from the first sub-blocks. As a result, the content contained in the second sub-block is the first sub-block with a relatively close second sub-block association relationship. Then, it can be ensured that the structure and relationship between the contents in the voice information can be determined based on the first sub-block association relationship between the first sub-blocks in the execution process of the voice recognition method of the embodiment of the present application, thereby reflecting the internal structure and relationship of the entire voice information. This enables the embodiment of the present application to deconstruct the voice according to the grammatical structure in the voice information during recognition, and to identify on this basis. This can avoid the problem of poor accuracy of the recognition result caused by not considering the grammatical structure in the current conventional recognition process which only establishes a text mapping based on audio features for recognition. The accuracy of recognition can be improved. In addition, the first sub-block association relationship between the first sub-blocks in the embodiment of the present application is determined based on the attention parameters calculated by the preset self-attention algorithm. This ensures that the first sub-block association relationship between the first sub-blocks can be determined according to the attention parameters. Since the preset self-attention algorithm can calculate based on the similarity between texts and words, the obtained attention parameters can reflect the degree of similarity between the first sub-blocks. Then, it provides a more scientific result for the determination of the first sub-block association relationship, thereby ensuring the accuracy of forming the second sub-block from the first sub-blocks based on the first sub-block association relationship, and thereby ensuring the accuracy of the recognition result as a whole.

[0066] The above description is only a summary of the technical solutions of the embodiments of the present application. In order to more clearly understand the technical means of the embodiments of the present application, the embodiments of the present application can be implemented according to the content of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific embodiments of the embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0067] Various other advantages and benefits will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiment. The accompanying drawings are included to provide a description of preferred embodiments, and are not intended to limit the scope of the application. Furthermore, the drawings are included to illustrate a preferred embodiment of the application, and, therefore, are not to be considered limitations on the description of the application. In the drawings:

[0068] Figure 1 A flow chart of a speech recognition method provided by an embodiment of the application is shown in FIG. 1;

[0069] Figure 2 A schematic diagram of a correction process when a speech recognition method provided by an embodiment of the application is executed is shown in FIG. 2;

[0070] Figure 3 A block diagram of a speech recognition device provided by an embodiment of the application is shown in FIG. 3;

[0071] Figure 4 A block diagram of a device for speech recognition provided by an embodiment of the application is shown in FIG. 4. DETAILED DESCRIPTION

[0072] Exemplary embodiments of the application will be described herein below with reference to the accompanying drawings. Although exemplary embodiments of the application are shown in the drawings, it should be understood that the application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0073] To solve the problem of poor accuracy of speech recognition in the current speech recognition process, an embodiment of the application provides a speech recognition method, as shown in FIG. 1, which includes: Figure 1

[0074] 101, obtaining speech information and splitting the speech information into a plurality of first sub-blocks.

[0075] In this embodiment, when splitting the speech information, the splitting can be performed based on the characteristics of the speech information, for example, when splitting according to a preset time length, 5 ms can be selected for splitting, so that a plurality of 5 ms long speech segments are obtained as first sub-blocks.

[0076] Of course, in actual applications, the splitting can also be performed based on the language category of the speech information. For example, when the speech information is "I ate instant noodles today" in Chinese, the first sub-blocks can be obtained by splitting according to the characters, i.e., "I" "today" "eat" "instant" "noodles".

[0077] ​It should be noted that the process of splitting the voice information into a plurality of first sub-blocks in the embodiment can be in any manner required by the user, which is not limited herein and can be selected as required.

[0078] 102. Determine the first sub-block association relationship.

[0079] The first sub-block association relationship is determined based on the attention parameters calculated by the preset self-attention algorithm for the first sub-blocks, and is used to represent the association degree between each of the first sub-blocks.

[0080] In the embodiment, the preset self-attention algorithm, also known as self-attention mechanism, is an algorithm derived from image processing technology and is commonly used in natural language processing. The main purpose of the algorithm is to convert a word in a sentence into a corresponding spatial vector, and to calculate based on the spatial vector to determine the attention degree, i.e. the attention parameter, between the other words and the word.

[0081] In the embodiment, when determining the first sub-block association relationship between the first sub-blocks, the preset self-attention algorithm is mainly used, i.e. each first sub-block is calculated to determine the attention parameters between the remaining first sub-blocks and the current first sub-block, and so on, so as to obtain the attention parameters between each first sub-block and the remaining first sub-blocks. Since the size of the attention parameter can represent the closeness of the association between the first sub-blocks, the first sub-block association relationship can reflect the association degree between each first sub-block and the other first sub-blocks.

[0082] 103. Group and combine each of the first sub-blocks into a second sub-block according to the first sub-block association relationship.

[0083] Since the first sub-block association relationship can reflect the association degree between the first sub-blocks, the first sub-blocks with a higher association degree can be combined to obtain the corresponding second sub-block. Taking the example of the previous step, among the first sub-blocks "me", "today", "eat", "side", "face", the first sub-block association relationship of the first sub-block "today" can be determined by the previous step, and based on the association degree with other first sub-blocks contained therein, it can be found that the association degree between the first sub-block "today" and the first sub-block "today" is higher. Therefore, the first sub-block "today" and the first sub-block "today" can be combined to obtain the second sub-block "today".

[0084] 104. Identify the voice information according to the second sub-block to obtain an identification result.

[0085] Since the second sub-block is composed of the first sub-block with higher relevance, when the second sub-block is obtained, the combination relationship between the first sub-block and the second sub-block in the speech information can be determined in fact. From the perspective of linguistics, the relationship between the parts in a sentence is actually the embodiment of the grammatical structure. Therefore, in the embodiment, the speech recognition based on the first sub-block and the second sub-block can ensure that the recognition can be performed according to the grammatical structure of the sentence in the actual recognition process, so that the obtained recognition result also conforms to the grammatical structure of the sentence, thereby making the recognition result have better accuracy.

[0086] It should be noted that in the process of recognition based on the second sub-block, the text information is obtained based on the existing speech-to-text recognition mode of speech recognition, that is, the process of converting audio into text is consistent with the existing conversion mode, and will not be described in detail here.

[0087] Based on the above scheme, for the problem of poor accuracy of speech recognition in the current speech recognition process, the method of the embodiment can first acquire speech information, and split the speech information into a plurality of first sub-blocks; then, determine the first sub-block association relationship; then, group and combine each first sub-block into a second sub-block according to the first sub-block association relationship; finally, identify the speech information according to the second sub-block to obtain the recognition result, thereby realizing the speech recognition function. In the above scheme, since the first sub-block association relationship is determined based on the attention parameters calculated by the preset self-attention algorithm on the first sub-block, the first sub-block association relationship is used to represent the association degree between each first sub-block. This ensures that the second sub-block can be constructed based on the association degree between the first sub-blocks during the process of forming the second sub-block from the first sub-blocks, so that the second sub-block contains first sub-blocks with relatively close association degrees. In turn, it can be ensured that the structure and relationship between the contents in the speech information can be determined based on the first sub-block association relationship during the execution of the speech recognition method in the embodiment of the application, thereby reflecting the internal structure and relationship of the entire speech information. This enables the embodiment of the application to deconstruct the speech according to the grammatical structure in the speech information during recognition, and to identify on this basis, which can avoid the problem of poor accuracy of the recognition result caused by not considering the grammatical structure during the current conventional recognition process based on only the audio features to establish a text mapping for recognition, and can improve the accuracy of recognition. In addition, since the first sub-block association relationship between the first sub-blocks in the embodiment of the application is determined based on the attention parameters calculated by the preset self-attention algorithm, it is ensured that the first sub-block association relationship between the first sub-blocks can be determined according to the attention parameters. Since the preset self-attention algorithm can calculate based on the similarity between texts and words, the obtained attention parameters can reflect the degree of similarity between the first sub-blocks, thereby providing a more scientific result for the determination of the first sub-block association relationship, thereby ensuring the accuracy of the formation of the second sub-block from the first sub-blocks based on the first sub-block association relationship, and thereby overall ensuring the accuracy of the recognition result.

[0088] In an alternative embodiment, since the speech information is a kind of audio, some processing is required during the determination of the first sub-blocks in order to obtain each first sub-block. Therefore, the step 101 of the foregoing embodiment of splitting the speech information into a plurality of first sub-blocks comprises:

[0089] First, the corresponding spectrum information is obtained according to the speech information, wherein the spectrum information is obtained after the speech information is processed by the Fourier transform algorithm;

[0090] Then, a convolution operation is performed according to the spectrum information to obtain a speech feature vector, and the first sub-blocks are determined according to the speech feature vector.

[0091] In a specific implementation process, when a piece of voice (voice information) is obtained, the corresponding spectral information Y can be obtained through Fourier transform operation ∈D*T wherein T and D represent the time and frequency spectrum characteristics of the spectral information, respectively. Then the spectral information Y is convolved through a convolution layer to obtain better representation of the spectral information, i.e., a voice representation vector is obtained. The voice representation vector can be understood as a matrix form of data obtained after convolution of the spectrum. At this time, each part of the matrix can be split according to time (equivalent to blocking according to syllables) to obtain a plurality of blocks, and each block is determined as a first sub-block.

[0092] In the above process, since the voice representation vector obtained through convolution operation after the voice information is processed by the Fourier transform algorithm to obtain the spectral information is determined as the first sub-block, it is ensured that the vectorization splitting can be combined with the audio characteristics of the voice in the process of obtaining the first sub-block, thereby ensuring that the obtained first sub-block can conform to the audio characteristics of the voice information, thereby ensuring that the split first sub-block is relatively accurate and can avoid the problem of splitting a word or a word, thereby laying a foundation for the accuracy of subsequent overall voice recognition.

[0093] In an optional embodiment, the step 102 of determining the first sub-block association relationship in the foregoing embodiment comprises:

[0094] First, the attention parameters between each two first sub-blocks are calculated by a preset self-attention algorithm. In this embodiment, the calculation method of the preset self-attention algorithm is consistent with the prior art, which is not described here. It should be noted that in the calculation of the attention parameters between the first sub-blocks, in order to distinguish each first sub-block, the position information of each first sub-block can also be determined based on the voice information. The position information is used to represent the relative position between each two first sub-blocks. Then each first sub-block is labeled based on the position information, so that the first sub-block to which the subsequently calculated attention parameter belongs is avoided from being confused.

[0095] Then, according to the association parameters of each first sub-block, a target number of first sub-blocks are determined as associated sub-blocks, and the relationship between the associated sub-blocks is recorded in numerical form as the first sub-block association relationship. The target number is determined according to the size of the association parameter. In this step, the process of determining the target number of first sub-blocks as associated sub-blocks can be the first sub-blocks with the highest attention parameters in the association parameters of each first sub-block. The specific target number can be selected according to user needs, which is not limited here.

[0096] For example, in the embodiment, the mask-attention module is further provided, and the mask-attention module is used to retain only the three highest attention scores (i.e., the target number). Meanwhile, the same operation is performed on each first sub-block by multiple mask-attention modules, so that each mask-attention module filters the three highest attention scores for each first sub-block, and the different positions of the voice are focused on, and the most relevant sub-blocks of the different positions of the first sub-blocks are determined. Details are as follows.

[0097] A = MA(D)

[0098] wherein A is the correlation parameter, MA is the target number selection algorithm of the mask-attention module, and D is the attention parameter. In the process of executing MA, the three highest attention scores are retained, and the other scores are all masked as 0, so that the attention parameters of the first sub-blocks with high correlation are highlighted. In actual application, the MA selection algorithm is specifically executed as follows: after a user sets a value k, the MA selection algorithm selects the maximum k values from multiple values and clears the remaining values. Since the MA algorithm is used to process the attention parameter D, that is, when there are multiple attention parameters D1 (0.4), D2 (0.35), D3 (0.01), D4 (0.0.2) and D5 (0.39), and k is 3, the MA selection algorithm retains only the three highest values D1 (0.4), D2 (0.35) and D5 (0.39), and the remaining D3 (0.01) and D4 (0.0.2) are cleared to D3 (0) and D4 (0).

[0099] In addition, according to the above formula, the correlation parameter A is determined by the attention parameter, and the correlation degree can be reflected to some extent. In order to further improve the reflection effect of the correlation degree, the A can be processed in the following manner:

[0100] M = softmax A

[0101] that is,

[0102] M = softmax(MA(D))

[0103] Wherein, M is the final correlation parameter, the softmax function operation can convert the value to the value between 0 and 1, that is, the value can be converted by the softmax function, which can intuitively reflect the actual correlation degree of each attention parameter (0-1, equivalent to percentage), so as to improve the effect of intuitively reflecting the correlation degree of the correlation parameter.

[0104] In an optional embodiment, since the correlation parameter is determined based on the preset self-attention algorithm for the first sub-block operation, in some cases, the attention parameter can be directly constructed as the correlation parameter, therefore, the correlation parameter contains multiple attention parameters, that is, the correlation parameter of each first sub-block is actually formed by the attention parameter of the first sub-block and each other first sub-block.

[0105] Based on this, the attention parameter between each two first sub-blocks in the foregoing step is calculated by the preset self-attention algorithm, and the execution can include:

[0106] Firstly, the whole first sub-block corresponding to the voice information is input from two input ends of the self-attention algorithm matrix respectively, which can be shown in Table 1.

[0107] Table 1

[0108]

[0109]

[0110] Then, according to the self-attention algorithm matrix and the multiple first sub-blocks input from two input ends respectively, inner product calculation is performed respectively to obtain the attention parameter of each remaining first sub-block corresponding to the current first sub-block, wherein the remaining first sub-block is the remaining sub-block in the whole first sub-block corresponding to the voice information except the current first sub-block, which can be shown in Table 2.

[0111] Table 2

[0112] Today 0.3 0.29 0.02 0.03 0.02 Hello 0.4 0.3 0.04 0.01 0.04 Today 0.01 0.01 0.3 0.29 0.03 Hello 0.02 0.02 0.4 0.3 0.06 Figure 2 0.01 0.01 0.03 0.04 0.01 Figure 2 Figure 1 Figure 1 Figure 3 Figure 4

[0113] Finally, the correlation parameter of the current first sub-block is obtained according to the attention parameter of each remaining first sub-block corresponding to the current first sub-block.

[0114] In addition, based on the foregoing embodiment, after obtaining the attention parameter as shown in Table 2, it can also be filtered based on the MA selection algorithm, so as to only retain the larger value in the attention parameter, so as to more intuitively reflect the first sub-block with high correlation degree. For example, when the MA selection algorithm retains only the three largest values, which can be shown in Table 3.

[0115] Table 3

[0116]

[0117]

[0118] Further, in the specific implementation process, the way of calculating the attention parameter can be as follows:

[0119]

[0120] wherein Q is the same as K, both of which are equal to the input speech representation vector Y, and the multiplication thereof constitutes a self-attention algorithm matrix, and d is the feature dimension of the speech representation vector Y.

[0121] Based on this, in combination with the foregoing embodiments, it can be known that the obtained correlation parameter can be:

[0122]

[0123] Since the attention parameter can be directly input and calculated from the self-attention algorithm matrix according to the speech information in this step, and the correlation parameter is determined based on the attention parameter, the function of calculating the correlation parameter of the first sub-block by the preset attention algorithm is realized. Since the calculated correlation parameter can reflect the degree of correlation between each first sub-block, the accuracy of the subsequent recognition result is guaranteed, thereby improving the accuracy of the recognition result.

[0124] In an optional embodiment, in the foregoing step, after the correlation parameter of the first sub-block is determined, based on the foregoing description, the correlation parameter is obtained based on the attention parameter obtained by the preset self-attention algorithm, and can reflect the degree of correlation between the first sub-blocks. Based on this, in the foregoing step, the target number of first sub-blocks is determined as the correlation sub-blocks according to the correlation parameter of each first sub-block, and the relationship between the correlation sub-blocks is recorded as the first sub-block correlation relationship, which specifically includes:

[0125] The plurality of attention parameters contained in the correlation parameter of the first sub-block are sorted to obtain a sorting sequence, and the sorting sequence is arranged in descending order based on the attention parameters;

[0126] The target number of attention parameters from the starting position of the sorting sequence are obtained as target attention parameters;

[0127] According to the corresponding relationship between the attention parameter and the remaining first sub-blocks, the target sub-block corresponding to the target attention parameter is determined, and the current first sub-block and the target sub-block are determined as the correlation sub-blocks, and the relationship of the correlation sub-blocks is recorded as the first sub-block correlation relationship.

[0128] Since there can be multiple attention parameters in the determined association parameters, the sequence obtained after sorting needs to be further screened to determine the first sub-block corresponding to one or several attention parameters with the highest association degree with the current first sub-block, so as to ensure that the first sub-block association relationship finally obtained contains one or several first sub-blocks with the highest relative association degree, facilitating the assembly of the second sub-block. It should be noted that the number of selected sub-blocks can be determined based on actual needs, and is not limited herein.

[0129] In an optional embodiment, the grouping and combining of each first sub-block according to the first sub-block association relationship to form a second sub-block comprises:

[0130] combining the associated sub-blocks according to the first sub-block association relationship to obtain the second sub-block;

[0131] Based on this, after the aforementioned step of grouping and combining each first sub-block according to the first sub-block association relationship to form a second sub-block, the method can further comprise a determination process of the association relationship of the second sub-block, i.e. determining the second sub-block association relationship, which can comprise:

[0132] adding up the attention parameters of each first sub-block in the associated sub-blocks to obtain the attention parameters corresponding to the second sub-block;

[0133] determining the second sub-block association relationship according to the attention parameters of each second sub-block, wherein the second sub-block association relationship is positively correlated with the value of the attention parameters.

[0134] It should be noted that in actual application, third sub-blocks, fourth sub-blocks, etc. can also be included, that is, sub-blocks can be continuously combined based on the method of this step to form new sub-blocks, thereby forming a hierarchical structure, such as Figure 1As shown, in the present example, since the sentence is relatively simple, a two-layer structure is formed (the uppermost layer has actually been combined into this speech information whole sentence), the first layer is six first sub-blocks of "today", "you", "good", and "ah", and based on the method of the foregoing steps, it can be determined that "today" and "you" are the associated sub-blocks with a higher degree of association in the first sub-block, and "good" and "ah" are the sub-blocks with a higher degree of association, so the second sub-blocks corresponding to the arrows can form two second sub-blocks of "today" and "hello" in the second layer. Of course, in actual application, when the sentence is complex and the content is more, a third layer can also be formed, and at this time, the second sub-blocks in the second layer can also determine the degree of association between each other, that is, the second sub-block association relationship, in the same way as described above, and of course, the association parameters can also be directly added by the association parameters of the first sub-blocks to directly obtain the association parameters between the second sub-blocks and other sub-blocks. Specifically, when the present embodiment is executed, it can be:

[0135] The embodiment of the present application can add the association parameters of the first sub-blocks with the first sub-block association relationship in the first sub-blocks, and directly add the association parameters of the second sub-blocks in the next layer through the jump link.

[0136]

[0137] Specifically, the accumulation method can be: first, the speech information corresponding to the spectrum information is convolved to obtain a speech feature vector, which can be the same as the process of determining the first sub-block in the foregoing embodiment. Since the obtained speech feature vector can be understood as a matrix form obtained by convolving the spectrum information, and the attention parameters between the first sub-blocks have been determined based on the foregoing steps, in the present embodiment, the attention parameters of the first sub-blocks can be assigned to the speech feature vector, and the specific assignment method can be to add the attention parameters of the first sub-blocks to the corresponding positions of the speech feature vector according to the position identifier, thereby obtaining a first vector that can reflect the relationship between the first sub-blocks. Then, using the position identifier of the first sub-blocks contained in the second sub-block, the first vector is split into corresponding second sub-block vectors, and the attention parameters between the second sub-block vectors are calculated in the same way as the attention parameters of the first sub-blocks, that is, by inputting the second sub-block vector to both ends of the self-attention algorithm matrix, and then performing an inner product operation to obtain the corresponding self-attention parameters.

[0138] Therefore, in the present scheme, since the calculated attention parameters actually contain the attention parameters of the first sub-block, the accumulation effect of the attention parameters of the first sub-block is achieved, so that the attention parameters between the second sub-blocks are also affected by the attention parameters of the first sub-block, thereby improving the accuracy of the calculation result of the attention parameters of the second sub-block, and based on the above scheme, the syntax tree structure can be formed through multiple combinations and stacking between sub-blocks.

[0139] In some optional embodiments, the step 104 of the foregoing embodiment, wherein the speech information is recognized according to the second sub-block to obtain a recognition result, when executed, can include:

[0140] First, the syntax structure of the speech information is determined according to the corresponding relationship between the second sub-block and the first sub-block, and the corresponding relationship between the speech information and the second sub-block.

[0141] Then, the speech information is recognized according to the syntax structure to obtain the recognition result.

[0142] As shown in the foregoing embodiments, Figure 1 The first layer only pays attention to the local attention score, i.e., the first sub-block correlation between each first sub-block, for example, the first mask-attention module of the first layer only pays attention to the attention scores of “today” and “day”, and the other positions are all 0, and the second mask-attention module of the first layer only pays attention to the attention scores of “you” and “good”. At the same time, the attention result of the first layer is transmitted to the next layer through a skip link (i.e., the correlation parameters of the first sub-block are superimposed into the corresponding second sub-block), so that the next layer (the second sub-block) obtains more global syntax information. In this way, a hierarchical structure can be formed through the relationship between the first sub-block and the second sub-block, so that the recognition process conforms to the syntax structure, and the accuracy of the recognition result is improved.

[0143] Further, as an implementation of the method shown in the foregoing ​ The embodiment of the present application also provides a speech recognition device for implementing the method shown in the foregoing ​ The device embodiment corresponds to the foregoing method embodiment, and for the sake of easy reading, the details of the foregoing method embodiment will not be described one by one, but it should be clear that the device in the present embodiment can correspondingly implement all the contents in the foregoing method embodiment. As shown in the foregoing ​ The device includes a splitting unit 31, a determining unit 32, a combining unit 33, and a recognizing unit 34, wherein

[0144] The splitting unit 31 can be configured to acquire voice information and split the voice information into a plurality of first sub-blocks.

[0145] The determining unit 32 can be configured to determine a first sub-block association relationship, wherein the first sub-block association relationship is determined based on an attention parameter calculated by a preset self-attention algorithm on the first sub-blocks, and the first sub-block association relationship can be used to represent an association degree between each of the first sub-blocks.

[0146] The combining unit 33 can be configured to group and combine each of the first sub-blocks into a second sub-block according to the first sub-block association relationship.

[0147] The recognizing unit 34 can be configured to recognize the voice information according to the second sub-block to obtain a recognition result.

[0148] According to the technical scheme, the voice recognition method and device are provided, and for the problem of poor accuracy of voice recognition in the current voice recognition process, the voice information is first acquired, and the voice information is split into a plurality of first sub-blocks; then, the first sub-block association relationship is determined; then, each first sub-block is grouped and combined into a second sub-block according to the first sub-block association relationship; finally, the voice information is recognized according to the second sub-block to obtain a recognition result, so as to realize the voice recognition function. In the above scheme, the first sub-block association relationship is determined based on the attention parameters calculated by the preset self-attention algorithm, and the first sub-block association relationship is used to represent the association degree between each first sub-block. This ensures that the second sub-block can be constructed based on the association degree between the first sub-blocks in the process of combining the first sub-blocks into the second sub-block, so that the content contained in the second sub-block is the first sub-block with a relatively close association degree. Then, it can be ensured that the structure and relationship between the contents in the voice information can be determined based on the first sub-block association relationship between the first sub-blocks in the execution process of the voice recognition method, so as to reflect the internal structure and relationship of the entire voice information. This enables the voice to be deconstructed according to the syntax structure in the voice information during recognition, and the recognition can be performed on this basis. This can avoid the problem of poor accuracy of the recognition result caused by not considering the syntax structure in the current conventional recognition process based on only the audio features to establish the text mapping for recognition, and can improve the accuracy of the recognition. In addition, in the present application, the first sub-block association relationship between the first sub-blocks is determined based on the attention parameters calculated by the preset self-attention algorithm. This ensures that the first sub-block association relationship between the first sub-blocks can be determined according to the attention parameters. Since the preset self-attention algorithm can calculate the similarity between the text and the word, the obtained attention parameters can reflect the degree of similarity between the first sub-blocks. Then, a more scientific result is provided for the determination of the first sub-block association relationship, so as to ensure the accuracy of the combination of the first sub-blocks into the second sub-block based on the first sub-block association relationship, and thus the accuracy of the recognition result is ensured as a whole.

[0149] The processor includes a core, and the core retrieves corresponding program units from the memory. The core can be set to one or more, and the accuracy of voice recognition in the current voice recognition process can be improved by adjusting the core parameters.

[0150] The present application provides a storage medium having a program stored thereon, which is executed by a processor to implement the voice recognition method.

[0151] The present application provides a processor for running a program, wherein the program is executed to perform the voice recognition method.

[0152] The embodiment of the present application provides a device 40, such as ​ As shown, the device comprises at least one processor 401 and at least one memory 402 connected with the processor, and a bus 403; wherein the processor 401 and the memory 402 complete mutual communication through the bus 403; the processor 401 is used for calling program instructions in the memory to execute the voice recognition method.

[0153] The device herein can be a server, a PC, a PAD, a mobile phone and the like.

[0154] The present application also provides a computer program product, when executed on a process management device, is suitable for executing the program of the following method steps: obtaining voice information, and splitting the voice information into a plurality of first sub-blocks; determining a first sub-block association relationship, wherein the first sub-block association relationship is determined based on attention parameters calculated by a preset self-attention algorithm on the first sub-block, and the first sub-block association relationship is used to represent the association degree between each first sub-block; grouping and combining each first sub-block into a second sub-block according to the first sub-block association relationship; and recognizing the voice information according to the second sub-block to obtain a recognition result.

[0155] Further, the splitting of the voice information into a plurality of first sub-blocks comprises:

[0156] According to the voice information, corresponding spectrum information is obtained, wherein the spectrum information is obtained after the voice information is processed by a Fourier transform algorithm;

[0157] A convolution operation is performed according to the spectrum information to obtain a voice feature vector, and the first sub-block is determined according to the voice feature vector.

[0158] Further, the determination of the first sub-block association relationship comprises:

[0159] The attention parameters between each two first sub-blocks are calculated by a preset self-attention algorithm;

[0160] A plurality of attention parameters corresponding to each first sub-block are determined as association parameters corresponding to the first sub-block;

[0161] According to the association parameters of each first sub-block, a target number of first sub-blocks are determined as associated sub-blocks, and the relationship between the associated sub-blocks is recorded as the first sub-block association relationship, wherein the target number is determined according to the numerical value of the association parameters.

[0162] Further, the association parameters comprise a plurality of attention parameters;

[0163] The attention parameters between each two first sub-blocks are calculated by a preset self-attention algorithm, including:

[0164] The first sub-blocks corresponding to the voice information are respectively input from two input ends of a self-attention algorithm matrix;

[0165] According to the self-attention algorithm matrix and the first sub-blocks input from the two input ends, inner product calculation is respectively performed to obtain the attention parameters of each remaining first sub-block corresponding to the current first sub-block, wherein the remaining first sub-blocks are the remaining sub-blocks of the first sub-blocks corresponding to the voice information except the current first sub-block;

[0166] According to the attention parameters of the current first sub-block corresponding to each remaining first sub-block, the correlation parameters of the current first sub-block are obtained.

[0167] Further, the target number of first sub-blocks are determined as correlation sub-blocks according to the correlation parameters of each first sub-block, and the relationship between the correlation sub-blocks is recorded as the first sub-block correlation relationship, including:

[0168] The multiple attention parameters contained in the correlation parameters of the first sub-blocks are sorted to obtain a sorting sequence, and the sorting sequence is arranged in descending order based on the attention parameters;

[0169] Target attention parameters are obtained from the starting position of the sorting sequence;

[0170] According to the correspondence between the attention parameters and the remaining first sub-blocks, target sub-blocks corresponding to the target attention parameters are determined, and the current first sub-block and the target sub-block are determined as the correlation sub-blocks, and the relationship of the correlation sub-blocks is recorded as the first sub-block correlation relationship.

[0171] Further, the first sub-blocks are grouped and combined into second sub-blocks according to the first sub-block correlation relationship, including:

[0172] The correlation sub-blocks are combined according to the first sub-block correlation relationship to obtain the second sub-blocks;

[0173] After the first sub-blocks are grouped and combined into second sub-blocks according to the first sub-block correlation relationship, the method further includes:

[0174] The attention parameters of each first sub-block in the correlation sub-blocks are accumulated to obtain the attention parameters corresponding to the second sub-blocks;

[0175] The second sub-block correlation is determined according to the attention parameter of each second sub-block, and the second sub-block correlation is positively correlated with the value of the attention parameter.

[0176] Further, the second sub-block is used to identify the speech information to obtain an identification result.

[0177] The syntax structure of the speech information is determined according to the corresponding relationship between the second sub-block and the first sub-block and the corresponding relationship between the speech information and the second sub-block.

[0178] The speech information is identified according to the syntax structure to obtain the identification result.

[0179] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable processing device to produce a machine, so that the instructions executed by the computer or other programmable processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. ​ The functions specified in one flow or multiple flows and / or blocks ​ The functions specified in one flow or multiple flows and / or blocks

[0180] In a typical configuration, the device includes one or more processors (CPU), memory, and a bus. The device can also include an input / output interface, a network interface, and the like.

[0181] The memory can include a non-persistent memory in a computer readable medium, a random access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip. The memory is an example of the computer readable medium.

[0182] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0183] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0184] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0185] The above merely provides embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A voice recognition method, characterized by, The method comprises: acquiring voice information and splitting the voice information into a plurality of first sub-blocks; determining a first sub-block association relationship, wherein the first sub-block association relationship is determined based on attention parameters calculated by a preset self-attention algorithm, and the first sub-block association relationship is used to represent the association degree between each first sub-block; grouping each first sub-block according to the first sub-block association relationship, and combining first sub-blocks with high association degrees to form second sub-blocks; recognizing the voice information according to the second sub-blocks to obtain a recognition result.

2. The method of claim 1, wherein, The splitting of the voice information into a plurality of first sub-blocks comprises: acquiring corresponding spectral information from the voice information, wherein the spectral information is obtained after the voice information is processed by a Fourier transform algorithm; performing a convolution operation on the spectral information to obtain a voice feature vector, and determining the first sub-blocks according to the voice feature vector.

3. The method of claim 1, wherein, The determination of the first sub-block association relationship comprises: calculating attention parameters between each two first sub-blocks by a preset self-attention algorithm; determining a plurality of attention parameters corresponding to each first sub-block as association parameters of the first sub-block; determining a target number of first sub-blocks as associated sub-blocks according to the association parameters of each first sub-block, and recording the relationship between the associated sub-blocks as the first sub-block association relationship, wherein the target number is determined according to the numerical size of the association parameters.

4. The method of claim 3, wherein, The calculation of the attention parameters between each two first sub-blocks by the preset self-attention algorithm comprises: inputting all first sub-blocks corresponding to the voice information from two input ends of a self-attention algorithm matrix respectively; performing inner product calculation on the self-attention algorithm matrix and a plurality of first sub-blocks input from the two input ends respectively to obtain the attention parameters of each remaining first sub-block corresponding to the current first sub-block respectively, wherein the remaining first sub-blocks are the remaining first sub-blocks other than the current first sub-block among all first sub-blocks corresponding to the voice information; obtaining the association parameters of the current first sub-block according to the attention parameters of each remaining first sub-block corresponding to the current first sub-block respectively.

5. The method of claim 4, wherein, The determination of a target number of first sub-blocks as associated sub-blocks according to the association parameters of each first sub-block and the recording of the relationship between the associated sub-blocks as the first sub-block association relationship comprises: performing sorting operation on a plurality of attention parameters contained in the association parameters of the first sub-blocks to obtain a sorting sequence, wherein the sorting sequence is arranged in descending order based on the attention parameters; obtaining the target number of attention parameters from the starting position of the sorting sequence as target attention parameters; determining target sub-blocks corresponding to the target attention parameters according to the corresponding relationship between the attention parameters and the remaining first sub-blocks; determining the current first sub-block and the target sub-block as the associated sub-blocks, and recording the relationship of the associated sub-blocks as the first sub-block association relationship.

6. The method of claim 5, wherein, The grouping and combining each first sub-block into a second sub-block according to the first sub-block association relationship comprises: combining the associated sub-blocks according to the first sub-block association relationship to obtain the second sub-block; After the grouping and combining each first sub-block into a second sub-block according to the first sub-block association relationship, the method further comprises: accumulating the attention parameters of each first sub-block in the associated sub-blocks to obtain the attention parameters corresponding to the second sub-block; determining a second sub-block association relationship according to the attention parameters of each second sub-block, the second sub-block association relationship being positively correlated with the values of the attention parameters.

7. The method according to any one of claims 1 to 6, characterized in that, The identifying the speech information according to the second sub-block to obtain an identification result comprises: determining a syntax structure of the speech information according to the correspondence between the second sub-block and the first sub-block and the correspondence between the speech information and the second sub-block; identifying the speech information according to the syntax structure to obtain the identification result.

8. A speech recognition apparatus characterized by comprising: comprises: a splitting unit configured to obtain speech information and split the speech information into a plurality of first sub-blocks; a determining unit configured to determine a first sub-block association relationship, wherein the first sub-block association relationship is determined based on the attention parameters of the first sub-blocks calculated by a preset self-attention algorithm, and the first sub-block association relationship is used to represent the association degree between each first sub-block; a combining unit configured to group each first sub-block according to the first sub-block association relationship, and combine the first sub-blocks with high association degrees into a second sub-block; an identifying unit configured to identify the speech information according to the second sub-block to obtain an identification result.

9. A storage medium, characterized by The storage medium comprises a stored program, wherein the program controls the device where the storage medium is located to execute the speech recognition method of any one of claims 1 to 7 when the program is running.

10. An apparatus, comprising: The device comprises at least one processor, at least one memory connected with the processor, and a bus; wherein the processor, the memory and the bus complete mutual communication through the bus; the processor is used to call the program instructions in the memory, and execute the speech recognition method of any one of claims 1 to 7.

Images