Speech recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application relate to a speech recognition method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-recognized speech signal, and generating a third-order speech tensor based on the to-be-recognized speech signal; determining a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor; and determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature. Thus, the redundancy information in the extracted to-be-recognized speech signal can be reduced, the accuracy of speech recognition can be improved, and the user experience can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of speech recognition, and particularly relate to a speech recognition method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In the intelligent speech interaction scenario, speech signal recognition is increasingly attracting widespread attention. In the prior art, a speech recognition model is generally trained to recognize a to-be-recognized speech.

[0003] However, in the intelligent speech interaction scenario, the input and output of sound are often subject to various kinds of interference, such as background noise interference, other speaker sound interference, and the like. This results in various kinds of redundant information in the collected speech signal. The existing speech recognition model is generally obtained by training a corpus. When the speech is recognized, the existing speech recognition model cannot completely remove the redundant information in the speech signal, which results in inaccurate speech recognition and affects the user experience. SUMMARY

[0004] In view of this, to solve the technical problem in the prior art that the existing speech recognition model is generally obtained by training a corpus, and when the speech is recognized, the existing speech recognition model cannot completely remove the redundant information in the speech signal, which results in inaccurate speech recognition and affects the user experience, embodiments of the present application provide a speech recognition method, device, electronic device, and storage medium.

[0005] In a first aspect, an embodiment of the present application provides a speech recognition method, comprising:

[0006] obtaining a to-be-recognized speech signal, and generating a third-order speech tensor based on the to-be-recognized speech signal;

[0007] determining a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor;

[0008] determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature.

[0009] As one possible implementation, the generating of the third-order speech tensor based on the to-be-recognized speech signal comprises:

[0010] extracting an initial frequency spectrum from the to-be-recognized speech signal, wherein the initial frequency spectrum includes time features and frequency features;

[0011] performing identity recognition on the to-be-recognized speech signal to determine a speaker feature corresponding to the to-be-recognized speech signal;

[0012] construct a third-order speech tensor corresponding to the speech signal to be recognized based on the time feature, the frequency feature, and the speaker feature of the speech signal to be recognized.

[0013] As a possible implementation, the determining the speech phase feature corresponding to the speech signal to be recognized from the third-order speech tensor comprises:

[0014] performing non-negative decomposition on the third-order speech tensor to obtain N groups of decomposition matrices; each group of decomposition matrices comprises a time matrix, a frequency matrix, and a speaker matrix, and N is a positive integer;

[0015] determining target time features and target frequency features by using the time matrix, the frequency matrix, and the speaker matrix in the group of decomposition matrices;

[0016] determining the target time features and the target frequency features as the speech phase feature corresponding to the speech signal to be recognized.

[0017] As a possible implementation, the determining target time features and target frequency features by using the time matrix, the frequency matrix, and the speaker matrix in the group of decomposition matrices comprises:

[0018] assigning each group of decomposition matrices a decomposition number according to a decomposition order, and inserting all groups of decomposition matrices into a decomposition matrix queue according to the order of the decomposition numbers;

[0019] selecting and removing the first group of decomposition matrices in the order from the decomposition matrix queue as a preceding matrix group, and performing orthogonal constraint processing on the preceding matrix group to obtain a preceding processed matrix group; the orthogonal constraint processing comprises orthogonal processing on the time matrix and the frequency matrix in the preceding matrix group, and constraint processing on the speaker matrix in the preceding matrix group;

[0020] determining a first decomposition loss value between the preceding processed matrix group and the third-order speech tensor, and determining whether the first decomposition loss value meets a preset convergence condition;

[0021] if the first decomposition loss value does not meet the preset convergence condition, then reselecting and removing the first group of decomposition matrices in the order from the decomposition matrix queue as a subsequent matrix group, performing orthogonal constraint processing on the subsequent matrix group based on the preceding matrix group to obtain a subsequent processed matrix group, determining a second decomposition loss value between the subsequent processed matrix group and the third-order speech tensor, and determining whether the second decomposition loss value meets the preset convergence condition;

[0022] If the second decomposition loss value does not satisfy the preset convergence condition, the post-processing matrix group is recorded as a previous matrix group, and the step of determining whether the second decomposition loss value satisfies the preset convergence condition is returned to be executed until the second decomposition loss value satisfies the preset convergence condition, and the target time feature and the target frequency feature are output.

[0023] As one possible implementation, the determining of the speech recognition result corresponding to the to-be-identified speech signal based on the speech phase feature comprises:

[0024] generating a spectrum graph based on the speech phase feature;

[0025] determining an acoustic unit sequence corresponding to the to-be-identified speech signal according to the spectrum graph;

[0026] determining a text sequence corresponding to the acoustic unit sequence, and determining the text sequence as the recognition result of the to-be-identified speech signal.

[0027] As one possible implementation, the determining of the acoustic unit sequence corresponding to the to-be-identified speech signal according to the spectrum graph comprises:

[0028] matching the spectrum graph with each spectrum graph in a preset speech database to obtain a target spectrum graph, the speech database comprising a plurality of acoustic unit sequences, a spectrum graph corresponding to each acoustic unit sequence, and a corresponding relationship therebetween;

[0029] determining a target acoustic unit sequence corresponding to the target spectrum graph according to the corresponding relationship;

[0030] determining the target acoustic unit sequence as the acoustic unit sequence corresponding to the to-be-identified speech signal.

[0031] As one possible implementation, the determining of the acoustic unit sequence corresponding to the to-be-identified speech signal according to the spectrum graph comprises:

[0032] inputting the spectrum graph into a preset frame model to obtain an encoded sequence after the spectrum graph is framed;

[0033] segmenting the encoded sequence to obtain a plurality of encoded segments;

[0034] For each encoded segment, the encoded segment is matched with encoded segments in a preset encoding library to obtain a target encoded segment, the encoding library including a plurality of encoded segments, an acoustic unit corresponding to each encoded segment, and a corresponding relationship therebetween;

[0035] According to the corresponding relationship, a target acoustic unit corresponding to each target encoded segment is determined to obtain a plurality of target acoustic units;

[0036] The order of the plurality of encoded segments is determined;

[0037] The plurality of target acoustic units are arranged according to the order to obtain an acoustic unit sequence corresponding to the speech signal to be recognized.

[0038] In a second aspect, an embodiment of the present application provides a speech recognition device, and the device comprises:

[0039] An acquisition module is configured to acquire a speech signal to be recognized;

[0040] A generation module is configured to generate a third-order speech tensor based on the speech signal to be recognized;

[0041] A determination module is configured to determine a speech phase feature corresponding to the speech signal to be recognized from the third-order speech tensor;

[0042] An identification module is configured to determine a speech recognition result corresponding to the speech signal to be recognized based on the speech phase feature.

[0043] As one possible implementation, the generation module is specifically configured to:

[0044] extract an initial spectrum graph from the speech signal to be recognized, the initial spectrum graph including a time feature and a frequency feature;

[0045] perform identity recognition on the speech signal to be recognized to determine a speaker feature corresponding to the speech signal to be recognized;

[0046] construct a third-order speech tensor corresponding to the speech signal to be recognized based on the time feature, the frequency feature, and the speaker feature of the speech signal to be recognized.

[0047] As one possible implementation, the determination module comprises:

[0048] A decomposition submodule is configured to perform non-negative decomposition on the third-order speech tensor to obtain N decomposition matrix groups; each decomposition matrix group includes a time matrix, a frequency matrix, and a speaker matrix, and N is a positive integer;

[0049] The first determining sub-module is configured to determine target time features and target frequency features by using the time matrix, the frequency matrix, and the speaker matrix in the decomposition matrix group.

[0050] The second determining sub-module is configured to determine the target time features and the target frequency features as speech phase features corresponding to the to-be-identified speech signal.

[0051] As a possible implementation, the first determining sub-module is specifically configured to:

[0052] Each of the decomposition matrix groups is assigned a decomposition number according to a decomposition order, and all the decomposition matrix groups are inserted into a decomposition matrix queue according to the decomposition numbers in the order;

[0053] The decomposition matrix group with the first order is selected and removed from the decomposition matrix queue as a preceding matrix group, and the preceding matrix group is subjected to orthogonal constraint processing to obtain a preceding processed matrix group; the orthogonal constraint processing includes orthogonal processing of the time matrix and the frequency matrix in the preceding matrix group, and constraint processing of the speaker matrix in the preceding matrix group;

[0054] A first decomposition loss value between the preceding processed matrix group and the three-order speech tensor is determined, and it is determined whether the first decomposition loss value meets a preset convergence condition;

[0055] If the first decomposition loss value does not meet the preset convergence condition, the decomposition matrix group with the first order is selected and removed from the decomposition matrix queue as a following matrix group based on the preceding matrix group, the following matrix group is subjected to orthogonal constraint processing based on the preceding matrix group to obtain a following processed matrix group, a second decomposition loss value between the following processed matrix group and the three-order speech tensor is determined, and it is determined whether the second decomposition loss value meets the preset convergence condition;

[0056] If the second decomposition loss value does not meet the preset convergence condition, the following processed matrix group is recorded as the preceding matrix group, and the step of selecting and removing the decomposition matrix group with the first order from the decomposition matrix queue as the following matrix group based on the preceding matrix group, the following matrix group is subjected to orthogonal constraint processing based on the preceding matrix group to obtain the following processed matrix group, the second decomposition loss value between the following processed matrix group and the three-order speech tensor is determined, and it is determined whether the second decomposition loss value meets the preset convergence condition is returned to be executed until the second decomposition loss value meets the preset convergence condition, and then the target time features and the target frequency features are output.

[0057] As a possible implementation, the identification module includes:

[0058] The third determining sub-module is configured to generate a frequency spectrum based on the speech phase features.

[0059] a fourth determining sub-module, configured to determine, according to the spectrogram, an acoustic unit sequence corresponding to the speech signal to be recognized;

[0060] a fifth determining sub-module, configured to determine a text sequence corresponding to the acoustic unit sequence, and determine the text sequence as a recognition result of the speech signal to be recognized.

[0061] As a possible implementation, the fourth determining sub-module comprises:

[0062] a matching unit, configured to match the spectrogram with each spectrogram in a preset speech database to obtain a target spectrogram, the speech database comprising a plurality of acoustic unit sequences, a spectrogram corresponding to each acoustic unit sequence, and a corresponding relationship therebetween;

[0063] a target determining unit, configured to determine, according to the corresponding relationship, a target acoustic unit sequence corresponding to the target spectrogram;

[0064] a sequence determining unit, configured to determine the target acoustic unit sequence as the acoustic unit sequence corresponding to the speech signal to be recognized.

[0065] As a possible implementation, the fourth determining sub-module comprises:

[0066] an input unit, configured to input the spectrogram into a preset frame segmentation model to obtain an encoded sequence after frame segmentation of the spectrogram;

[0067] a segmenting unit, configured to segment the encoded sequence to obtain a plurality of encoded segments;

[0068] a matching unit, configured to, for each encoded segment, match the encoded segment with an encoded segment in a preset encoded library to obtain a target encoded segment, the encoded library comprising a plurality of encoded segments, an acoustic unit corresponding to each encoded segment, and a corresponding relationship therebetween;

[0069] a plurality of target determining units, configured to determine, according to the corresponding relationship, a target acoustic unit corresponding to each target encoded segment to obtain a plurality of target acoustic units;

[0070] a sequence determining unit, configured to determine an order of the plurality of encoded segments;

[0071] an arranging unit, configured to arrange the plurality of target acoustic units according to the order to obtain the acoustic unit sequence corresponding to the speech signal to be recognized.

[0072] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, the processor being configured to execute a speech recognition program stored in the memory to implement the speech recognition method according to any one of the first aspect.

[0073] In a fourth aspect, an embodiment of the present application provides a storage medium, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the speech recognition method according to any one of the first aspect.

[0074] The technical solution provided by the embodiment of the present application comprises the following steps: obtaining a to-be-recognized speech signal, generating a third-order speech tensor based on the to-be-recognized speech signal, determining a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor, and determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature. This technical solution extracts the speech phase feature of the to-be-recognized speech signal. Regardless of the physiological structure of the cochlea or the sound signal processing process, there is processing and perception of the speech phase feature. Therefore, extracting the speech phase feature of the to-be-recognized speech signal can reduce the redundant information in the to-be-recognized speech signal, so that the speech signal can be more accurately recognized based on the speech phase feature, the redundant information in the to-be-recognized speech signal is reduced, the accuracy of speech recognition is improved, and the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 An embodiment flowchart of a speech recognition method provided by the embodiment of the present application is provided.

[0076] Figure 2 An embodiment flowchart of another speech recognition method provided by the embodiment of the present application is provided.

[0077] Figure 3 An embodiment flowchart of still another speech recognition method provided by the embodiment of the present application is provided.

[0078] Figure 4 An embodiment block diagram of a speech recognition device provided by the embodiment of the present application is provided.

[0079] Figure 5 A structural schematic diagram of an electronic device provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0080] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0081] The following will be combined with the accompanying drawings to make a further explanation of the speech recognition method provided by the present application with specific embodiments, and the embodiments do not constitute a limitation on the embodiments of the present application.

[0082] Referring to Figure 1 An embodiment flowchart of the speech recognition method provided by the present application is shown in FIG. 1. As shown in FIG. 1, the flowchart can include the following steps: Figure 1

[0083] Step 101, obtaining a to-be-recognized speech signal, and generating a third-order speech tensor based on the to-be-recognized speech signal.

[0084] The to-be-recognized speech signal is a to-be-recognized speech signal, which can be a speech signal collected in a speech interaction scene.

[0085] The third-order speech tensor is a feature tensor corresponding to the to-be-recognized speech signal, and the tensor features can be time features, frequency features, and speaker features.

[0086] The execution subject of the embodiments of the present application can be a speech recognition system, which can obtain a to-be-recognized speech signal and recognize the to-be-recognized speech signal according to the speech recognition method provided by the embodiments of the present application, so as to obtain a recognition result corresponding to the to-be-recognized speech signal.

[0087] As one possible implementation, the execution subject of the embodiments of the present application can collect the to-be-recognized speech signal through a preset speech collection device. The speech collection device can be at least one microphone, and can also be other speech collection devices, which are not limited by the embodiments of the present application.

[0088] As another possible implementation, after the execution subject of the embodiments of the present application collects the speech signal through the preset speech collection device, the speech signal can be stored in a preset storage medium. Then, the execution subject of the embodiments of the present application can obtain the to-be-recognized speech signal from the storage medium.

[0089] In an embodiment, when the execution subject of the embodiments of the present application generates the third-order speech tensor based on the to-be-recognized speech signal, it can extract an initial frequency spectrum graph from the to-be-recognized speech signal, which can include time features and frequency features.​

[0090] Specifically, the to-be-recognized speech signal can be pre-processed, filtered, dimensionally reduced, and projected, and the like, so as to extract an initial frequency spectrum of the to-be-recognized speech signal.

[0091] Then, the to-be-recognized speech signal can be identified to determine a speaker feature corresponding to the to-be-recognized speech signal, which can be a sound signal unique to the cochlea of each speaker.

[0092] Finally, a three-order speech tensor corresponding to the to-be-recognized speech signal can be constructed based on the time feature, the frequency feature, and the speaker feature of the to-be-recognized speech signal.

[0093] Step 102, determining a speech phase feature corresponding to the to-be-recognized speech signal from the three-order speech tensor.

[0094] Step 103, determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature.

[0095] The steps 102 and 103 are described as follows:

[0096] The speech phase feature refers to a sound feature and a frequency feature corresponding to the to-be-recognized speech signal.

[0097] The recognition result can be a text content corresponding to the to-be-recognized speech signal.

[0098] In the prior art, when recognizing speech, a speech recognition model is generally trained to recognize the to-be-recognized speech. However, in the intelligent speech interaction scene, the input and output of the sound are often interfered by various kinds of interference, such as background noise interference, other speaker sound interference, and the like. This results in various kinds of redundant information in the collected speech signal. The existing speech recognition model is generally obtained by training a corpus, and cannot completely remove the redundant information in the speech signal when recognizing the speech, which results in an inaccurate recognition result of the to-be-recognized speech.

[0099] Therefore, the execution subject of the embodiment of the present application can determine the speech phase feature of the speech signal to be recognized from the third-order speech tensor corresponding to the speech signal to be recognized, and then can recognize the speech signal to be recognized according to the speech phase feature of the speech signal to be recognized, and can obtain the recognition result of the speech signal to be recognized. Since the speech phase feature of the speech signal to be recognized is extracted by the method provided by the embodiment of the present application, and there is processing and perception of the sound phase from the physiological structure of the cochlea or the sound signal processing process, the speech phase feature of the speech signal to be recognized can reduce the redundant information in the speech signal to be recognized, and therefore, the speech signal to be recognized can be recognized based on the speech phase feature, and the accuracy of recognizing the speech to be recognized can be improved.

[0100] In an embodiment, as can be known from the above description, the third-order speech tensor corresponding to the speech signal to be recognized includes the time feature, the frequency feature, and the speaker identity feature of the speech signal to be recognized. Since the speech signal to be recognized is recognized to recognize the text content of the speech signal to be recognized, and the speaker feature is not beneficial to the recognition of the speech signal, the execution subject of the embodiment of the present application can determine the speech phase feature, i.e., the time feature and the frequency feature, from the third-order speech tensor of the speech signal to be recognized.

[0101] Specifically, the third-order speech tensor can be non-negative decomposed to obtain N decomposition matrix groups, wherein each decomposition matrix group can include a decomposed time matrix, a frequency matrix, and a speaker feature, and N can be a positive integer.

[0102] Then, the target time feature and the target frequency feature can be determined by using the time matrix, the frequency matrix, and the speaker matrix in the decomposition matrix group, and the target time feature and the target frequency feature are determined as the speech phase feature corresponding to the speech signal to be recognized.

[0103] As to how to determine the target time feature and the target frequency feature by using the time matrix, the frequency matrix, and the speaker matrix in the decomposition matrix group, the process will be described below by referring to the flowchart shown in FIG. 4, which will not be described in detail here. Figure 2

[0104] As to how to determine the speech recognition result corresponding to the speech signal to be recognized based on the speech phase feature, the process will be described below by referring to the flowchart shown in FIG. 5, which will not be described in detail here. Figure 3

[0105] ​​The technical scheme provided by the embodiment of the present application comprises the following steps: obtaining a to-be-recognized speech signal, generating a third-order speech tensor based on the to-be-recognized speech signal, determining a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor, and determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature. This technical scheme extracts the speech phase feature of the to-be-recognized speech signal, and the speech phase feature is processed and perceived in the process of the physiological structure of the cochlea and the processing of the sound signal, so that the redundant information in the to-be-recognized speech signal can be reduced, the speech recognition can be more accurate, the redundant information in the to-be-recognized speech signal can be reduced, the accuracy of the speech recognition can be improved, and the user experience can be improved.

[0106] Referring to Figure 2 An embodiment flowchart of another speech recognition method provided by the embodiment of the present application is shown. Figure 2 The flowchart is shown in Figure 1 Based on the flowchart shown in the figure, how to determine the target time feature and the target frequency feature by using the time matrix, the frequency matrix, and the speaker matrix in the decomposition matrix group is described in detail. As shown in the figure Figure 2 The flowchart can comprise the following steps:

[0107] Step 201: A decomposition number is assigned to each decomposition matrix group according to the decomposition order, and all the decomposition matrix groups are inserted into the decomposition matrix queue according to the order of the decomposition numbers.

[0108] The decomposition matrix group is a decomposition matrix group obtained by performing non-negative decomposition on the third-order speech tensor of the to-be-recognized speech signal, and each decomposition matrix group can comprise a time matrix, a frequency matrix, and a speaker matrix.

[0109] The decomposition matrix queue is a queue composed of decomposition matrix groups arranged according to the decomposition numbers.

[0110] In an embodiment, when performing non-negative decomposition on the third-order speech tensor, a decomposition number can be assigned to each decomposition matrix group as the decomposition matrix group is decomposed, and the decomposition matrix group corresponding to the decomposition number is sequentially inserted into the decomposition matrix queue.

[0111] In another embodiment, the execution subject of the embodiment of the present application can assign a decomposition number to each decomposition matrix group according to the decomposition order after completing the non-negative decomposition on the third-order speech tensor, and all the decomposition matrix groups are inserted into the decomposition matrix queue according to the order of the decomposition numbers.

[0112] Step 202, selecting and removing the decomposition matrix group ranked first from the decomposition matrix queue as a preceding matrix group, and performing orthogonal constraint processing on the preceding matrix group to obtain a preceding processed matrix group, the orthogonal constraint processing including orthogonal processing on the time matrix and the frequency matrix in the preceding matrix group, and constraint processing on the speaker matrix in the preceding matrix group.

[0113] The orthogonal constraint processing can include orthogonal processing and constraint processing, wherein the orthogonal processing can be orthogonal constraint processing, for example, the time matrix is U, and the time matrix after orthogonal processing is T U||1, wherein α1 is a preset coefficient, and the value range of α1 can be 0-1, U T is the transpose of U, and ||1 is the first norm, and the frequency matrix is the same.

[0114] The constraint processing can be orthogonal constraint or sparse constraint, and the embodiments of the present application do not limit this. When the constraint processing is sparse constraint, a sparse constraint coefficient can be added to the speaker matrix, for example, the speaker matrix is W, and after adding the sparse constraint to the speaker matrix, it can be wherein α3 is a sparse constraint coefficient, is the square of the Euclidean distance, that is, the square of the Euclidean distance of the speaker matrix W.

[0115] In an embodiment, after determining the decomposition matrix group ranked first in the decomposition matrix queue as the preceding matrix group, the time matrix and the frequency matrix in the preceding matrix group can be subjected to orthogonal processing, and the speaker matrix in the preceding matrix group can be subjected to sparse constraint processing, to obtain the preceding processed matrix group.

[0116] For example, assuming that the time matrix is U, the frequency matrix is V, and the speaker matrix is W, after orthogonal processing on the time matrix and the frequency matrix and constraint processing on the speaker matrix, the preceding processed matrix group can be the following formula (I):

[0117]

[0118] wherein α1 is a preset coefficient, and the value range of α1 can be 0-1, U T is the transpose of U, ||1 is the first norm, α2 is a preset coefficient, and the value range of α2 can be 0-1, V T is the transpose of V, and α3 is a sparse constraint coefficient, is the square of the Euclidean distance.

[0119] Step 203, determining the first decomposition loss value between the preceding processing matrix group and the third-order speech tensor, and judging whether the first decomposition loss value meets the preset convergence condition, if the first decomposition loss value does not meet the preset convergence condition, step 204 is executed; if the first decomposition loss value meets the preset convergence condition, step 206 is executed.

[0120] In the embodiment of the application, in order to more accurately determine the speech phase feature of the to-be-recognized speech signal, the more accurate time matrix and frequency matrix can be determined from the plurality of matrix groups obtained by decomposition, and therefore the decomposition loss value (for convenience of description, hereinafter referred to as the first decomposition loss value) between the preceding processing matrix group and the third-order speech tensor can be determined.

[0121] Optionally, if it is determined that the first decomposition loss value meets the preset convergence condition, it indicates that the time matrix and the frequency matrix in the preceding matrix group corresponding to the preceding processing matrix group are more accurate, and then step 206 can be executed. Optionally, when the first decomposition loss value is less than a preset loss value threshold, it can be determined that the first decomposition loss value meets the preset convergence condition.

[0122] On the contrary, if it is determined that the first decomposition loss value does not meet the preset convergence condition, it indicates that the time matrix and the frequency matrix in the preceding matrix group corresponding to the preceding processing matrix group are not accurate, and then step 204 can be executed.

[0123] Specifically, when the first decomposition loss value between the preceding processing matrix group and the third-order speech tensor is determined, the first decomposition loss value can be calculated by the following formula (two) or the following formula (three):

[0124]

[0125] wherein X is the third-order speech tensor, is the decomposition matrix group, U is the time matrix, V is the frequency matrix, W is the speaker matrix, um is the mth time feature of U, vm is the mth frequency feature of V, wm is the mth speaker feature of W, UT is the transpose of U, VT is the transpose of V, α1 and α2 are orthogonal constraint coefficients, and α3 is a sparse constraint coefficient.

[0126] or,

[0127]

[0128] wherein X is the third-order speech tensor, is the decomposition matrix group, U is the time matrix, V is the frequency matrix, W is the speaker matrix, um is the mth time feature of U, vm is the mth frequency feature of V, wm is the mth speaker feature of W, UT is the transpose of U, VT is the transpose of V, α1 and α2 are orthogonal constraint coefficients, and α3 is a sparse constraint coefficient.

[0129] Step 204, reselect and remove the decomposition matrix group ranked first from the decomposition matrix queue as the posterior matrix group, and perform orthogonal constraint processing on the posterior matrix group based on the prior matrix group to obtain the posterior processing matrix group.

[0130] It can be understood that, since the decomposition matrix group ranked first in the decomposition matrix queue (the decomposition matrix queue without removing any decomposition matrix group) does not have a corresponding prior matrix group, assuming that the decomposition matrix group ranked first is U1, V1 and W1, U0, V0 and W0 can be randomly set as the prior matrix group of U1, V1 and W1, only need to ensure that the values in U0, V0 and W0 are all greater than 0. The orthogonal constraint processing on the posterior matrix group based on the prior matrix group includes two steps: first, adjusting the elements in the posterior matrix group through the prior matrix group; second, performing orthogonal processing on the time matrix and the frequency matrix in the element-adjusted posterior matrix group, and performing constraint processing on the speaker matrix in the element-adjusted posterior matrix group.

[0131] In the embodiment of the application, after determining the posterior matrix group, in order to more accurately determine the time feature and the frequency feature of the to-be-recognized speech signal, the execution subject of the embodiment of the application can adjust the elements in the posterior matrix group based on the prior matrix group, and then perform orthogonal constraint processing on the posterior matrix group to obtain the posterior processing matrix group.

[0132] Specifically, the elements in the posterior matrix group can be adjusted through the prior matrix group as follows:

[0133]

[0134] Wherein, is the time matrix in the adjusted posterior matrix group, U (k+1) is the time matrix in the posterior matrix group, U k is the time matrix in the prior matrix group, V k is the frequency matrix in the prior matrix group, W k is the speaker matrix in the prior matrix group, 1 is a matrix with elements all being 1, the number of elements of which is the same as that of U, * represents multiplication according to the corresponding elements in the matrix (i.e. the same position elements); / represents division according to the corresponding elements in the matrix; · represents logical operation (XOR operation) according to the corresponding elements in the matrix.

[0135]

[0136] Wherein, is the frequency matrix in the adjusted posterior matrix group, V (k+1) is the frequency matrix in the posterior matrix group, Uk V is a time matrix in the former matrix group, V k W is a frequency matrix in the former matrix group, W k I is a matrix with all elements being 1, and the number of elements of I is the same as that of U, * represents multiplication of corresponding elements in matrices (i.e. same position elements), / represents division of corresponding elements in matrices, and · represents logical operation (XOR operation) of corresponding elements in matrices.

[0137]

[0138] wherein, W is an adjusted speaker matrix in the former matrix group, W (k+1) U is a speaker matrix in the latter matrix group, U k V is a time matrix in the former matrix group, V k W is a frequency matrix in the former matrix group, W k I is a matrix with all elements being 1, and the number of elements of I is the same as that of U, * represents multiplication of corresponding elements in matrices (i.e. same position elements), / represents division of corresponding elements in matrices, and · represents logical operation (XOR operation) of corresponding elements in matrices.

[0139] After that, the adjusted latter matrix group can be subjected to orthogonal constraint processing, that is, the adjusted matrix group is taken as a new latter matrix group, and the new latter matrix group is subjected to orthogonal constraint processing. Specifically, the time matrix and the frequency matrix in the adjusted latter matrix group can be subjected to orthogonal processing, and the speaker matrix in the former matrix group can be subjected to sparse constraint processing, so as to obtain a processed latter matrix group.

[0140] Specifically, the detailed process of orthogonal constraint processing of the adjusted latter matrix group is consistent with the steps of orthogonal constraint processing of the former matrix group, and can be referred to step 202, which will not be described here.

[0141] Step 205, a second decomposition loss value between the processed latter matrix group and the third-order speech tensor is determined, and it is judged whether the second decomposition loss value meets a preset convergence condition. If not, the processed latter matrix group is recorded as the former matrix group mentioned above, and step 204 is returned to be executed. If yes, step 206 is executed.

[0142] Step 206, target time features and target frequency features are outputted.

[0143] The following uniformly describes step 205 and step 206:

[0144] In the embodiment of the present application, after the post-processing matrix set is determined, a decomposition loss value (hereinafter referred to as a second decomposition loss value for convenience of description) between the post-processing matrix set and the third-order speech tensor of the speech signal to be recognized can be determined, and it is determined whether the second decomposition loss value meets a preset convergence condition.

[0145] Optionally, if the second decomposition loss value meets the preset convergence condition, it is indicated that the post-matrix set corresponding to the post-processing matrix set is the most accurate decomposition matrix set in the decomposition matrix set obtained by the third-order speech tensor decomposition, and therefore, the time matrix and the frequency matrix in the post-matrix set can be output, and the time matrix is taken as the target time feature and the frequency matrix is taken as the target frequency feature.

[0146] On the contrary, if the second decomposition loss value does not meet the preset convergence condition, it is indicated that the post-matrix set corresponding to the post-processing matrix set is not accurate, and therefore, the post-processing matrix set can be recorded as the pre-matrix set, and the step 204 is returned to be executed until the second decomposition loss value meets the preset convergence condition, the time matrix and the frequency matrix in the current post-matrix set can be output, and the time matrix is taken as the target time feature and the frequency matrix is taken as the target frequency feature.

[0147] In addition, after it is determined that the second decomposition loss value meets the preset convergence condition, the number of the remaining decomposition matrix sets in the decomposition matrix queue can be further determined, and if the number is greater than a first number threshold, it is indicated that the number of the current calculated decomposition matrix set is small, and in order to improve the accuracy, the step 204 can be returned to be executed continuously.

[0148] Optionally, if the number is less than the first number threshold, it is indicated that the number of the current calculated decomposition matrix set is large, and the accuracy can be improved, and at this time, the time matrix and the frequency matrix in the current post-matrix set can be output, and the time matrix is taken as the target time feature and the frequency matrix is taken as the target frequency feature.

[0149] The technical scheme provided by the embodiment of the application comprises the following steps: assigning a decomposition number to each decomposition matrix group according to a decomposition order, inserting all the decomposition matrix groups into a decomposition matrix queue according to the order of the decomposition numbers, selecting and removing the decomposition matrix group with the first order from the decomposition matrix queue as a preceding matrix group, performing orthogonal constraint processing on the preceding matrix group to obtain a preceding processed matrix group, the orthogonal constraint processing comprises orthogonal processing of time matrices and frequency matrices in the preceding matrix group, and constraint processing of speaker matrices in the preceding matrix group, determining a first decomposition loss value between the preceding processed matrix group and the three-order speech tensor, and judging whether the first decomposition loss value meets a preset convergence condition; if the first decomposition loss value does not meet the preset convergence condition, then a decomposition matrix group with the first order is selected and removed from the decomposition matrix queue as a subsequent matrix group, the subsequent matrix group is subjected to orthogonal constraint processing based on the preceding matrix group to obtain a subsequent processed matrix group, a second decomposition loss value between the subsequent processed matrix group and the three-order speech tensor is determined, and whether the second decomposition loss value meets the preset convergence condition is judged; if the second decomposition loss value does not meet the preset convergence condition, then the subsequent processed matrix group is recorded as the preceding matrix group, and the step of selecting and removing the decomposition matrix group with the first order from the decomposition matrix queue as the subsequent matrix group, performing orthogonal constraint processing on the subsequent matrix group based on the preceding matrix group to obtain the subsequent processed matrix group, determining the second decomposition loss value between the subsequent processed matrix group and the three-order speech tensor, and judging whether the second decomposition loss value meets the preset convergence condition is returned to be executed until the second decomposition loss value meets the preset convergence condition, and then target time features and target frequency features are output. Through iteration and orthogonal constraint processing of the decomposition matrix groups decomposed from the three-order speech tensor, the speech phase features of the speech signal to be recognized can be accurately extracted, the speech information in the speech phase features of the speech signal to be recognized is reduced, the accuracy of speech recognition is improved, and the user experience is improved.

[0150] Referring to Figure 3 An embodiment flowchart of another speech recognition method provided by the embodiment of the application is shown. Figure 3 The flowchart is shown in Figure 1 Based on the flowchart shown in

[0151] As shown in Figure 3 The flowchart can comprise the following steps:

[0152] Step 301: generating a frequency spectrum based on speech phase features.

[0153] The frequency spectrum is a frequency spectrum of speech phase features of a speech signal to be recognized.

[0154] In the embodiment of the present application, after the speech phase feature of the to-be-recognized speech signal is extracted, the corresponding frequency spectrum graph can be determined according to the speech phase feature. Since the speech phase feature contains the time feature and the frequency feature in the to-be-recognized speech signal, the frequency spectrum graph with the time of the to-be-recognized speech signal as the horizontal axis and the frequency of the to-be-recognized speech signal as the vertical axis can be directly constructed according to the speech phase feature. It should be noted that the frequency spectrum graph generated according to the speech phase feature at this time is different from the initial frequency spectrum graph described above. The speech phase feature in the initial frequency spectrum graph contains redundant information such as environmental noise and speaker characteristics, while the frequency spectrum graph in the embodiment of the present application is the speech phase feature with a large part of the redundant information reduced or eliminated. The subsequent steps S302-S303 performed through the frequency spectrum graph can reduce the adverse effects of the redundant information and improve the accuracy of speech recognition.

[0155] Optionally, the obtained phase feature can be input into a preset frequency spectrum graph generation model to obtain a frequency spectrum graph corresponding to the phase feature.

[0156] In step 302, the acoustic unit sequence corresponding to the to-be-recognized speech signal is determined according to the frequency spectrum graph.

[0157] The acoustic unit sequence is a sequence composed of sound production units corresponding to the to-be-recognized speech, for example, if the to-be-recognized speech is the pronunciation of “hello”, the corresponding acoustic unit sequence can be “nihao”.

[0158] In the embodiment of the present application, when the acoustic unit sequence corresponding to the to-be-recognized speech signal is determined, the frequency spectrum graph can be used for determination.

[0159] As one possible implementation, the execution subject of the embodiment of the present application can pre-store a speech database, which can include a plurality of acoustic unit sequences, a frequency spectrum graph corresponding to each acoustic unit sequence, and a corresponding relationship therebetween. Based on this, the execution subject of the embodiment of the present application can match the frequency spectrum graph corresponding to the speech phase feature of the to-be-recognized speech signal with each frequency spectrum graph in the above-mentioned preset speech database to obtain a target frequency spectrum graph.

[0160] Specifically, when the frequency spectrum graph is matched with each frequency spectrum graph in the speech database, the similarity of the frequency spectrum graph and each frequency spectrum graph in the speech database can be calculated to obtain a similarity value. Optionally, the DTW (Dynamic Time Warping) algorithm can be used to calculate the similarity value of the frequency spectrum graph of the phase feature of the to-be-recognized speech signal and each frequency spectrum graph in the speech database. Preferably, the frequency spectrum graph with the highest similarity value can be determined as the target frequency spectrum graph.

[0161] Afterwards, the target acoustic unit sequence corresponding to the target spectrogram can be determined according to the above correspondence relationship, and the target acoustic unit sequence is determined as the acoustic unit sequence corresponding to the speech signal to be recognized.

[0162] As another possible implementation, the spectrogram can be divided into multiple frame segments, and then the acoustic unit corresponding to each frame segment is determined, and the multiple acoustic units are combined to obtain the acoustic unit sequence of the speech signal to be recognized.

[0163] Specifically, the execution body of the embodiment of the present application can input the spectrogram into the preset frame segmentation model to obtain the encoding sequence of the spectrogram after frame segmentation output by the frame segmentation model (for example, 11223345556), and segment the encoding sequence to obtain multiple encoding segments.

[0164] Optionally, since different speakers will speak the same text content, but the speaking speed is different, which will lead to the fact that one pronunciation unit occupies different number of frames in the speech signal to be recognized. Therefore, in order to accurately recognize the text content of the speech signal to be recognized, the consecutive N same encodings in the encoding sequence can be classified into an encoding segment to obtain multiple encoding segments, and the N is a positive integer.

[0165] Afterwards, one acoustic unit corresponding to each encoding segment can be determined. Specifically, for each encoding segment, the encoding segment can be matched with the encoding segments in the encoding library to obtain a target encoding segment. The execution body of the embodiment of the present application can pre-store the encoding library, and the encoding library can include multiple encoding segments, an acoustic unit corresponding to each encoding segment, and the correspondence relationship therebetween.

[0166] Afterwards, according to the correspondence relationship between the encoding segment and the acoustic unit, the target acoustic unit corresponding to each target encoding segment can be determined, thereby obtaining multiple target acoustic units.

[0167] Finally, the order of the multiple encoding segments can be determined, and the multiple target acoustic units are arranged according to the order to obtain the acoustic unit sequence corresponding to the speech signal to be recognized.

[0168] For example, assume that the acoustic unit sequence of the speech signal to be recognized output by the frame model is "11222344344567789", and then the coding sequence is segmented according to the rule of grouping continuous same codes into a coding segment, and a plurality of coding segments are obtained: 11, 222, 3, 44, 3, 44, 5, 6, 77, 8, 9. It is further assumed that the coding library has the following coding sequence: 122334443344456789, and the correspondence between the coding segments and the acoustic units in the coding sequence is: 1->j, 22->in, 33->t, 444->ian, 5->qi, 6->h, 7->en, 8->h, 9->ao.

[0169] Then, the matching between the plurality of coding segments corresponding to the speech signal to be recognized and the coding segments in the coding database is performed, and the following results are obtained: the target coding segment corresponding to "11" is "1", the coding segment corresponding to "222" is "22", the coding segment corresponding to "3" is "3", the coding segment corresponding to "44" is "444", the coding segment corresponding to "3" is "33", the coding segment corresponding to "44" is "444", the coding segment corresponding to "5" is "5", the coding segment corresponding to "6" is "6", the coding segment corresponding to "77" is "7", the coding segment corresponding to "8" is "8", and the coding segment corresponding to "9" is "9".

[0170] Then, according to the correspondence between the coding segments and the acoustic units, the following results are obtained: 11->j, 222->in, 3->t, 44->ian, 5->qi, 6->h, 77->en, 8->h, 9->ao. Therefore, according to the above order of the coding segments, the acoustic unit sequence corresponding to the speech to be recognized is determined to be "jintiantianqihenhao".

[0171] Step 303, determining the text sequence corresponding to the above acoustic unit sequence, and determining the text sequence as the recognition result of the speech signal to be recognized.

[0172] In the embodiment of the present application, after obtaining the acoustic unit sequence corresponding to the speech signal to be recognized, the text sequence corresponding to the acoustic unit sequence is determined, and the text sequence is determined as the recognition result of the speech signal to be recognized.

[0173] Optionally, the acoustic unit sequence can be input into a preset acoustic unit recognition model, a text sequence output by the acoustic unit recognition model is obtained, and the text sequence is determined as the recognition result of the speech signal to be recognized.

[0174] The technical scheme provided by the embodiment of the present application comprises the following steps: determining a spectrum graph corresponding to a phase feature, determining an acoustic unit sequence corresponding to a to-be-recognized speech signal according to the spectrum graph, determining a text sequence corresponding to the acoustic unit sequence, and determining the text sequence as a recognition result of the to-be-recognized speech signal. According to the spectrum graph of the phase feature, the acoustic unit sequence of the to-be-recognized speech signal is determined, and the acoustic unit sequence is recognized to obtain the recognition result of the to-be-recognized speech signal. Since the phase feature does not have much redundant information, the acoustic unit sequence determined according to the spectrum graph of the phase feature is more accurate, so that the recognition result of the to-be-recognized speech signal is more accurate. The accuracy of the recognition of the to-be-recognized speech signal is improved, and the user experience is improved.

[0175] Reference Figure 4 An embodiment block diagram of a speech recognition device provided by the embodiment of the present application is provided.

[0176] As Figure 4 shown, the device comprises:

[0177] The acquisition module 41 is configured to acquire a to-be-recognized speech signal.

[0178] The generation module 42 is configured to generate a third-order speech tensor based on the to-be-recognized speech signal.

[0179] The determination module 43 is configured to determine a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor.

[0180] The recognition module 44 is configured to determine a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature.

[0181] As a possible implementation manner, the generation module 42 is specifically configured to:

[0182] extract an initial spectrum graph from the to-be-recognized speech signal, wherein the initial spectrum graph comprises a time feature and a frequency feature;

[0183] perform identity recognition on the to-be-recognized speech signal to determine a speaker feature corresponding to the to-be-recognized speech signal;

[0184] construct a third-order speech tensor corresponding to the to-be-recognized speech signal based on the time feature, the frequency feature, and the speaker feature of the to-be-recognized speech signal.

[0185] As a possible implementation manner, the determination module 43 comprises (not shown in the figure):

[0186] The decomposition sub-module is configured to perform non-negative decomposition on the third-order speech tensor to obtain N groups of decomposition matrices, each of which comprises a time matrix, a frequency matrix, and a speaker matrix, and N is a positive integer.

[0187] The first determination sub-module is configured to determine target time features and target frequency features by using the time matrix, the frequency matrix, and the speaker matrix in the group of decomposition matrices.

[0188] The second determination sub-module is configured to determine the target time features and the target frequency features as speech phase features corresponding to the speech signal to be recognized.

[0189] As a possible implementation, the first determination sub-module is specifically configured to:

[0190] Each of the groups of decomposition matrices is assigned a decomposition number according to a decomposition sequence, and all the groups of decomposition matrices are inserted into a decomposition matrix queue according to the order of the decomposition numbers;

[0191] The first group of decomposition matrices in the decomposition matrix queue is selected and removed as a preceding group of matrices, and the preceding group of matrices is subjected to orthogonal constraint processing to obtain a preceding processed group of matrices. The orthogonal constraint processing includes orthogonal processing of the time matrix and the frequency matrix in the preceding group of matrices, and constraint processing of the speaker matrix in the preceding group of matrices.

[0192] A first decomposition loss value between the preceding processed group of matrices and the third-order speech tensor is determined, and it is determined whether the first decomposition loss value meets a preset convergence condition.

[0193] If the first decomposition loss value does not meet the preset convergence condition, the first group of decomposition matrices in the decomposition matrix queue is selected and removed as a subsequent group of matrices, the subsequent group of matrices is subjected to orthogonal constraint processing based on the preceding group of matrices to obtain a subsequent processed group of matrices, a second decomposition loss value between the subsequent processed group of matrices and the third-order speech tensor is determined, and it is determined whether the second decomposition loss value meets the preset convergence condition.

[0194] If the second decomposition loss value does not meet the preset convergence condition, the subsequent processed group of matrices is recorded as the preceding group of matrices, and the step of selecting and removing the first group of decomposition matrices in the decomposition matrix queue as the subsequent group of matrices, performing orthogonal constraint processing on the subsequent group of matrices based on the preceding group of matrices to obtain a subsequent processed group of matrices, determining a second decomposition loss value between the subsequent processed group of matrices and the third-order speech tensor, and determining whether the second decomposition loss value meets the preset convergence condition is performed until the second decomposition loss value meets the preset convergence condition, and then the target time features and the target frequency features are output.

[0195] As a possible implementation, the recognition module 44 includes (not shown in the figure):

[0196] The third determination sub-module is configured to generate a spectrogram based on the voice phase feature;

[0197] The fourth determination sub-module is configured to determine an acoustic unit sequence corresponding to the voice signal to be recognized according to the spectrogram;

[0198] The fifth determination sub-module is configured to determine a text sequence corresponding to the acoustic unit sequence, and determine the text sequence as the recognition result of the voice signal to be recognized.

[0199] As a possible implementation, the fourth determination sub-module includes:

[0200] The matching unit is configured to match the spectrogram with each spectrogram in a preset voice database to obtain a target spectrogram, the voice database including a plurality of acoustic unit sequences, a spectrogram corresponding to each acoustic unit sequence, and a corresponding relationship therebetween;

[0201] The target determination unit is configured to determine a target acoustic unit sequence corresponding to the target spectrogram according to the corresponding relationship;

[0202] The sequence determination unit is configured to determine the target acoustic unit sequence as the acoustic unit sequence corresponding to the voice signal to be recognized.

[0203] As a possible implementation, the fourth determination sub-module includes:

[0204] The input unit is configured to input the spectrogram into a preset frame segmentation model to obtain an encoded sequence after frame segmentation of the spectrogram;

[0205] The segmentation unit is configured to segment the encoded sequence to obtain a plurality of encoded segments;

[0206] The matching unit is configured to, for each encoded segment, match the encoded segment with an encoded segment in a preset encoded library to obtain a target encoded segment, the encoded library including a plurality of encoded segments, an acoustic unit corresponding to each encoded segment, and a corresponding relationship therebetween;

[0207] The multi-target determination unit is configured to determine a target acoustic unit corresponding to each target encoded segment according to the corresponding relationship to obtain a plurality of target acoustic units;

[0208] The order determination unit is configured to determine an order of the plurality of encoded segments;

[0209] An arrangement unit is configured to arrange the plurality of target acoustic units according to the sequence to obtain an acoustic unit sequence corresponding to the to-be-recognized speech signal.

[0210] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1, Figure 5 The electronic device 500 shown includes at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. The various components in the electronic device 500 are coupled together by a bus system 505. It can be understood that the bus system 505 is used to realize the connection and communication between the components. In addition to including a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all the buses are marked as the bus system 505 in Figure 5 FIG. 1.

[0211] The user interface 503 can include a display, a keyboard, or a pointing device (for example, a mouse, a trackball, a touchpad, or a touchscreen).

[0212] It can be understood that the memory 502 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct Rambus random access memory (DRRAM). The memory 502 described herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0213] In some embodiments, the memory 502 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system 5021 and application programs 5022.

[0214] The operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application programs 5022 include various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., for implementing various application services. The programs for implementing the method embodiments of the present application can be included in the application programs 5022.

[0215] In the embodiments of the present application, by invoking the programs or instructions stored in the memory 502, specifically, the programs or instructions stored in the application programs 5022, the processor 501 is configured to execute the method steps provided by the various method embodiments, for example, including:

[0216] obtaining a to-be-recognized speech signal, and generating a third-order speech tensor based on the to-be-recognized speech signal;

[0217] determining a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor;

[0218] determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature.

[0219] The method disclosed by the embodiments of the present application can be applied to the processor 501 or implemented by the processor 501. The processor 501 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the method can be completed by the integrated logic electric circuit in the hardware of the processor 501 or the instruction in the form of software. The processor 501 described above can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software units in the code processor for execution. The software unit can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the storage 502, and the processor 501 reads the information in the storage 502 and combines the hardware to complete the steps of the above method.

[0220] It can be understood that the embodiments described herein can be realized by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units for executing the functions described in the present application or a combination thereof.

[0221] For software implementation, the technology described herein can be implemented by units performing the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0222] The electronic device provided by the embodiments can be, for example Figure 5The electronic device shown in the specification can perform the steps of the speech recognition method as Figures 1 to 3 all steps of the speech recognition method in the specification, and further realize Figures 1 to 3 the technical effects of the speech recognition method shown in the specification, please refer to Figures 1 to 3 The relevant description is not repeated here for brevity.

[0223] The embodiment of the present application also provides a storage medium (computer readable storage medium). The storage medium stores one or more programs. Wherein, the storage medium can include volatile memory, such as random access memory; the memory can also include non-volatile memory, such as read only memory, flash memory, hard disk or solid state disk; the memory can also include a combination of the above kinds of memory.

[0224] When one or more programs in the storage medium can be executed by one or more processors to implement the speech recognition method executed on the electronic device side as described above.

[0225] The processor is configured to execute the speech recognition program stored in the memory to implement the steps of the speech recognition method executed on the electronic device side as follows:

[0226] Obtaining a to-be-recognized speech signal, and generating a third-order speech tensor based on the to-be-recognized speech signal;

[0227] Determining a speech phase feature corresponding to the to-be-recognized speech signal from the third-order speech tensor;

[0228] Determining a speech recognition result corresponding to the to-be-recognized speech signal based on the speech phase feature.

[0229] The skilled person should also further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0230] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented in hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in random access memory (RAM), memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0231] The above detailed description of the specific embodiments of the present application is provided for the purpose of further explaining the objects, technical solutions and advantages of the present application, and it should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A voice recognition method, characterized by, The method comprises: acquiring a speech signal to be recognized, and generating a third-order speech tensor based on the speech signal to be recognized; determining a speech phase feature corresponding to the speech signal to be recognized from the third-order speech tensor; determining a speech recognition result corresponding to the speech signal to be recognized based on the speech phase feature; wherein the determination of the speech phase feature corresponding to the speech signal to be recognized from the third-order speech tensor comprises: performing non-negative decomposition on the third-order speech tensor to obtain N groups of decomposition matrices; each group of the decomposition matrices comprises a time matrix, a frequency matrix and a speaker matrix, and N is a positive integer; target time and frequency features are determined by using the time matrix, the frequency matrix and the speaker matrix in the group of the decomposition matrices; and the target time and frequency features are determined as the speech phase feature corresponding to the speech signal to be recognized; the determination of the target time and frequency features by using the time matrix, the frequency matrix and the speaker matrix in the group of the decomposition matrices comprises: assigning each group of the decomposition matrices with a decomposition number according to a decomposition order, and inserting all groups of the decomposition matrices into a decomposition matrix queue according to the order of the decomposition numbers; selecting and removing a group of decomposition matrices with a first order from the decomposition matrix queue as a preceding matrix group, performing orthogonal constraint processing on the preceding matrix group to obtain a preceding processed matrix group; the orthogonal constraint processing comprises orthogonal processing on the time matrix and the frequency matrix in the preceding matrix group, and constraint processing on the speaker matrix in the preceding matrix group; determining a first decomposition loss value between the preceding processed matrix group and the third-order speech tensor, and judging whether the first decomposition loss value meets a preset convergence condition; if it is determined that the first decomposition loss value meets the preset convergence condition, the time matrix in the preceding matrix group is determined as the target time feature, and the frequency matrix in the preceding matrix group is determined as the target frequency feature.

2. The method of claim 1, wherein, the generation of the third-order speech tensor based on the speech signal to be recognized comprises: extracting an initial frequency spectrum from the speech signal to be recognized, wherein the initial frequency spectrum comprises time and frequency features; performing identity recognition on the speech signal to be recognized to determine a speaker feature corresponding to the speech signal to be recognized; constructing a third-order speech tensor corresponding to the speech signal to be recognized based on the time and frequency features and the speaker feature of the speech signal to be recognized.

3. The method of claim 1, wherein, The method further comprises: if the first decomposition loss value does not meet the preset convergence condition, a group of decomposition matrices with a second order is selected and removed from the decomposition matrix queue as a subsequent matrix group, orthogonal constraint processing is performed on the subsequent matrix group based on the preceding matrix group to obtain a subsequent processed matrix group, a second decomposition loss value between the subsequent processed matrix group and the third-order speech tensor is determined, and it is judged whether the second decomposition loss value meets the preset convergence condition. If the second decomposition loss value does not satisfy the preset convergence condition, the post-processing matrix group is recorded as a previous matrix group, and the steps of selecting and removing the decomposition matrix group ranked first from the decomposition matrix queue as a post-matrix group, performing orthogonal constraint processing on the post-matrix group based on the previous matrix group to obtain a post-processing matrix group, determining a second decomposition loss value between the post-processing matrix group and the third-order speech tensor, and judging whether the second decomposition loss value satisfies the preset convergence condition are returned to be executed until the second decomposition loss value satisfies the preset convergence condition, and the target time feature and the target frequency feature are output.

4. The method of claim 1, wherein, The method comprises: generating a frequency spectrum based on the speech phase feature; determining an acoustic unit sequence corresponding to the speech signal to be recognized according to the frequency spectrum; determining a text sequence corresponding to the acoustic unit sequence, and determining the text sequence as the recognition result of the speech signal to be recognized.

5. The method of claim 4, wherein, The method comprises: matching the frequency spectrum with each frequency spectrum in a preset speech database to obtain a target frequency spectrum, the speech database comprising a plurality of acoustic unit sequences, a frequency spectrum corresponding to each acoustic unit sequence, and a corresponding relationship therebetween; determining a target acoustic unit sequence corresponding to the target frequency spectrum according to the corresponding relationship; determining the target acoustic unit sequence as the acoustic unit sequence corresponding to the speech signal to be recognized.

6. The method of claim 4, wherein, The method comprises: inputting the frequency spectrum into a preset frame model to obtain an encoded sequence after the frequency spectrum is framed; segmenting the encoded sequence to obtain a plurality of encoded segments; for each encoded segment, matching the encoded segment with an encoded segment in a preset encoded library to obtain a target encoded segment, the encoded library comprising a plurality of encoded segments, an acoustic unit corresponding to each encoded segment, and a corresponding relationship therebetween; determining a target acoustic unit corresponding to each target encoded segment according to the corresponding relationship to obtain a plurality of target acoustic units; determining the order of the plurality of encoded segments; arranging the plurality of target acoustic units according to the order to obtain the acoustic unit sequence corresponding to the speech signal to be recognized.

7. A speech recognition apparatus characterized by comprising: The device comprises: an acquisition module configured to acquire a speech signal to be recognized; a generation module configured to generate a third-order speech tensor based on the speech signal to be recognized; a determination module configured to determine a speech phase feature corresponding to the speech signal to be recognized from the third-order speech tensor; an identification module configured to determine a speech recognition result corresponding to the speech signal to be recognized based on the speech phase feature; The determination module comprises: a decomposition submodule configured to perform non-negative decomposition on the third-order speech tensor to obtain N decomposition matrix groups; each decomposition matrix group comprises a time matrix, a frequency matrix, and a speaker matrix, and N is a positive integer; a first determining sub-module, configured to determine a target time feature and a target frequency feature by using a time matrix, a frequency matrix and a speaker matrix in the decomposition matrix group; a second determining sub-module, configured to determine a speech phase feature corresponding to the to-be-identified speech signal by using the target time feature and the target frequency feature; the first determining sub-module is specifically configured to: assign a decomposition number to each of the decomposition matrix groups according to a decomposition order, and insert all the decomposition matrix groups into a decomposition matrix queue according to the decomposition numbers in the order; select and remove a decomposition matrix group with a first order from the decomposition matrix queue as a preceding matrix group, perform orthogonal constraint processing on the preceding matrix group to obtain a preceding processed matrix group; the orthogonal constraint processing includes orthogonal processing on a time matrix and the frequency matrix in the preceding matrix group, and constraint processing on a speaker matrix in the preceding matrix group; determine a first decomposition loss value between the preceding processed matrix group and the third-order speech tensor, and determine whether the first decomposition loss value meets a preset convergence condition; if it is determined that the first decomposition loss value meets the preset convergence condition, determine the time matrix in the preceding matrix group as the target time feature, and determine the frequency matrix in the preceding matrix group as the target frequency feature.

8. An electronic device, comprising: comprise: a processor and a memory, the processor being configured to execute a speech recognition program stored in the memory to implement the speech recognition method in any one of claims 1-6.

9. A storage medium, characterized by The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the speech recognition method in any one of claims 1-6.

Images