Method and device for recognizing speech

Abstract

The present invention discloses a method and apparatus for speech recognition, relating to the field of computer technology. A specific implementation of the method includes: setting initial values ​​for a Chinese character coefficient and a pinyin coefficient, generating a Chinese character mapping function based on the initial values ​​of the Chinese character coefficient, and generating a pinyin mapping function based on the initial values ​​of the pinyin coefficient; training the Chinese character mapping function and the pinyin mapping function using a plurality of preset training samples, calculating the training results as parameters of a joint loss function, and generating a target mapping function based on the calculation results; and recognizing a speech to be recognized based on the target mapping function to obtain Chinese character recognition results and pinyin recognition results for the speech to be recognized. This implementation ensures recognition accuracy while reducing recognition costs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for recognizing speech. Background Art

[0002] At present, when using existing Chinese character recognition methods to recognize speech, only Chinese character recognition results can be recognized, and when using existing pinyin recognition methods to recognize speech, only pinyin recognition results can be recognized.

[0003] In the process of implementing the present invention, the inventors discovered that the prior art has at least the following problems:

[0004] When recognizing speech containing homophones, polyphones, or ambiguous pronunciations, if only Chinese character recognition results or only Pinyin recognition results are recognized, the recognition accuracy is low. To ensure the recognition accuracy of speech containing homophones, polyphones, or ambiguous pronunciations, those skilled in the art may easily think of using Chinese character recognition methods and Pinyin recognition methods to recognize the speech separately, thereby obtaining Chinese character recognition results and Pinyin recognition results. However, using two recognition methods to recognize speech separately will result in high recognition costs. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for recognizing speech, which can ensure the recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations while reducing recognition costs.

[0006] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for recognizing speech is provided.

[0007] The method for recognizing speech according to an embodiment of the present invention includes:

[0008] Set the initial values ​​of the Chinese character coefficient and the pinyin coefficient respectively, generate a Chinese character mapping function according to the initial value of the Chinese character coefficient, and generate a pinyin mapping function according to the initial value of the pinyin coefficient;

[0009] Using a plurality of preset training samples to train the Chinese character mapping function and the pinyin mapping function, using the training results as parameters of a joint loss function for calculation, and generating a target mapping function according to the calculation results;

[0010] The speech to be recognized is recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0011] In one embodiment, the Chinese character mapping function and the pinyin mapping function are trained using a plurality of preset training samples, the training results are used as parameters of a joint loss function for calculation, and a target mapping function is generated according to the calculation results, including:

[0012] Obtaining a Chinese character loss value and a pinyin loss value of each training sample according to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples;

[0013] The Chinese character loss value and the pinyin loss value of each training sample are used as parameters of a joint loss function to calculate and obtain a joint loss value of each training sample;

[0014] The joint loss value of each training sample is calculated using a back propagation algorithm to obtain target values ​​of the Chinese character coefficient and the pinyin coefficient, and a target mapping function is generated based on the target values.

[0015] In one embodiment, the training sample includes a spectrum of a sample speech, Chinese characters matching the sample speech, and pinyin matching the sample speech;

[0016] Obtaining a Chinese character loss value and a pinyin loss value of each training sample according to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples, including:

[0017] For each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to calculate and obtain the Chinese character recognition result and the pinyin recognition result of the training sample;

[0018] The difference between the Chinese character recognition result of the training sample and the Chinese character in the training sample is used as the Chinese character loss value of the training sample;

[0019] The difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample.

[0020] In one embodiment, the Chinese character loss value and the pinyin loss value of each training sample are used as parameters of a joint loss function to calculate the joint loss value of each training sample, including:

[0021] For each training sample, multiplying the preset Chinese character weight and the Chinese character loss value of the training sample as a first value;

[0022] The product of a preset pinyin weight and the pinyin loss value of the training sample is used as a second value; wherein the sum of the Chinese character weight and the pinyin weight is 1;

[0023] The sum of the two norms of the initial values ​​of the Chinese character coefficients and the two norms of the initial values ​​of the pinyin coefficients is used as a third value, and the product of the third value and a preset regularization term coefficient is used as a fourth value;

[0024] The sum of the first value, the second value, and the fourth value is used as the joint loss value of the training sample.

[0025] In one embodiment, the speech to be recognized is recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized, including:

[0026] Performing a short-time Fourier transform on the time domain of the speech to be recognized to obtain a frequency spectrum of the speech to be recognized;

[0027] The frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to perform calculations to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0028] To achieve the above objective, according to another aspect of an embodiment of the present invention, a device for recognizing speech is provided.

[0029] The apparatus for recognizing speech according to an embodiment of the present invention includes:

[0030] A generating unit, configured to respectively set initial values ​​of the Chinese character coefficient and the pinyin coefficient, generate a Chinese character mapping function according to the initial value of the Chinese character coefficient, and generate a pinyin mapping function according to the initial value of the pinyin coefficient;

[0031] a processing unit, configured to train the Chinese character mapping function and the pinyin mapping function using a plurality of preset training samples, calculate the training results as parameters of a joint loss function, and generate a target mapping function according to the calculation results;

[0032] The recognition unit is used to recognize the speech to be recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0033] In one embodiment, the processing unit is configured to:

[0034] Obtaining a Chinese character loss value and a pinyin loss value of each training sample according to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples;

[0035] The Chinese character loss value and the pinyin loss value of each training sample are used as parameters of a joint loss function to calculate and obtain a joint loss value of each training sample;

[0036] The joint loss value of each training sample is calculated using a back propagation algorithm to obtain target values ​​of the Chinese character coefficient and the pinyin coefficient, and a target mapping function is generated based on the target values.

[0037] In one embodiment, the training sample includes a spectrum of a sample speech, Chinese characters matching the sample speech, and pinyin matching the sample speech;

[0038] The processing unit is used for:

[0039] For each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to calculate and obtain the Chinese character recognition result and the pinyin recognition result of the training sample;

[0040] The difference between the Chinese character recognition result of the training sample and the Chinese character in the training sample is used as the Chinese character loss value of the training sample;

[0041] The difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample.

[0042] In one embodiment, the processing unit is configured to:

[0043] For each training sample, multiplying the preset Chinese character weight and the Chinese character loss value of the training sample as a first value;

[0044] The product of a preset pinyin weight and the pinyin loss value of the training sample is used as a second value; wherein the sum of the Chinese character weight and the pinyin weight is 1;

[0045] The sum of the two norms of the initial values ​​of the Chinese character coefficients and the two norms of the initial values ​​of the pinyin coefficients is used as a third value, and the product of the third value and a preset regularization term coefficient is used as a fourth value;

[0046] The sum of the first value, the second value, and the fourth value is used as the joint loss value of the training sample.

[0047] In one embodiment, the identification unit is configured to:

[0048] Performing a short-time Fourier transform on the time domain of the speech to be recognized to obtain a frequency spectrum of the speech to be recognized;

[0049] The frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to perform calculations to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0050] To achieve the above objective, according to another aspect of an embodiment of the present invention, an electronic device is provided.

[0051] An electronic device according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for recognizing speech provided by the embodiment of the present invention.

[0052] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided.

[0053] A computer-readable medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements the method for recognizing speech provided by an embodiment of the present invention.

[0054] An embodiment of the above invention has the following advantages or beneficial effects: a Chinese character mapping function is generated according to the set initial value of the Chinese character coefficient, a pinyin mapping function is generated according to the set initial value of the pinyin coefficient, the Chinese character mapping function and the pinyin mapping function are trained using a preset plurality of training samples, the training results are used as parameters of the joint loss function for calculation, a target mapping function is generated according to the calculation results, and the target mapping function is used to recognize speech, so that Chinese character recognition results and pinyin recognition results can be obtained at the same time without sending two recognition requests or deploying equipment to process the two recognition requests, thereby ensuring the recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations, while reducing recognition costs and reducing the deviation in user intention understanding.

[0055] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0057] Figure 1 is a schematic diagram of the main process of a method for recognizing speech according to an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of a deep neural network in a method for recognizing speech according to an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of the main process of a method for recognizing speech according to another embodiment of the present invention;

[0060] Figure 4 1 is a schematic diagram of the main process of calculating Chinese character loss values ​​and pinyin loss values ​​in a method for recognizing speech according to another embodiment of the present invention;

[0061] Figure 5 is a schematic diagram of the main process of calculating the joint loss value in the method for recognizing speech according to another embodiment of the present invention;

[0062] Figure 6 is a schematic diagram of the accuracy of Chinese character recognition in a method for recognizing speech according to another embodiment of the present invention;

[0063] Figure 7is a schematic diagram of pinyin recognition accuracy in a method for recognizing speech according to another embodiment of the present invention;

[0064] Figure 8 is a schematic diagram of Chinese character loss values ​​in a method for speech recognition according to another embodiment of the present invention;

[0065] Figure 9 is a schematic diagram of pinyin loss values ​​in a method for recognizing speech according to another embodiment of the present invention;

[0066] Figure 10 is a schematic diagram of main units of a device for recognizing speech according to an embodiment of the present invention;

[0067] Figure 11 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0068] Figure 12 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0070] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0071] In recent years, with the rapid development of artificial intelligence (AI) technology, intelligent customer service systems have been applied in various fields, such as e-commerce, logistics, and finance. Automatic speech recognition (ASR) is one of the core technologies of intelligent customer service systems. ASR aims to convert user speech into text, understand user intent through text, and provide appropriate responses. Therefore, the accuracy of ASR is crucial. However, in actual customer service conversations, user speech often contains various issues such as accents, dialects, and background noise, which poses a significant challenge to accurate speech recognition.

[0072] To address these issues, traditional methods use a Gaussian Mixture Model (GMM) and a Hidden Markov Model (HMM) to build an acoustic model. The spectrum of the speech to be recognized is input into the acoustic model to obtain the pronunciation state (which can be a syllable). This pronunciation state is then used to query a pronunciation dictionary (which stores the correspondence between pronunciation states and recognition results) to obtain the recognition result. Traditional methods achieve speech recognition by obtaining the context-dependent state relationship of the speech (i.e., obtaining the conditional probability distribution of each time frame of the time series signal).

[0073] With the rapid development of deep learning technology, speech recognition methods based on deep neural networks (DNNs) have emerged, demonstrating significant improvements in recognition performance compared to traditional methods. For example, in 2015, D. Amodei et al. proposed Deep Speech 2 (DS2). Traditional methods build acoustic models and use the output of these models as input to a pronunciation dictionary, which results in cumulative errors. DS2, on the other hand, uses deep neural networks to directly establish a correspondence between the spectrum of the speech to be recognized and the recognition results, resulting in higher recognition accuracy.

[0074] Whether it is the traditional method or DS2, only one recognition result can be identified, that is, either the Chinese character recognition result or the pinyin recognition result. For the recognition of speech containing homophones, polyphones or ambiguous pronunciations, if only one recognition result is identified, the recognition accuracy is not high. The above-mentioned speech involves important information such as the user's affirmation, negation or key numbers, and there will be problems with the user's intention understanding deviation, and the user experience is poor. In order to ensure the recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations, and reduce the situation of user intention understanding deviation, the Chinese character recognition result and the pinyin recognition result can be identified, and the final recognition result can be determined by combining the Chinese character recognition result and the pinyin recognition result (the process of combining the Chinese character recognition result and the pinyin recognition result to determine the final recognition result is the prior art). To identify the Chinese character recognition result and the pinyin recognition result, the solution that a person skilled in the art can easily think of is: use the existing Chinese character recognition method to establish a set of models, use the existing pinyin recognition method to establish another set of models, train the two sets of models respectively, and use the two sets of models after training to perform recognition respectively. However, model establishment and model training require costs, so the prior art has the problem of high recognition cost.

[0075] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a method for recognizing speech, such as Figure 1 As shown, the method includes:

[0076] Step S101 : setting initial values ​​of the Chinese character coefficient and the pinyin coefficient respectively, generating a Chinese character mapping function according to the initial values ​​of the Chinese character coefficient, and generating a pinyin mapping function according to the initial values ​​of the pinyin coefficient.

[0077] In this step, when it is implemented, Figure 2 As shown, the Chinese character mapping function includes the operation functions of a three-layer convolutional neural network (CNN), the operation functions of a one-layer gated recurrent unit (GRU), and the operation functions of a two-layer fully connected network (FC). The pinyin mapping function also includes the operation functions of a three-layer convolutional neural network, the operation functions of a one-layer gated recurrent unit (GRU), and the operation functions of a two-layer fully connected network. The CNN operation function in the Chinese character mapping function is the same as the CNN operation function in the pinyin mapping function, and the GRU operation function in the Chinese character mapping function is the same as the GRU operation function in the pinyin mapping function, but the FC operation function in the Chinese character mapping function is different from the FC operation function in the pinyin mapping function.

[0078] The following is a specific example to illustrate step S101:

[0079] l1=c(x,P1,O1)

[0080] l2=c(l1,P2,O2)

[0081] l3=c(l2,P3,O3)

[0082] Here, c(x, P1, O1) represents the operation function of the first convolutional neural network layer, c(l1, P2, O2) represents the operation function of the second convolutional neural network layer, and c(l2, P3, O3) represents the operation function of the third convolutional neural network layer. l1 represents the operation result of the first convolutional neural network layer, l2 represents the operation result of the second convolutional neural network layer, and l3 represents the operation result of the third convolutional neural network layer. x represents the spectrum (x is the independent variable), P1 represents the convolution kernel parameter of the first convolutional neural network layer, P2 represents the convolution kernel parameter of the second convolutional neural network layer, and P3 represents the convolution kernel parameter of the third convolutional neural network layer. O1 represents the step size parameter of the first convolutional neural network layer, O2 represents the step size parameter of the second convolutional neural network layer, and O3 represents the step size parameter of the third convolutional neural network layer. c(x, P1, O1), c(l1, P2, O2), and c(l2, P3, O3) can use the existing convolutional neural network operation functions.

[0083] l4=g(l3,N1,M1)

[0084] Here, g(l3,N1,M1) represents the gated recurrent network's operation function, l4 represents the gated recurrent network's operation result, N1 represents the gated recurrent network's parameters, and M1 represents the number of hidden layers in the gated recurrent network. g(l3,N1,M1) can be implemented using any existing gated recurrent network's operation function.

[0085] l5_1=W4_1*l4+b4_1

[0086] l5_2=W4_2*l4+b4_2

[0087] Among them, W4_1*l4+b4_1 represents the operation function of the first layer of fully connected network of Chinese characters, W4_2*l4+b4_2 represents the operation function of the first layer of fully connected network of pinyin, l5_1 represents the operation result of the first layer of fully connected network of Chinese characters, l5_2 represents the operation result of the first layer of fully connected network of pinyin, W4_1 represents the weight matrix of the first layer of fully connected network of Chinese characters, W4_2 represents the weight matrix of the first layer of fully connected network of pinyin, b4_1 represents the bias vector of the first layer of fully connected network of Chinese characters, and b4_2 represents the bias vector of the first layer of fully connected network of pinyin.

[0088] l6_1=W5_1*l5_1+b5_1

[0089] l6_2=W5_2*l5_2+b5_2

[0090] Among them, W5_1*l5_1+b5_1 represents the operation function of the second-layer fully connected network of Chinese characters, W5_2*l5_2+b5_2 represents the operation function of the second-layer fully connected network of pinyin, l6_1 (l6_1 is the first dependent variable) represents the operation result of the second-layer fully connected network of Chinese characters, l6_2 (l6_2 is the second dependent variable) represents the operation result of the second-layer fully connected network of pinyin, W5_1 represents the weight matrix of the second-layer fully connected network of Chinese characters, W5_2 represents the weight matrix of the second-layer fully connected network of pinyin, b5_1 represents the bias vector of the second-layer fully connected network of Chinese characters, and b5_2 represents the bias vector of the second-layer fully connected network of pinyin.

[0091] Set the initial values ​​of the Chinese character coefficients (the Chinese character coefficients include P1, O1, P2, O2, P3, O3, N1, M1, W4_1, b4_1, W5_1 and b5_1) and the pinyin coefficients (the pinyin coefficients include P1, O1, P2, O2, P3, O3, N1, M1, W4_2, b4_2, W5_2 and b5_2), substitute the initial values ​​of the Chinese character coefficients into the operation function of the second-layer fully connected network of the Chinese characters, and use the resulting function as the Chinese character mapping function; substitute the initial values ​​of the pinyin coefficients into the operation function of the second-layer fully connected network of the pinyin, and use the resulting function as the pinyin mapping function.

[0092] Step S102: Use a plurality of preset training samples to train the Chinese character mapping function and the pinyin mapping function, use the training results as parameters of the joint loss function for calculation, and generate a target mapping function according to the calculation results.

[0093] In this step, the plurality of training samples may include 3000 kinds of Chinese characters and 1200 kinds of pinyin. The more training samples there are, the more kinds of Chinese characters and pinyin there are. Figure 3 The embodiment shown.

[0094] Step S103: Recognize the speech to be recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0095] In this step, when implemented, short-time Fourier transform is performed on the time domain of the speech to be recognized to obtain the spectrum of the speech to be recognized; the spectrum of the speech to be recognized is used as the parameter of the target mapping function to calculate and obtain the Chinese character recognition result and pinyin recognition result of the speech to be recognized.

[0096] In this embodiment, a Chinese character mapping function is generated according to the set initial value of the Chinese character coefficient, and a pinyin mapping function is generated according to the set initial value of the pinyin coefficient. The Chinese character mapping function and the pinyin mapping function are trained using a preset plurality of training samples. The training results are used as parameters of the joint loss function for calculation, and a target mapping function is generated according to the calculation results. The target mapping function is used to recognize speech, and Chinese character recognition results and pinyin recognition results can be obtained at the same time without sending two recognition requests or deploying equipment to process the two recognition requests. This ensures the recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations, while reducing recognition costs and reducing deviations in user intention understanding.

[0097] In order to solve the problems existing in the prior art, another embodiment of the present invention provides a method for recognizing speech. In this embodiment, Figure 1 Based on the embodiment shown, Figure 3 As shown, step S102 includes:

[0098] Step S301: Obtain a Chinese character loss value and a Pinyin loss value of each training sample according to the Chinese character mapping function, the Pinyin mapping function and a plurality of preset training samples.

[0099] In this step, the specific implementation of step S301 can refer to Figure 4 The embodiment shown.

[0100] Step S302: The Chinese character loss value and the pinyin loss value of each training sample are used as parameters of a joint loss function to calculate and obtain a joint loss value of each training sample.

[0101] In this step, the joint loss function is similar to the existing connectionist temporal classification (CTC) loss function, except that the parameters of the joint loss function include Chinese character loss value and pinyin loss value, while the parameters of the existing connectionist temporal classification loss function only include Chinese character loss value or only include pinyin loss value. The specific implementation of step S302 can refer to Figure 5 The embodiment shown.

[0102] Step S303: Calculate the joint loss value of each training sample using a back propagation algorithm to obtain target values ​​of the Chinese character coefficient and the pinyin coefficient, and generate a target mapping function based on the target values.

[0103] In this step, the Adaptive Moment Estimation (ADAM) method is used to calculate the target values ​​for the Chinese character coefficient and the pinyin coefficient based on the joint loss value of each training sample. The ADAM method is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process.

[0104] It should be understood that the initial values ​​of the Chinese character coefficient and the pinyin coefficient are set arbitrarily. Only through training with multiple training samples, joint loss function calculation, and back propagation algorithm calculation can the target values ​​of the Chinese character coefficient and the pinyin coefficient be obtained.

[0105] Based on the example given in step S101, a specific example is used to illustrate how to generate a target mapping function based on the target value: the target value of the Chinese character coefficient is substituted into the operation function of the second-layer fully connected network of the Chinese character, and the obtained function is used as the target Chinese character mapping function, and the target value of the pinyin coefficient is substituted into the operation function of the second-layer fully connected network of the pinyin, and the obtained function is used as the target pinyin mapping function. The target mapping function includes a target Chinese character mapping function and a target pinyin mapping function.

[0106] In this embodiment, the Chinese character loss value and the pinyin loss value of each training sample are obtained according to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples; the Chinese character loss value and the pinyin loss value of each training sample are used as parameters of the joint loss function for calculation to obtain the joint loss value of each training sample; the back propagation algorithm is used to calculate according to the joint loss value of each training sample to obtain the target values ​​of the Chinese character coefficient and the pinyin coefficient, and the target mapping function is generated according to the target values, so as to simultaneously identify the Chinese character recognition result and the pinyin recognition result, without sending two recognition requests and without deploying equipment to process the two recognition requests, further ensuring the recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations, while reducing the recognition cost and the deviation in understanding the user's intention.

[0107] In the embodiment of the present invention, Figure 4 As shown, the training sample includes the spectrum of the sample speech, the Chinese characters matching the sample speech, and the pinyin matching the sample speech; step S301 includes:

[0108] Step S401: for each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to perform calculations, and obtain the Chinese character recognition result and the pinyin recognition result of the training sample.

[0109] In this step, the training sample can be obtained as follows: a short-time Fourier transform is performed on the time domain of the sample speech to obtain the spectrum of the sample speech; the Chinese characters that match the sample speech are manually annotated (after the manually annotated Chinese characters that match the sample speech are screened, any annotated errors must be corrected promptly) and the pinyin that matches the sample speech (the pinyin that matches the sample speech can also be obtained by converting the Chinese characters that match the sample speech using an open source tool for converting Chinese characters to pinyin). Of course, only one training sample can be obtained using a piece of sample speech.

[0110] The following is a specific example to illustrate the training sample. The expression of the training sample is:

[0111]

[0112] Among them, i∈[1,s], s represents the number of training samples, x i represents the spectrum of the sample speech, Indicates the Chinese characters that match the sample speech. Indicates the pinyin that matches the sample speech.

[0113] It should be understood that the more scenarios the sample speech involves and the greater the number of training samples, the more robust the speech recognition method provided by the embodiments of the present invention will be. Of course, the sample speech can be speech in a financial customer service scenario, the total length of all sample speech can be 5,000 hours, and can include Chinese characters, numbers, or letters, etc. The sample speech can be sampled at a rate of 8 kHz.

[0114] It should be noted that there is a gap between the Chinese character recognition results of the training samples and the Chinese characters in the training samples. Similarly, there is a gap between the pinyin recognition results of the training samples and the pinyin in the training samples. The embodiment of the present invention aims to make both gaps as small as possible, so as to obtain the target values ​​of the Chinese character coefficient and the pinyin coefficient, and then obtain the target mapping function.

[0115] Based on the example given in step S101, this step is illustrated with a specific example: the spectrum in the Chinese character mapping function is an independent variable, and the spectrum in the training sample is calculated as the independent variable of the Chinese character mapping function to obtain the Chinese character recognition result of the training sample (i.e., l6_1); similarly, the spectrum in the pinyin mapping function is also an independent variable, and the spectrum in the training sample is calculated as the independent variable of the pinyin mapping function to obtain the pinyin recognition result of the training sample (i.e., l6_2).

[0116] Step S402: The difference between the Chinese character recognition result of the training sample and the Chinese characters in the training sample is used as the Chinese character loss value of the training sample.

[0117] In this step, the Chinese characters in the training sample are the Chinese characters included in the training sample and matching the sample speech.

[0118] Step S403: The difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample.

[0119] In this step, the pinyin in the training sample is the pinyin included in the training sample that matches the sample speech.

[0120] In this embodiment, for each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to calculate the Chinese character recognition result and the pinyin recognition result of the training sample; the difference between the Chinese character recognition result of the training sample and the Chinese characters in the training sample is used as the Chinese character loss value of the training sample; and the difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample. The target mapping function is obtained based on the Chinese character loss value and the pinyin loss value of the training sample, thereby simultaneously identifying the Chinese character recognition result and the pinyin recognition result, without sending two recognition requests or deploying equipment to process the two recognition requests. This further ensures the recognition accuracy of speech containing homophones, polyphones, or ambiguous pronunciations, while reducing recognition costs and reducing the situation of deviations in understanding user intent.

[0121] In the embodiment of the present invention, Figure 5 As shown, step S302 includes:

[0122] Step S501: For each training sample, the product of a preset Chinese character weight and the Chinese character loss value of the training sample is used as a first value.

[0123] In this step, the value range of Chinese character weight is [0,1].

[0124] Step S502: taking the product of the preset pinyin weight and the pinyin loss value of the training sample as the second value; wherein the sum of the Chinese character weight and the pinyin weight is 1.

[0125] In this step, the pinyin weight ranges from [0, 1]. By changing the Chinese character weight and the pinyin weight, the purpose of the target mapping function can be changed. For example, if the Chinese character weight is 0 and the pinyin weight is 1, then the target mapping function can only recognize the pinyin recognition result; if the Chinese character weight is 1 and the pinyin weight is 0, then the target mapping function can only recognize the Chinese character recognition result; if the Chinese character weight is 0.5 and the pinyin weight is 0.5, then the target mapping function can recognize both the pinyin recognition result and the Chinese character recognition result.

[0126] The Chinese character weights are taken as 0.25, 0.5 and 0.75 respectively, and the corresponding pinyin weights are taken as 0.75, 0.5 and 0.25 respectively, and the Chinese character recognition accuracy, pinyin recognition accuracy, Chinese character loss value and pinyin loss value are obtained. Figure 6 As shown in the figure, the pinyin recognition accuracy is as follows Figure 7 As shown, from Figure 6 and Figure 7It can be seen that the target mapping function can converge effectively and quickly, and the Chinese character recognition accuracy and pinyin recognition accuracy are both greater than 80%, among which the Chinese character recognition accuracy converges to 83% and the pinyin recognition accuracy converges to 87%. Figure 8 As shown, the pinyin loss value is as follows Figure 9 As shown, from Figure 8 and Figure 9 It can be seen that the loss value of Chinese characters converges to 12.3, and the loss value of Pinyin converges to 9.3. From this, it can be seen that the Pinyin recognition accuracy is better than the Chinese character recognition accuracy. This is because there are 3000 types of Chinese characters and 1200 types of Pinyin, and there are more types of Chinese characters than Pinyin. At the same time, Figure 6-Figure 9 It can be seen that for the joint loss value, the value of the Chinese character weight has a greater impact on the convergence rate. A smaller value of the Chinese character weight slows the convergence rate, but has little impact on the final convergence result. Therefore, the most appropriate value of the Chinese character weight is 0.5, which balances the Chinese character loss value and the pinyin loss value.

[0127] Step S503: taking the sum of the binary norm of the initial value of the Chinese character coefficient and the binary norm of the initial value of the pinyin coefficient as the third value, and taking the product of the third value and the preset regularization term coefficient as the fourth value.

[0128] In this step, the regularization term coefficient can be set to 10 -5 The role of the regularization term coefficient is to suppress the Chinese character coefficient and the pinyin coefficient, and prevent the target mapping function from failing to converge because the Chinese character coefficient and the pinyin coefficient are too large.

[0129] Step S504: taking the sum of the first value, the second value, and the fourth value as the joint loss value of the training sample.

[0130] The following is a specific example to illustrate this embodiment:

[0131] The expression of the joint loss function is:

[0132]

[0133] Among them, L represents the joint loss value of the training sample, α represents the weight of Chinese characters, (1-α) represents the weight of pinyin, β represents the regularization coefficient, and L h Represents the Chinese character loss value of the training sample, L p represents the pinyin loss value of the training sample, w k Indicates the initial value of the Chinese character coefficient and the initial value of the pinyin coefficient.

[0134] The weight of Chinese characters is set to 0.5, the weight of Pinyin is set to 0.5, and the regularization coefficient is set to 10 -5 .

[0135] For each training sample, the Chinese character loss value, the pinyin loss value, the Chinese character weight (0.5), the pinyin weight (0.5), the regularization coefficient (10 -5 ) and the initial values ​​of the pre-set Chinese character coefficient and pinyin coefficient are substituted into the expression of the joint loss function for calculation to obtain the joint loss value of the training sample.

[0136] In this embodiment, for each training sample, the product of a preset Chinese character weight and the Chinese character loss value of the training sample is used as a first value; the product of a preset pinyin weight and the pinyin loss value of the training sample is used as a second value; wherein the sum of the Chinese character weight and the pinyin weight is 1; the sum of the two norms of the initial values ​​of the Chinese character coefficient and the two norms of the initial values ​​of the pinyin coefficient is used as a third value, and the product of the third value and the preset regularization term coefficient is used as a fourth value; and the sum of the first value, the second value, and the fourth value is used as a joint loss value of the training sample. The target mapping function is obtained from the joint loss value of the training sample, thereby simultaneously identifying the Chinese character recognition result and the pinyin recognition result, without sending two recognition requests or deploying equipment to process the two recognition requests, further ensuring the recognition accuracy of speech containing homophones, polyphones, or ambiguous pronunciations, while reducing recognition costs and reducing the situation of deviation in understanding user intent.

[0137] In this embodiment of the present invention, step S103 includes:

[0138] Performing short-time Fourier transform on the time domain of the speech to be recognized to obtain the frequency spectrum of the speech to be recognized.

[0139] The frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to perform calculations to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0140] In this embodiment, based on the example given in step S303, a specific example is used to illustrate that the frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to calculate and obtain the Chinese character recognition result and pinyin recognition result of the speech to be recognized: the frequency spectrum in the target Chinese character mapping function is the independent variable, and the frequency spectrum in the target pinyin mapping function is the independent variable. The frequency spectrum of the speech to be recognized is used as the independent variable of the target Chinese character mapping function and the target pinyin mapping function respectively to calculate and obtain the Chinese character recognition result (i.e., l6_1) and pinyin recognition result (i.e., l6_2) of the speech to be recognized.

[0141] The Short-Time Fourier Transform (STFT) uses a 20ms window size and a 10ms window step. Therefore, the resulting spectrum of the speech to be recognized is an 81-dimensional spectral information sequence. The STFT is a Fourier transform method used to determine the frequency and phase of a localized sinusoidal wave in a time-varying signal.

[0142] In this embodiment, by performing a short-time Fourier transform on the time domain of the speech to be recognized, the spectrum of the speech to be recognized is obtained, and the spectrum is used as the parameter of the target mapping function for calculation to obtain the Chinese character recognition result and the pinyin recognition result of the speech to be recognized. There is no need to send two recognition requests and there is no need to deploy equipment to process the two recognition requests, which further ensures the recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations, while reducing the recognition cost.

[0143] It should be understood that the embodiment of the present invention combines the model used in the Chinese character recognition method and the model used in the pinyin recognition method into one, and simultaneously performs Chinese character recognition training and pinyin recognition training on the combined model, thereby obtaining a target mapping function that can simultaneously recognize Chinese character recognition results and pinyin recognition results, thereby reducing the recognition cost.

[0144] Combined with the above Figures 1-9 The method of recognizing speech is described below. Figure 10 A device for recognizing speech is described.

[0145] In order to solve the problems existing in the prior art, the embodiment of the present invention provides a device for recognizing speech, such as Figure 10 As shown, the device includes:

[0146] The generating unit 1001 is used to set initial values ​​of the Chinese character coefficient and the pinyin coefficient respectively, generate a Chinese character mapping function according to the initial values ​​of the Chinese character coefficient, and generate a pinyin mapping function according to the initial values ​​of the pinyin coefficient.

[0147] The processing unit 1002 is configured to train the Chinese character mapping function and the pinyin mapping function using a plurality of preset training samples, calculate the training results as parameters of a joint loss function, and generate a target mapping function according to the calculation results.

[0148] The recognition unit 1003 is configured to recognize the speech to be recognized according to the target mapping function, and obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0149] In this embodiment of the present invention, the processing unit 1002 is configured to:

[0150] Obtaining a Chinese character loss value and a pinyin loss value of each training sample according to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples;

[0151] The Chinese character loss value and the pinyin loss value of each training sample are used as parameters of a joint loss function to calculate and obtain a joint loss value of each training sample;

[0152] The joint loss value of each training sample is calculated using a back propagation algorithm to obtain target values ​​of the Chinese character coefficient and the pinyin coefficient, and a target mapping function is generated based on the target values.

[0153] In an embodiment of the present invention, the training sample includes a spectrum of a sample speech, Chinese characters matching the sample speech, and pinyin matching the sample speech;

[0154] The processing unit 1002 is configured to:

[0155] For each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to calculate and obtain the Chinese character recognition result and the pinyin recognition result of the training sample;

[0156] The difference between the Chinese character recognition result of the training sample and the Chinese character in the training sample is used as the Chinese character loss value of the training sample;

[0157] The difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample.

[0158] In this embodiment of the present invention, the processing unit 1002 is configured to:

[0159] For each training sample, multiplying the preset Chinese character weight and the Chinese character loss value of the training sample as a first value;

[0160] The product of a preset pinyin weight and the pinyin loss value of the training sample is used as a second value; wherein the sum of the Chinese character weight and the pinyin weight is 1;

[0161] The sum of the two norms of the initial values ​​of the Chinese character coefficients and the two norms of the initial values ​​of the pinyin coefficients is used as a third value, and the product of the third value and a preset regularization term coefficient is used as a fourth value;

[0162] The sum of the first value, the second value, and the fourth value is used as the joint loss value of the training sample.

[0163] In this embodiment of the present invention, the identification unit 1003 is configured to:

[0164] Performing a short-time Fourier transform on the time domain of the speech to be recognized to obtain a frequency spectrum of the speech to be recognized;

[0165] The frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to perform calculations to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

[0166] It should be understood that the functions performed by the various components of the apparatus for recognizing speech provided in the embodiment of the present invention have been described in detail in the method for recognizing speech in the above embodiment, and will not be repeated here.

[0167] Figure 11 An exemplary system architecture 1100 is shown to which the method for recognizing speech or the apparatus for recognizing speech according to the embodiments of the present invention may be applied.

[0168] like Figure 11 As shown, system architecture 1100 may include terminal devices 1101, 1102, and 1103, a network 1104, and a server 1105. Network 1104 is used to provide a medium for communication links between terminal devices 1101, 1102, and 1103 and server 1105. Network 1104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0169] Users can use terminal devices 1101 , 1102 , and 1103 to interact with server 1105 via network 1104 to receive or send messages, etc.

[0170] The terminal devices 1101 , 1102 , and 1103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0171] Server 1105 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 1101, 1102, and 1103. The backend management server may analyze and process received data such as product information query requests, and feed back processing results (for example, target push information and product information—for example only) to the terminal device.

[0172] It should be noted that the method for recognizing speech provided in the embodiment of the present invention is generally executed by the server 1105 , and accordingly, the device for recognizing speech is generally set in the server 1105 .

[0173] It should be understood that Figure 11 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0174] Reference below Figure 12, which shows a schematic structural diagram of a computer system 1200 of a terminal device suitable for implementing an embodiment of the present invention. Figure 12 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0175] like Figure 12 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage unit 1208 into a random access memory (RAM) 1203. Various programs and data required for the operation of the system 1200 are also stored in the RAM 1203. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0176] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, and the like; an output section 1207 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1208 including a hard disk; and a communication section 1209 including a network interface card such as a LAN card or a modem. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. Removable media 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1210 as needed, so that computer programs read therefrom can be installed into the storage section 1208 as needed.

[0177] In particular, according to the embodiments disclosed herein, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed herein include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from a removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the system of the present invention are performed.

[0178] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a unit, program segment, or a part of code, and the above-mentioned unit, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The units involved in the embodiments of the present invention may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as: a processor comprising a generation unit, a processing unit, and a recognition unit. The names of these units do not, in some cases, limit the units themselves. For example, the recognition unit may also be described as "a unit that recognizes the speech to be recognized according to the target mapping function and obtains the Chinese character recognition result and the pinyin recognition result of the speech to be recognized."

[0181] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: setting initial values ​​of Chinese character coefficients and pinyin coefficients respectively, generating a Chinese character mapping function according to the initial values ​​of the Chinese character coefficients, and generating a pinyin mapping function according to the initial values ​​of the pinyin coefficients; training the Chinese character mapping function and the pinyin mapping function with a plurality of preset training samples, calculating the training results as parameters of the joint loss function, and generating a target mapping function according to the calculation results; recognizing the speech to be recognized according to the target mapping function to obtain the Chinese character recognition result and the pinyin recognition result of the speech to be recognized.

[0182] According to the technical solution of the embodiment of the present invention, a Chinese character mapping function is generated according to the set initial value of the Chinese character coefficient, and a pinyin mapping function is generated according to the set initial value of the pinyin coefficient. The Chinese character mapping function and the pinyin mapping function are trained using a preset plurality of training samples. The training results are used as parameters of the joint loss function for calculation. A target mapping function is generated according to the calculation results. The target mapping function is used to recognize speech, and Chinese character recognition results and pinyin recognition results can be obtained at the same time without sending two recognition requests or deploying equipment to process the two recognition requests. The recognition accuracy of speech containing homophones, polyphones or ambiguous pronunciations is guaranteed, while reducing the recognition cost and the deviation in user intention understanding.

[0183] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for recognizing speech, characterized in that: include: Set the initial values ​​of the Chinese character coefficient and the pinyin coefficient respectively, generate a Chinese character mapping function according to the initial value of the Chinese character coefficient, and generate a pinyin mapping function according to the initial value of the pinyin coefficient; According to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples, a Chinese character loss value and a pinyin loss value of each training sample are obtained; the Chinese character loss value and the pinyin loss value of each training sample are used as parameters of a joint loss function for calculation to obtain a joint loss value of each training sample; according to the joint loss value of each training sample, a back propagation algorithm is used to calculate to obtain target values ​​of a Chinese character coefficient and a pinyin coefficient, and a target mapping function is generated according to the target values; The speech to be recognized is recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

2. The method according to claim 1, characterized in that The training sample includes a spectrum of a sample speech, Chinese characters matching the sample speech, and pinyin matching the sample speech; Obtaining a Chinese character loss value and a pinyin loss value of each training sample according to the Chinese character mapping function, the pinyin mapping function and a plurality of preset training samples, including: For each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to calculate and obtain the Chinese character recognition result and the pinyin recognition result of the training sample; The difference between the Chinese character recognition result of the training sample and the Chinese character in the training sample is used as the Chinese character loss value of the training sample; The difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample.

3. The method according to claim 1, characterized in that The Chinese character loss value and the pinyin loss value of each training sample are used as parameters of the joint loss function to calculate the joint loss value of each training sample, including: For each training sample, multiplying the preset Chinese character weight and the Chinese character loss value of the training sample as a first value; The product of a preset pinyin weight and the pinyin loss value of the training sample is used as a second value; wherein the sum of the Chinese character weight and the pinyin weight is 1; The sum of the two norms of the initial values ​​of the Chinese character coefficients and the two norms of the initial values ​​of the pinyin coefficients is used as a third value, and the product of the third value and a preset regularization term coefficient is used as a fourth value; The sum of the first value, the second value, and the fourth value is used as the joint loss value of the training sample.

4. The method according to claim 1, wherein Recognizing the speech to be recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized, including: Performing a short-time Fourier transform on the time domain of the speech to be recognized to obtain a frequency spectrum of the speech to be recognized; The frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to perform calculations to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

5. A device for recognizing speech, characterized in that: include: A generating unit, configured to respectively set initial values ​​of the Chinese character coefficient and the pinyin coefficient, generate a Chinese character mapping function according to the initial value of the Chinese character coefficient, and generate a pinyin mapping function according to the initial value of the pinyin coefficient; a processing unit, configured to obtain a Chinese character loss value and a Pinyin loss value for each training sample based on the Chinese character mapping function, the Pinyin mapping function, and a plurality of preset training samples; calculate the Chinese character loss value and the Pinyin loss value for each training sample as parameters of a joint loss function to obtain a joint loss value for each training sample; calculate using a back propagation algorithm based on the joint loss value of each training sample to obtain target values ​​for a Chinese character coefficient and a Pinyin coefficient, and generate a target mapping function based on the target values; The recognition unit is used to recognize the speech to be recognized according to the target mapping function to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

6. The device according to claim 5, characterized in that The training sample includes a spectrum of a sample speech, Chinese characters matching the sample speech, and pinyin matching the sample speech; The processing unit is used for: For each preset training sample, the frequency spectrum in the training sample is used as the parameters of the Chinese character mapping function and the pinyin mapping function to calculate and obtain the Chinese character recognition result and the pinyin recognition result of the training sample; The difference between the Chinese character recognition result of the training sample and the Chinese character in the training sample is used as the Chinese character loss value of the training sample; The difference between the pinyin recognition result of the training sample and the pinyin in the training sample is used as the pinyin loss value of the training sample.

7. The device according to claim 5, characterized in that The processing unit is used for: For each training sample, multiplying the preset Chinese character weight and the Chinese character loss value of the training sample as a first value; The product of a preset pinyin weight and the pinyin loss value of the training sample is used as a second value; wherein the sum of the Chinese character weight and the pinyin weight is 1; The sum of the two norms of the initial values ​​of the Chinese character coefficients and the two norms of the initial values ​​of the pinyin coefficients is used as a third value, and the product of the third value and a preset regularization term coefficient is used as a fourth value; The sum of the first value, the second value, and the fourth value is used as the joint loss value of the training sample.

8. The device according to claim 5, characterized in that The identification unit is used for: Performing a short-time Fourier transform on the time domain of the speech to be recognized to obtain a frequency spectrum of the speech to be recognized; The frequency spectrum of the speech to be recognized is used as a parameter of the target mapping function to perform calculations to obtain a Chinese character recognition result and a pinyin recognition result of the speech to be recognized.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

10. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images