Speech recognition processing method, device and equipment

Abstract

The present invention provides a speech recognition processing method, device and equipment. The method of the present invention comprises: obtaining the parameter similarity between the same network layer in a first teacher model and a second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; in the process of training a student model using pure accent data, adjusting the total loss function of the corresponding network layer in the student model according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent. The present invention utilizes the difference in acoustic characteristics between Mandarin data and accent data to calculate the parameter similarity of each intermediate network layer in the teacher model, and uses the parameter similarity to guide the adjustment of the student model loss function, thereby achieving the purpose of dynamically guiding the training of the student model, thereby improving the accuracy of speech recognition with an accent.

Description

Technical Field

[0001] The present invention relates to the field of information technology application technology, and in particular to a speech recognition processing method, device and equipment. Background Art

[0002] In countries with multiple accents, the presence of accents means that speakers from different regions will pronounce with different regional accent characteristics. As the first language of many speakers, accents have a certain impact on their pronunciation during the process of learning standard Mandarin, causing many pronunciations to deviate from standard Mandarin. This causes speech recognition systems built using standard Mandarin to have a reduced recognition rate when encountering accented speech due to the mismatch. Using accent data alone to train a model is not only difficult to obtain specific accent data, making it impossible to fully model using complex network structures, but also difficult to achieve high-accuracy recognition results if a lightweight network with a simple network structure is used directly to model it. Therefore, improving the recognition of accented speech has become a key research topic in speech recognition. Summary of the Invention

[0003] The object of the present invention is to provide a speech recognition processing method, apparatus and device to solve the problem of low accuracy in recognizing accented speech.

[0004] In order to achieve the above object, the present invention provides a speech recognition processing method, comprising:

[0005] Obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, where the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data;

[0006] In the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent.

[0007] The method of obtaining the parameter similarity between the same network layer in the first teacher model and the second teacher model includes:

[0008] Using pure Mandarin data to train the first teacher model to obtain parameters of each network layer in the first teacher model;

[0009] Training the second teacher model using a mixture of Mandarin data and accent data to obtain parameters of each network layer in the second teacher model;

[0010] Based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model, the parameter similarity between the same network layers in the first teacher model and the second teacher model is calculated.

[0011] The step of calculating the parameter similarity between the same network layers in the first teacher model and the second teacher model based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model includes:

[0012] Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model;

[0013] The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

[0014] According to the parameter similarity, the total loss function of the corresponding network layer in the student model is adjusted to obtain the student model after training, including:

[0015] According to the parameter similarity, weighting the first loss function in the total loss function of each network layer in the student model to obtain an adjusted total loss function of each network layer in the student model;

[0016] The student model is trained using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

[0017] Wherein, according to the parameter similarity, before weighting the first loss function in the total loss function of each network layer in the student model to obtain the adjusted total loss function of each network layer in the student model, the method further includes:

[0018] Obtaining a first loss function for each network layer in the student model according to an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data;

[0019] Obtaining a second loss function for each network layer in the student model based on the output results of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer;

[0020] The first loss function of each network layer in the student model and the second loss function of the network layer are summed up respectively to obtain the total loss function of each network layer in the student model.

[0021] The present invention also provides a speech recognition processing device, comprising:

[0022] A first acquisition module is used to obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data;

[0023] The first processing module is used to adjust the total loss function of the corresponding network layer in the student model according to the parameter similarity during the process of training the student model using pure accent data to obtain a student model after training, and the student model is used to recognize speech information with an accent.

[0024] The present invention also provides a speech recognition processing device, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, and the processor is configured to perform the following operations:

[0025] Obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, where the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data;

[0026] In the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent.

[0027] The processor is further configured to:

[0028] Using pure Mandarin data to train the first teacher model to obtain parameters of each network layer in the first teacher model;

[0029] Using the mixed data of Mandarin data and accent data to train the second teacher model to obtain parameters of each network layer in the second teacher model;

[0030] Based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model, the parameter similarity between the same network layers in the first teacher model and the second teacher model is calculated.

[0031] The processor is further configured to:

[0032] Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model;

[0033] The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

[0034] The processor is further configured to:

[0035] According to the parameter similarity, weighting the first loss function in the total loss function of each network layer in the student model to obtain an adjusted total loss function of each network layer in the student model;

[0036] The student model is trained using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

[0037] The processor is further configured to:

[0038] Obtaining a first loss function for each network layer in the student model according to an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data;

[0039] Obtaining a second loss function for each network layer in the student model based on the output results of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer;

[0040] The first loss function of each network layer in the student model and the second loss function of the network layer are summed up respectively to obtain the total loss function of each network layer in the student model.

[0041] The present invention also provides a speech recognition processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the speech recognition processing method as described above is implemented.

[0042] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the speech recognition processing method as described above when the program is executed by a processor.

[0043] The above technical solution of the present invention has at least the following beneficial effects:

[0044] In an embodiment of the present invention, the parameter similarity between the same network layers in the first teacher model and the second teacher model is obtained, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; in the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training. The student model is used to recognize speech information with an accent. In this way, the difference in acoustic characteristics between Mandarin data and accent data is used to calculate the parameter similarity of each intermediate network layer in the teacher model, and the parameter similarity is used to guide the adjustment of the student model loss function, thereby achieving the purpose of dynamically guiding the training of the student model, thereby improving the accuracy of speech recognition with an accent. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram showing a flow chart of a speech recognition processing method according to an embodiment of the present invention;

[0046] Figure 2 A schematic diagram showing the composition structure of a teacher model according to an embodiment of the present invention;

[0047] Figure 3 A schematic diagram showing a teacher model guiding a student model according to an embodiment of the present invention;

[0048] Figure 4 A schematic diagram showing modules of a speech recognition processing device according to an embodiment of the present invention;

[0049] Figure 5 A schematic diagram showing the structure of a speech recognition processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0051] The present invention addresses the problem of low accuracy in the prior art of speech recognition with accents and provides a speech recognition processing method, apparatus and device.

[0052] like Figure 1 FIG. 1 is a flow chart of a speech recognition processing method according to an embodiment of the present invention. The method may include:

[0053] Step 101: Obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data;

[0054] Here, the first teacher model and the second teacher model are both models for speech recognition and have the same composition structure. They are called teacher models because they are relative to the student model. The teacher model has large parameters and a relatively complex structure.

[0055] Step 102, in the process of training the student model using pure accent data, adjust the total loss function of the corresponding network layer in the student model according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent.

[0056] It should be noted that the greater the parameter similarity value between the first and second teacher models at the same network layer, the smaller the difference between the accent data features and the pure Mandarin data features in that layer. The subsequent student model corresponding to that layer should learn more knowledge from the large model (i.e., the teacher model). Conversely, the smaller the parameter similarity value between the first and second teacher models at the same network layer, the greater the difference between the accent data features and the pure Mandarin data features in that layer. The subsequent student model corresponding to that layer should rely more on the features of its own model's accent data and learn less knowledge from the large model, thereby dynamically guiding the training of the student model.

[0057] The speech recognition processing method of an embodiment of the present invention obtains the parameter similarity between the same network layers in a first teacher model and a second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; in the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent. In this way, the difference in acoustic characteristics between Mandarin data and accent data is used to calculate the parameter similarity in each intermediate network layer of the teacher model, and the parameter similarity is used to guide the adjustment of the student model loss function, thereby achieving the purpose of dynamically guiding the training of the student model, and further improving the accuracy of speech recognition with an accent.

[0058] As an optional implementation, refer to Figure 2 In step 101 of the method of the embodiment of the present invention, obtaining the parameter similarity between the same network layer in the first teacher model and the second teacher model may include:

[0059] Using pure Mandarin data to train the first teacher model to obtain parameters of each network layer in the first teacher model;

[0060] Here, pure Mandarin data is used to train the first teacher model, and the parameters of different network layers in the teacher model are obtained, such as Figure 2 , expressed as ωi,1 ,ω i,2 …,ω i,L , where i represents different output nodes of the network (output nodes of different network layers), and L represents the number of model layers.

[0061] Training the second teacher model using a mixture of Mandarin data and accent data to obtain parameters of each network layer in the second teacher model;

[0062] Here, the second teacher model is trained using a mixture of Mandarin data and accent data to obtain the parameters of different network layers in the teacher model, such as Figure 2 , which can be expressed as ω j,1 ,ω j,2 …,ω j,L , where j represents different output nodes of the network (output nodes of different network layers), and L represents the number of model layers.

[0063] Based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model, the parameter similarity between the same network layers in the first teacher model and the second teacher model is calculated.

[0064] It should be noted that there is a parameter similarity between every two identical network layers in the first teacher model and the second teacher model. Figure 2 , through the above calculation, we can get L parameter similarities.

[0065] This step may specifically include:

[0066] Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model;

[0067] Here, the distance value between the parameters corresponding to the same network layer in the first teacher model and the second teacher model can be calculated using the following formula:

[0068] d k =||ω i,k -ω j,k || 2

[0069] Among them, k represents different network layers, d k Represents the distance value between the parameters corresponding to the same network layer k in the first teacher model and the second teacher model.

[0070] The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

[0071] Here, the parameter similarity between the same network layer in the first teacher model and the second teacher model can be calculated using the following formula:

[0072] Among them, λ k Represents the parameter similarity between the same network layer k in the first teacher model and the second teacher model.

[0073] As an optional implementation, step 102 of the method of the embodiment of the present invention, adjusting the total loss function of the corresponding network layer in the student model according to the parameter similarity to obtain the trained student model, may include:

[0074] According to the parameter similarity, weighting the first loss function in the total loss function of each network layer in the student model to obtain an adjusted total loss function of each network layer in the student model, wherein the first loss function is a loss function that depends on the teacher model;

[0075] It should be noted that the student model is trained using pure accent data. The first loss function is a loss function that depends on the teacher model. The composition structure of the teacher model is the same as that of the first teacher model and is also trained using pure accent data.

[0076] Here, each network layer of the student model corresponds to a total loss function. In addition to the first loss function, the total loss function also includes a second loss function, which is a loss function that does not rely on the teacher model.

[0077] It should be noted that the parameter similarity between every two identical network layers in the first teacher model and the second teacher model corresponds to each network layer in the student model, that is, the first loss function in different network layers corresponds to different parameter similarities.

[0078] The student model is trained using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

[0079] Based on this, as an optional implementation, before weighting the first loss function in the total loss function of each network layer in the student model according to the parameter similarity to obtain the adjusted total loss function of each network layer in the student model, the method of the embodiment of the present invention further includes:

[0080] Obtaining a first loss function for each network layer in the student model according to an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data;

[0081] Here, see Figure 3 ,q i,k represents the output of network layer k in the student model, p i,k Represents the output of network layer k in the teacher model, and the loss function of network layer k is expressed as L k .

[0082] Specifically, the first loss function is the cross entropy loss corresponding to the first target output result of the student model and the soft target. Here, the soft target specifically refers to the output result p of each network layer of the teacher model that inputs the pure accent data. i,k After dividing by the temperature parameter T, classification processing (such as softmax transformation) is performed to obtain p i,k Softened probability distribution. Here, soft targets can be understood as soft labels.

[0083] Here, the first target output of the student model is to i,k Divide by the temperature parameter T, and then perform classification processing (such as softmax transformation) to obtain it.

[0084] Among them, the first loss function can be expressed by the following formula:

[0085]

[0086] Obtaining a second loss function for each network layer in the student model based on the output results of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer;

[0087] Here, the true annotation of the pure accent data of the corresponding network layer of the student model is also called the true label of the corresponding network layer of the student model, also called the hard target.

[0088] Specifically, the second loss function is the cross entropy loss between the second target output result of the student model and the true label. Specifically, the second target output result of the student model is the softmax output result of the student model.

[0089] Among them, the second loss function can be expressed by the following formula:

[0090]

[0091] Among them, y true,i is the one-hot encoding of the true label.

[0092] It should be noted that the second loss function indicates that the output of the student model should be as close to the true label as possible.

[0093] The first loss function of each network layer in the student model and the second loss function of the network layer are summed up respectively to obtain the total loss function of each network layer in the student model.

[0094] Here, according to the parameter similarity, the first loss function in the total loss function of each network layer in the student model is weighted, and the adjusted total loss function of each network layer in the student model can be expressed as follows:

[0095]

[0096] It should be noted that the total loss function is the cross entropy loss L corresponding to the hard targets of each network layer of the student model hard,k and the cross entropy loss L corresponding to the soft target soft,k The sum of the soft target cross entropy and the parameter similarity is greater, indicating that the student model is more dependent on the contribution of the teacher model.

[0097] In the embodiment of the present invention, according to the parameter similarity λ k The total loss function of each network layer in the student model is dynamically adjusted to dynamically guide the training of the student model and improve the performance of the student model in the accented speech recognition task.

[0098] The speech recognition processing method of an embodiment of the present invention obtains the parameter similarity between the same network layers in the first teacher model and the second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; in the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent. In this way, the difference in acoustic characteristics between Mandarin data and accent data is used to calculate the parameter similarity in each intermediate network layer of the teacher model, and the parameter similarity is used to guide the adjustment of the student model loss function, thereby achieving the purpose of dynamically guiding the training of the student model, thereby improving the accuracy of speech recognition with an accent.

[0099] like Figure 4 As shown, an embodiment of the present invention further provides a speech recognition processing device, the device comprising:

[0100] A first acquisition module 401 is configured to obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data;

[0101] The first processing module 402 is used to adjust the total loss function of the corresponding network layer in the student model according to the parameter similarity during the process of training the student model using pure accent data to obtain a student model after training, and the student model is used to recognize speech information with an accent.

[0102] Optionally, the first obtaining module 401 may include:

[0103] A first training unit is used to train a first teacher model using pure Mandarin data to obtain parameters of each network layer in the first teacher model;

[0104] A second training unit is configured to train a second teacher model using a mixture of Mandarin data and accent data to obtain parameters of each network layer in the second teacher model;

[0105] The first acquisition unit is used to calculate the parameter similarity between the same network layers in the first teacher model and the second teacher model based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model.

[0106] Optionally, the first acquiring unit is specifically configured to:

[0107] Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model;

[0108] The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

[0109] Optionally, the processing module 402 includes:

[0110] A first processing unit is configured to perform weighted processing on first loss functions in the total loss functions of each network layer in the student model according to the parameter similarity, to obtain an adjusted total loss function of each network layer in the student model, wherein the first loss function is a loss function that depends on a teacher model;

[0111] The second processing unit is used to train the student model by using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

[0112] Optionally, the apparatus according to the embodiment of the present invention further includes:

[0113] A second processing module is configured to obtain a first loss function for each network layer in the student model based on an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data;

[0114] A third processing module is configured to obtain a second loss function for each network layer in the student model based on the output result of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer;

[0115] The fourth processing module is used to sum the first loss function of each network layer in the student model and the second loss function of the network layer to obtain the total loss function of each network layer in the student model.

[0116] The speech recognition processing device of an embodiment of the present invention obtains the parameter similarity between the same network layers in a first teacher model and a second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; in the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent. In this way, the difference in acoustic characteristics between Mandarin data and accent data is used to calculate the parameter similarity in each intermediate network layer of the teacher model, and the parameter similarity is used to guide the adjustment of the student model loss function, thereby achieving the purpose of dynamically guiding the training of the student model, and further improving the accuracy of speech recognition with an accent.

[0117] In order to better achieve the above goals, Figure 5 As shown, an embodiment of the present invention further provides a speech recognition processing device, including a processor 500 and a transceiver 510, wherein the processor 500 is configured to perform the following process:

[0118] Obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, where the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data;

[0119] In the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent.

[0120] Optionally, the processor 500 is further configured to:

[0121] Using pure Mandarin data to train the first teacher model to obtain parameters of each network layer in the first teacher model;

[0122] Using the mixed data of Mandarin data and accent data to train the second teacher model to obtain parameters of each network layer in the second teacher model;

[0123] Based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model, the parameter similarity between the same network layers in the first teacher model and the second teacher model is calculated.

[0124] Optionally, the processor 500 is further configured to:

[0125] Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model;

[0126] The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

[0127] Optionally, the processor 500 is further configured to:

[0128] According to the parameter similarity, weighting the first loss function in the total loss function of each network layer in the student model to obtain an adjusted total loss function of each network layer in the student model;

[0129] The student model is trained using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

[0130] Optionally, the processor 500 is further configured to:

[0131] Obtaining a first loss function for each network layer in the student model according to an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data;

[0132] Obtaining a second loss function for each network layer in the student model based on the output results of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer;

[0133] The first loss function of each network layer in the student model and the second loss function of the network layer are summed up respectively to obtain the total loss function of each network layer in the student model.

[0134] The speech recognition processing device of an embodiment of the present invention obtains the parameter similarity between the same network layers in a first teacher model and a second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; in the process of training the student model using pure accent data, the total loss function of the corresponding network layer in the student model is adjusted according to the parameter similarity to obtain a student model after training, and the student model is used to recognize speech information with an accent. In this way, the difference in acoustic characteristics between Mandarin data and accent data is used to calculate the parameter similarity in each intermediate network layer of the teacher model, and the parameter similarity is used to guide the adjustment of the student model loss function, thereby achieving the purpose of dynamically guiding the training of the student model, and further improving the accuracy of speech recognition with an accent.

[0135] An embodiment of the present invention also provides a speech recognition processing device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the various processes in the speech recognition processing method embodiment described above are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0136] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the various processes in the above-described speech recognition processing method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 means for performing functions specified in one or more processes and / or one or more blocks.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable storage medium produce a paper product including an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device to cause the computer or other programmable device to execute a series of operating steps to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0141] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A speech recognition processing method, characterized in that: include: Obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, where the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; In the process of training the student model using the pure accent data, adjusting the total loss function of the corresponding network layer in the student model according to the parameter similarity to obtain a trained student model, wherein the student model is used to recognize speech information with an accent; According to the parameter similarity, the total loss function of the corresponding network layer in the student model is adjusted to obtain a student model after training, including: According to the parameter similarity, weighting the first loss function in the total loss function of each network layer in the student model to obtain an adjusted total loss function of each network layer in the student model, wherein the first loss function is a loss function that depends on the teacher model; The student model is trained using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

2. The method according to claim 1, characterized in that Obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model, including: Using pure Mandarin data to train the first teacher model to obtain parameters of each network layer in the first teacher model; Using the mixed data of Mandarin data and accent data to train the second teacher model to obtain parameters of each network layer in the second teacher model; Based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model, the parameter similarity between the same network layers in the first teacher model and the second teacher model is calculated.

3. The method according to claim 2, characterized in that Calculating the parameter similarity between the same network layers in the first teacher model and the second teacher model based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model includes: Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model; The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

4. The method according to claim 1, wherein Before weighting the first loss function in the total loss function of each network layer in the student model according to the parameter similarity to obtain the adjusted total loss function of each network layer in the student model, the method further includes: Obtaining a first loss function for each network layer in the student model according to an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data; Obtaining a second loss function for each network layer in the student model based on the output results of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer; The first loss function of each network layer in the student model and the second loss function of the network layer are summed up respectively to obtain the total loss function of each network layer in the student model.

5. A speech recognition processing device, characterized in that: include: A first acquisition module is used to obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, wherein the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; A first processing module is configured to adjust the total loss function of the corresponding network layer in the student model according to the parameter similarity during the training of the student model using the pure accent data, thereby obtaining a trained student model, wherein the student model is used to recognize accented speech information; The first processing module includes: A first processing unit is configured to perform weighted processing on first loss functions in the total loss functions of each network layer in the student model according to the parameter similarity, to obtain an adjusted total loss function of each network layer in the student model, wherein the first loss function is a loss function that depends on a teacher model; The second processing unit is used to train the student model by using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

6. A speech recognition processing device, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, characterized in that: The processor is configured to perform the following operations: Obtain parameter similarity between the same network layer in the first teacher model and the second teacher model, where the training data of the first teacher model is pure Mandarin data, and the training data of the second teacher model is a mixture of Mandarin data and accent data; In the process of training the student model using the pure accent data, adjusting the total loss function of the corresponding network layer in the student model according to the parameter similarity to obtain a trained student model, wherein the student model is used to recognize speech information with an accent; The processor is further configured to: According to the parameter similarity, weighting the first loss function in the total loss function of each network layer in the student model to obtain an adjusted total loss function of each network layer in the student model, wherein the first loss function is a loss function that depends on the teacher model; The student model is trained using the adjusted total loss function of each network layer in the student model to obtain a trained student model.

7. The speech recognition processing device according to claim 6, characterized in that The processor is further configured to: Using pure Mandarin data to train the first teacher model to obtain parameters of each network layer in the first teacher model; Training the second teacher model using a mixture of Mandarin data and accent data to obtain parameters of each network layer in the second teacher model; Based on the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model, the parameter similarity between the same network layers in the first teacher model and the second teacher model is calculated.

8. The speech recognition processing device according to claim 7, characterized in that The processor is further configured to: Calculate the distance between the parameters corresponding to the same network layer in the first teacher model and the second teacher model according to the parameters of each network layer in the first teacher model and the parameters of each network layer in the second teacher model; The reciprocal of the distance value is calculated to obtain the parameter similarity between the same network layer in the first teacher model and the second teacher model.

9. The speech recognition processing device according to claim 6, characterized in that The processor is further configured to: Obtaining a first loss function for each network layer in the student model according to an output result of each network layer of the student model and an output result of each network layer of the teacher model input with the pure accent data; Obtaining a second loss function for each network layer in the student model based on the output results of each network layer of the student model and the true annotation of the pure accent data of the corresponding network layer; The first loss function of each network layer in the student model and the second loss function of the network layer are summed up respectively to obtain the total loss function of each network layer in the student model.

10. A speech recognition processing device comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the speech recognition processing method according to any one of claims 1 to 4 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the speech recognition processing method according to any one of claims 1 to 4 are implemented.

Images