A speech recognition method, system, storage medium and electronic device

Abstract

The present application relates to a kind of speech recognition method, system, storage medium and electronic equipment, comprising: based on multiple speech samples, the CTC coding network added with multiple intermediate loss modules is trained, obtain speech recognition model, and delete multiple intermediate loss modules in the speech recognition model, obtain target speech recognition model;Intermediate loss module is used to: receive and process the first speech feature output by the previous encoding layer, obtain second speech feature, and the second speech feature is spliced with the first speech feature, and the spliced speech feature is input to the next encoding layer;The speech data to be identified is input into target speech recognition model, and recognition result is obtained.The present application improves CTC coding network in training stage, fuses more speech features, accelerates the convergence of speech recognition model;And by deleting the improved part in inference stage, without affecting the model inference speed, also improve speech recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and particularly relates to a speech recognition method and system, a storage medium and an electronic device. BACKGROUND

[0002] In current speech recognition models, there are generally problems of low recognition accuracy or high model complexity. If a complex encoding network is constructed only to improve the recognition accuracy of the model, the calculation difficulty will undoubtedly be greatly increased, thereby increasing the model inference speed.

[0003] How to improve the model accuracy while not reducing the model recognition efficiency is a technical problem that needs to be solved at present. SUMMARY

[0004] To solve the above technical problems, the present application provides a speech recognition method and system, a storage medium and an electronic device.

[0005] The technical scheme of a speech recognition method of the present application is as follows:

[0006] Based on a plurality of speech training samples, the original CTC encoding network to which a plurality of intermediate loss modules are added is trained to obtain a first speech recognition model, and the plurality of intermediate loss modules in the first speech recognition model are deleted to obtain a target speech recognition model; wherein any intermediate loss module is used to: receive and sequentially perform normalization processing, linear processing and mapping processing on the first speech feature output by the previous original encoding layer in the original CTC encoding network to obtain a second speech feature, and splice the second speech feature with the first speech feature to obtain and input the spliced speech feature to the next original encoding layer of the original CTC encoding network.

[0007] The speech recognition result is obtained by inputting the speech data to be recognized into the target speech recognition model.

[0008] The speech recognition method of the present application has the following advantages:

[0009] The method of the present application improves the CTC encoding network in the training stage, integrates more speech features, and accelerates the convergence of the speech recognition model; and by deleting the improved part in the inference stage, the model inference speed is not affected, and the speech recognition accuracy is also improved.

[0010] On the basis of the above scheme, the speech recognition method of the present application can also be improved as follows.

[0011] Further, the any intermediate loss module comprises: a first normalization layer, a first linear layer, a first softmax layer and a first convolutional layer arranged in sequence; parameters of the first convolutional layer are same as parameters of an original convolutional layer of the original CTC encoding network, and parameters of the first normalization layer are same as parameters of an original normalization layer of the original CTC encoding network; the any intermediate loss module is specifically used for:

[0012] receiving the first speech feature output by a previous original encoding layer in the original CTC encoding network, inputting the first speech feature into the first normalization layer for normalization processing to obtain a normalized speech feature;

[0013] inputting the normalized speech feature into the first linear layer for linear processing, and inputting the speech feature after linear processing into the first softmax layer for processing to obtain the second speech feature;

[0014] outputting the second speech feature through the first convolutional layer, so that the second speech feature is spliced with the first speech feature to obtain a spliced speech feature, and inputting the spliced speech feature into a next original encoding layer of the original CTC encoding network.

[0015] Further, the step of training the original CTC encoding network added with the plurality of intermediate loss modules based on the plurality of speech training samples to obtain the first speech recognition model comprises:

[0016] inputting each speech training sample into the original CTC encoding network added with the plurality of intermediate loss modules for training to obtain a main loss value corresponding to each speech training sample and an intermediate loss value corresponding to each intermediate loss module, and obtaining a target loss value of each training speech sample according to the main loss value corresponding to each speech training sample and all intermediate loss values;

[0017] optimizing parameters of the CTC encoding network added with the intermediate loss module according to all target loss values to obtain an optimized CTC encoding network as the original CTC encoding network, and returning to execute the step of training the original CTC encoding network added with the plurality of intermediate loss modules based on the plurality of speech training samples until the optimized CTC encoding network meets a preset condition to obtain the first speech recognition model.

[0018] Further, the step of obtaining the target loss value of each training speech sample according to the main loss value corresponding to each speech training sample and all intermediate loss values comprises:

[0019] Based on the first preset formula and all intermediate loss values of any speech training sample, an intermediate loss average value of the any speech training sample is obtained, and an intermediate loss average value of each speech training sample is obtained; wherein the first preset formula is:

[0020] N is the number of intermediate loss values of the any speech training sample, is the nth intermediate loss value of the any speech training sample, L IntermediateCTC is the intermediate loss average value of the any speech training sample;

[0021] Based on the second preset formula and the backbone loss value and the intermediate loss average value of the any speech training sample, a target loss value of the any speech training sample is obtained, and a target loss value of each speech training sample is obtained; wherein the second preset formula is: L = (1-λ)L finalCTC +λL IntermediateCTC , λ is a weight coefficient, L finalCTC is the backbone loss value of the any speech training sample.

[0022] Further, the step of inputting the first speech feature into the first normalization layer for normalization processing to obtain a normalized speech feature comprises:

[0023] Based on a third preset formula, the first speech feature is input into the first normalization layer for normalization processing to obtain a normalized speech feature; wherein the third preset formula is: is the first speech feature output by the previous original encoding layer of the original CTC encoding network, indicates that the first speech feature is normalized, is the speech feature after normalization processing;

[0024] The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the speech feature after linear processing into the first softmax layer for processing to obtain the second speech feature comprises:

[0025] Based on a fourth preset formula, the normalized speech feature is input into the first linear layer for linear processing, and the speech feature after linear processing is input into the first softmax layer for processing to obtain the second speech feature;

[0026] Wherein, the fourth preset formula is: wherein, D represents the original feature dimension of the normalized speech feature, V represents the target feature dimension of the linearly processed speech feature, and Z represents the linearly processed speech feature. represents processing the linearly processed speech feature by the first Softmax layer, Z L represents the second speech feature.

[0027] Further, the process of obtaining any intermediate loss value is as follows:

[0028] Based on a fifth preset formula, any intermediate loss value of the any speech training sample is obtained.

[0029] wherein, the fifth preset formula is: L CTC = -log P CTC (y|Z L ), wherein P CTC (y|Z L ) represents the conditional probability of output y given input Z L of any intermediate loss module, L CTC is the any intermediate loss value, and y is the true label of the any speech training sample.

[0030] Further, the step of outputting the second speech feature by the first convolutional layer to splice the second speech feature with the first speech feature to obtain the spliced speech feature and inputting the spliced speech feature to the next original encoding layer of the original CTC encoding network includes:

[0031] Based on a sixth preset formula and the first convolutional layer, the second speech feature is transposed to obtain a transposed second speech feature, and the transposed second speech feature is convoluted by the first convolutional layer to obtain a convoluted second speech feature; wherein, the sixth preset formula is: transpose(Z L ) is the transposed second speech feature, Conv V→D (transpose(Z L )) represents convoluting the transposed second speech feature, is the convoluted second speech feature.

[0032] Based on a seventh preset formula, the convoluted second speech feature and the first speech feature, the spliced speech feature is obtained and sent to the next original encoding layer of the original CTC encoding network; wherein, the seventh preset formula is: concatenate the first speech feature and the second speech feature to obtain a spliced speech feature.

[0033] The technical scheme of the speech recognition system of the present application is as follows:

[0034] The speech recognition system comprises a construction module and a recognition module.

[0035] The construction module is configured to train an original CTC encoding network added with a plurality of intermediate loss modules based on a plurality of speech training samples to obtain a first speech recognition model, and delete the plurality of intermediate loss modules in the first speech recognition model to obtain a target speech recognition model; wherein any intermediate loss module is configured to receive and sequentially perform normalization processing, linear processing and mapping processing on a first speech feature output by a previous original encoding layer in the original CTC encoding network to obtain a second speech feature, concatenate the first speech feature and the second speech feature to obtain a spliced speech feature, and input the spliced speech feature to a next original encoding layer of the original CTC encoding network.

[0036] The recognition module is configured to input to-be-recognized speech data into the target speech recognition model to obtain a speech recognition result.

[0037] The beneficial effects of the speech recognition system of the present application are as follows:

[0038] The system of the present application improves the CTC encoding network in the training stage, fuses more speech features, and accelerates the convergence of the speech recognition model; and deletes the improved part in the inference stage, which improves the speech recognition accuracy without affecting the inference speed of the model.

[0039] The technical scheme of the storage medium of the present application is as follows:

[0040] The storage medium stores instructions, and when a computer reads the instructions, the computer executes the steps of the speech recognition method of the present application.

[0041] The technical scheme of the electronic device of the present application is as follows:

[0042] The electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to make the computer execute the steps of the speech recognition method of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The flowchart of the speech recognition method of the embodiment of the present application is shown in the figure.

[0044] Figure 2 The structure diagram of the original CTC encoding network in the speech recognition method of the embodiment of the present application is shown in the figure.

[0045] Figure 3 A structural schematic diagram of an intermediate loss module in a speech recognition method according to an embodiment of the present application;

[0046] Figure 4 A structural schematic diagram of a speech recognition system according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] As shown in Figure 1 , a speech recognition method according to an embodiment of the present application comprises the following steps:

[0048] S1, based on a plurality of speech training samples, training an original CTC encoding network to which a plurality of intermediate loss modules are added, obtaining a first speech recognition model, and deleting the plurality of intermediate loss modules in the first speech recognition model to obtain a target speech recognition model.

[0049] Among them, any intermediate loss module is used to: receive and sequentially perform normalization processing, linear processing and mapping processing on the first speech feature output by the last original encoding layer in the original CTC encoding network, to obtain a second speech feature, and splice the second speech feature with the first speech feature to obtain and input the spliced speech feature to the next original encoding layer of the original CTC encoding network.

[0050] Among them, the speech training sample is a speech data collected in any way, and the number of speech training samples is not limited.

[0051] Among them, as shown in Figure 2 , the original CTC encoding network comprises: 12 layers of original encoding layers, original normalization layers, original linear layers and softmax layers arranged in sequence.

[0052] Each intermediate loss module is arranged between any two adjacent original encoding layers, for example, when an intermediate loss module is arranged between the first original encoding layer and the second original encoding layer, another intermediate loss module can only be arranged between any two adjacent original encoding layers other than the first original encoding layer and the second original encoding layer, which can be the second original encoding layer and the third original encoding layer, without limitation. In this embodiment, the plurality of intermediate loss modules include a first intermediate loss module, a second intermediate loss module, and a third intermediate loss module; the first intermediate loss module is arranged between the third original encoding layer and the fourth original encoding layer, the second intermediate loss module is arranged between the sixth original encoding layer and the seventh original encoding layer, and the third intermediate loss module is arranged between the ninth original encoding layer and the tenth original encoding layer. The last original encoding layer in the original CTC encoding network refers to the last original encoding layer of each intermediate loss module; for example, the last original encoding layer of the first intermediate loss module is the third original encoding layer, the last original encoding layer of the second intermediate loss module is the sixth original encoding layer, and the last original encoding layer of the third intermediate loss module is the ninth original encoding layer.

[0053] The first speech feature is a speech feature output by the last original encoding layer. It should be noted that each original encoding layer includes a feedforward module, a multi-head attention module, and a convolution module. Each original encoding layer is configured to sequentially process an input speech feature through the feedforward module, the multi-head attention module, and the convolution module, and output a processed speech feature. The specific process of processing the speech feature by the feedforward module, the multi-head attention module, and the convolution module is known in the art, and will not be described in detail.

[0054] The second speech feature is a speech feature processed by the intermediate loss module, and the processing manner includes normalization processing, linear processing, and mapping processing. The specific process of normalizing, linearly processing, and mapping the speech feature is known in the art, and will not be described in detail.

[0055] The concatenated speech feature is a feature obtained by concatenating a plurality of speech features. There are mainly two ways of feature concatenation: one is element-wise addition, referred to as add, and the other is to stack image features together, referred to as concat. Both of them are known in the art. In this embodiment, the first add method is used to add the elements of the two features correspondingly.

[0056] The first speech recognition model is a speech recognition model obtained by training the original CTC encoding network containing the plurality of intermediate loss modules. The target speech recognition model is a speech recognition model obtained by deleting the plurality of intermediate loss modules in the first speech recognition model and only retaining the trained backbone network.

[0057] It should be noted that in this embodiment, fbank features need to be extracted from the speech training samples, and the fbank features of each speech training sample are input into the model for training.

[0058] S2. Input the speech data to be recognized into the target speech recognition model to obtain the speech recognition result.

[0059] The speech data to be recognized is arbitrarily collected speech data. Similarly, it is necessary to extract the fbank features of the speech data to be recognized, and input the fbank features of the speech data to be recognized into the target speech recognition model to obtain the speech recognition result of the speech data to be recognized.

[0060] The speech recognition result is the text content corresponding to the speech data to be recognized.

[0061] Preferably, any intermediate loss module includes: a first normalization layer, a first linear layer, a first softmax layer, and a first convolutional layer arranged sequentially; the parameters of the first convolutional layer are the same as the parameters of the original convolutional layer of the original CTC coding network, and the parameters of the first normalization layer are the same as the parameters of the original normalization layer of the original CTC coding network.

[0062] Among them, such as Figure 3 As shown, each intermediate loss module includes: a first normalization layer, a first linear layer, a first softmax layer, and a first convolutional layer, arranged sequentially.

[0063] It should be noted that the parameters of the linear layer and normalization layer of the intermediate loss module are the same as those of the first linear layer and the first normalization layer in the original CTC coding network.

[0064] The intermediate loss module is specifically used for:

[0065] The first speech feature is received from the output of the previous original coding layer in the original CTC coding network, and the first speech feature is input into the first normalization layer for normalization processing to obtain the normalized speech feature.

[0066] The process of normalizing speech features through a normalization layer is an existing technology and will not be elaborated on here.

[0067] The normalized speech features are input into the first linear layer for linear processing, and the linearly processed speech features are input into the first softmax layer for further processing to obtain the second speech features.

[0068] The processes of linearly processing speech features through linear layers and processing speech features through softmax layers (softmax function) are existing technologies and will not be elaborated upon here.

[0069] The second speech feature is output through the first convolutional layer, so that the second speech feature is concatenated with the first speech feature, and the concatenated speech feature is input into the next original coding layer of the original CTC coding network.

[0070] The first convolutional layer is a new addition to the intermediate loss module, and its parameters can be continuously adjusted during training.

[0071] Preferably, the step of training the original CTC encoding network with multiple intermediate loss modules based on multiple speech training samples to obtain the first speech recognition model includes:

[0072] Each speech training sample is input into the original CTC encoding network with multiple intermediate loss modules for training, to obtain the backbone loss value and the intermediate loss value corresponding to each speech training sample and each intermediate loss module. Based on the backbone loss value and all intermediate loss values ​​corresponding to each speech training sample, the target loss value of each training speech sample is obtained.

[0073] The backbone loss value is: based on the corresponding loss function in the original CTC coding network ( Figure 2 The loss value is calculated from the original CTC coding network (CTC LOSS). The intermediate loss value is: based on the loss function of each intermediate loss module (CTC LOSS). Figure 3 CTC LOSS of the intermediate loss module in the middle.

[0074] The parameters of the CTC coding network with added intermediate loss modules are optimized based on all target loss values. The optimized CTC coding network is then used as the original CTC coding network. The process of training the original CTC coding network with added intermediate loss modules based on multiple speech training samples is then repeated until the optimized CTC coding network meets the preset conditions, thus obtaining the first speech recognition model.

[0075] The preset conditions are any of the following: ① The expected indicator (word accuracy, which is a common existing indicator) is achieved; ② The model loss converges to a specified threshold and no longer decreases; ③ The specified maximum number of training iterations is reached.

[0076] Specifically, all target loss values are added to the parameters of the CTC encoding network with an intermediate loss module to perform backward optimization, and an optimized CTC encoding network is obtained. It is judged whether the optimized CTC encoding network meets a preset condition. If yes, the optimized CTC encoding network is determined as the first speech recognition model; if no, the optimized CTC encoding network is taken as the original CTC encoding network, and step S1 is returned to be executed until the optimized CTC encoding network meets the preset condition, and the first speech recognition model is obtained.

[0077] Preferably, the step of obtaining the target loss value of each training speech sample according to the backbone loss value corresponding to each speech training sample and all intermediate loss values comprises:

[0078] Based on the first preset formula and all intermediate loss values of any speech training sample, an intermediate loss average value of the any speech training sample is obtained, and the intermediate loss average value of each speech training sample is obtained.

[0079] The first preset formula is:

[0080] N is the number of intermediate loss values of the any speech training sample, is the nth intermediate loss value of the any speech training sample, L IntermediateCTC is the intermediate loss average value of the any speech training sample;

[0081] Based on the second preset formula and the backbone loss value and the intermediate loss average value of the any speech training sample, a target loss value of the any speech training sample is obtained, and the target loss value of each speech training sample is obtained.

[0082] The second preset formula is: L = (1-λ)L finalCTC +λL IntermediateCTC , λ is a weight coefficient, L finalCTC is the backbone loss value of the any speech training sample.

[0083] Preferably, the step of inputting the first speech feature into the first normalization layer for normalization processing to obtain a normalized speech feature comprises:

[0084] Based on a third preset formula, the first speech feature is input into the first normalization layer for normalization processing to obtain a normalized speech feature.

[0085] The third preset formula is: is the first speech feature output by the previous original encoding layer of the original CTC encoding network, normalizing the first speech feature, normalizing the first speech feature.

[0086] The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the linearly processed speech feature into the first softmax layer for processing to obtain the second speech feature, comprises:

[0087] The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the linearly processed speech feature into the first softmax layer for processing to obtain the second speech feature, comprises:

[0088] The fourth preset formula is: The fourth preset formula is: The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the linearly processed speech feature into the first softmax layer for processing to obtain the second speech feature, comprises: L normalizing the first speech feature.

[0089] The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the linearly processed speech feature into the first softmax layer for processing to obtain the second speech feature, comprises:

[0090] The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the linearly processed speech feature into the first softmax layer for processing to obtain the second speech feature, comprises:

[0091] The fifth preset formula is: L CTC = -log P CTC (y|Z L ), wherein P CTC (y|Z L ) represents the conditional probability of output y given input Z L of any intermediate loss module, L CTC is the any intermediate loss value, and y is the true label of any speech training sample.

[0092] The step of inputting the normalized speech feature into the first linear layer for linear processing, and inputting the linearly processed speech feature into the first softmax layer for processing to obtain the second speech feature, comprises:

[0093] The second speech feature is transposed based on a sixth preset formula and the first convolution layer, to obtain a transposed second speech feature, and the transposed second speech feature is subjected to a convolution operation through the first convolution layer, to obtain a convoluted second speech feature.

[0094] The sixth preset formula is:

[0095]

[0096] transpose(Z L ) is the transposed second speech feature, Conv V→D (transpose(Z L )) represents that the transposed second speech feature is subjected to a convolution operation, is the convoluted second speech feature.

[0097] The spliced speech feature is obtained based on a seventh preset formula, the convoluted second speech feature and the first speech feature, and the spliced speech feature is sent to a next original encoding layer of the original CTC encoding network.

[0098] The seventh preset formula is: is the spliced speech feature.

[0099] The technical scheme of the embodiment improves the CTC encoding network in the training stage, fuses more speech features, accelerates the convergence of the speech recognition model, and deletes the improved part in the inference stage, which improves the speech recognition accuracy without affecting the inference speed of the model.

[0100] As shown in Figure 4 , a speech recognition system 200 of an embodiment of the application includes a construction module 210 and a recognition module 220.

[0101] The construction module 210 is configured to train an original CTC encoding network to which a plurality of intermediate loss modules are added based on a plurality of speech training samples, to obtain a first speech recognition model, and to delete the plurality of intermediate loss modules in the first speech recognition model, to obtain a target speech recognition model; wherein any intermediate loss module is configured to receive and sequentially perform normalization processing, linear processing and mapping processing on a first speech feature output by a previous original encoding layer in the original CTC encoding network, to obtain a second speech feature, to splice the second speech feature with the first speech feature, to obtain a spliced speech feature, and to input the spliced speech feature to a next original encoding layer of the original CTC encoding network.

[0102] The identification module 220 is configured to input the to-be-identified voice data into the target voice recognition model to obtain a voice recognition result.

[0103] The technical solution of the embodiment improves the CTC encoding network in the training stage, fuses more voice features, accelerates the convergence of the voice recognition model, deletes the improved part in the inference stage, and improves the voice recognition accuracy without affecting the inference speed of the model.

[0104] The steps of implementing the functions of the parameters and the modules in the voice recognition system 200 according to the embodiment can refer to the parameters and the steps in the embodiments of the voice recognition method, which will not be repeated here.

[0105] The storage medium provided by the embodiment of the present application includes instructions stored in the storage medium, which make the computer execute the steps of the voice recognition method when the computer reads the instructions. The specific parameters and steps can refer to the embodiments of the voice recognition method, which will not be repeated here.

[0106] The computer storage medium includes, for example, a USB flash disk, a mobile hard disk, etc.

[0107] The electronic device provided by the embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the computer executes the steps of the voice recognition method. The specific parameters and steps can refer to the embodiments of the voice recognition method, which will not be repeated here.

[0108] Those skilled in the art know that the present application can be implemented as a method, a system, a storage medium, and an electronic device.

[0109] Therefore, the present application can be embodied in the form of a hardware completely, a software completely (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which can be referred to as a "circuit", "module" or "system" hereinafter. Furthermore, in some embodiments, the present application can also be embodied in the form of a computer program product stored in one or more computer readable storage medium (media) of the present application that contains computer readable program codes. Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present document, the computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. Although the present application has been shown and described with respect to the embodiments thereof, it should be understood by the skilled in the art that the foregoing and various other changes, substitutions, and alterations can be made hereto without departing from the scope of the present application.

Claims

1. A speech recognition method, characterized in that, include: Based on multiple speech training samples, the original CTC coding network with multiple intermediate loss modules is trained to obtain a first speech recognition model. Then, the multiple intermediate loss modules in the first speech recognition model are removed to obtain a target speech recognition model. Each intermediate loss module is used to: receive and perform normalization, linearization, and mapping processing on the first speech feature output from the previous original coding layer in the original CTC coding network to obtain a second speech feature, and concatenate the second speech feature with the first speech feature to obtain and input the concatenated speech feature into the next original coding layer of the original CTC coding network. The speech data to be recognized is input into the target speech recognition model to obtain the speech recognition result; Each intermediate loss module includes: a first normalization layer, a first linear layer, a first softmax layer, and a first convolutional layer arranged sequentially; the parameters of the first convolutional layer are the same as the parameters of the original convolutional layer of the original CTC coding network, and the parameters of the first normalization layer are the same as the parameters of the original normalization layer of the original CTC coding network; each intermediate loss module is specifically used for: The first speech feature is received from the output of the previous original coding layer in the original CTC coding network, and the first speech feature is input into the first normalization layer for normalization processing to obtain the normalized speech feature. The normalized speech features are input into the first linear layer for linear processing, and the linearly processed speech features are input into the first softmax layer for processing to obtain the second speech features. The second speech feature is output through the first convolutional layer, so that the second speech feature is concatenated with the first speech feature, and the concatenated speech feature is input into the next original coding layer of the original CTC coding network.

2. The speech recognition method according to claim 1, characterized in that, The step of training the original CTC encoding network with multiple intermediate loss modules based on multiple speech training samples to obtain the first speech recognition model includes: Each speech training sample is input into the original CTC coding network with multiple intermediate loss modules for training, to obtain the backbone loss value and the intermediate loss value corresponding to each speech training sample and each intermediate loss module. Based on the backbone loss value and all intermediate loss values ​​corresponding to each speech training sample, the target loss value of each training speech sample is obtained. The parameters of the CTC coding network with added intermediate loss modules are optimized based on all target loss values. The optimized CTC coding network is then used as the original CTC coding network. The process of training the original CTC coding network with added intermediate loss modules based on multiple speech training samples is then repeated until the optimized CTC coding network meets the preset conditions, thus obtaining the first speech recognition model.

3. The speech recognition method according to claim 2, characterized in that, The step of obtaining the target loss value for each training speech sample based on the backbone loss value and all intermediate loss values ​​corresponding to each speech training sample includes: Based on the first preset formula and all intermediate loss values ​​of any speech training sample, the average intermediate loss of any speech training sample is obtained, until the average intermediate loss of each speech training sample is obtained; wherein, the first preset formula is: N is the number of intermediate loss values ​​for any given speech training sample. This is the nth intermediate loss value for any given speech training sample. The median average loss of any given speech training sample; Based on the second preset formula and the average of the backbone loss and intermediate loss of any given speech training sample, the target loss value of any given speech training sample is obtained, until the target loss value of each speech training sample is obtained; wherein, the second preset formula is: , These are the weighting coefficients. is the backbone loss value for any of the given speech training samples.

4. The speech recognition method according to claim 3, characterized in that, The step of inputting the first speech feature into the first normalization layer for normalization processing to obtain the normalized speech feature includes: Based on the third preset formula, the first speech feature is input into the first normalization layer for normalization processing to obtain the normalized speech feature; wherein, the third preset formula is: , The first speech feature is the output of the previous original coding layer of the original CTC coding network. This indicates that the first speech feature has been normalized. The normalized speech features are referred to as such. The step of inputting the normalized speech features into the first linear layer for linear processing, and then inputting the linearly processed speech features into the first softmax layer for further processing to obtain the second speech features includes: Based on the fourth preset formula, the normalized speech features are input into the first linear layer for linear processing, and the linearly processed speech features are input into the first softmax layer for processing to obtain the second speech features. The fourth preset formula is: , Let D represent the speech features after linear processing, where D is the original feature dimension of the speech features after normalization, and V is the target feature dimension of the speech features after linear processing. This indicates that the linearly processed speech features are processed through the first Softmax layer. This is the second speech feature.

5. The speech recognition method according to claim 4, characterized in that, The process of obtaining any intermediate loss value is as follows: Based on the fifth preset formula, any intermediate loss value of any speech training sample is obtained; The fifth preset formula is as follows: ,in, The loss function of any intermediate loss module given input is represented as... Output the conditional probability of y. Let y be any intermediate loss value, and y be the true label of any speech training sample.

6. The speech recognition method according to claim 5, characterized in that, The step of outputting the second speech feature through the first convolutional layer, concatenating the second speech feature with the first speech feature, and inputting the concatenated speech feature into the next original coding layer of the original CTC coding network includes: Based on the sixth preset formula and the first convolutional layer, the second speech feature is transposed to obtain the transposed second speech feature, and then the transposed second speech feature is convolved by the first convolutional layer to obtain the convolutional second speech feature; wherein, the sixth preset formula is: , The transposed second speech feature. This indicates that a convolution operation is performed on the transposed second speech feature. The second speech feature after convolution; Based on the seventh preset formula, the second speech feature after convolution, and the first speech feature, the concatenated speech feature is obtained, and the concatenated speech feature is sent to the next original coding layer of the original CTC coding network; wherein, the seventh preset formula is: , The spliced ​​speech features are described above.

7. A speech recognition system, characterized in that, include: Modules for building and identifying; The construction module is used to: train an original CTC coding network with multiple intermediate loss modules added based on multiple speech training samples to obtain a first speech recognition model, and delete multiple intermediate loss modules in the first speech recognition model to obtain a target speech recognition model; wherein, any intermediate loss module is used to: receive and perform normalization, linear processing and mapping processing on the first speech feature output by the previous original coding layer in the original CTC coding network in sequence to obtain a second speech feature, and concatenate the second speech feature with the first speech feature to obtain and input the concatenated speech feature into the next original coding layer of the original CTC coding network; The recognition module is used to: input the speech data to be recognized into the target speech recognition model to obtain the speech recognition result; Each intermediate loss module includes: a first normalization layer, a first linear layer, a first softmax layer, and a first convolutional layer arranged sequentially; the parameters of the first convolutional layer are the same as the parameters of the original convolutional layer of the original CTC coding network, and the parameters of the first normalization layer are the same as the parameters of the original normalization layer of the original CTC coding network; each intermediate loss module is specifically used for: The first speech feature is received from the output of the previous original coding layer in the original CTC coding network, and the first speech feature is input into the first normalization layer for normalization processing to obtain the normalized speech feature. The normalized speech features are input into the first linear layer for linear processing, and the linearly processed speech features are input into the first softmax layer for processing to obtain the second speech features. The second speech feature is output through the first convolutional layer, so that the second speech feature is concatenated with the first speech feature, and the concatenated speech feature is input into the next original coding layer of the original CTC coding network.

8. A storage medium, characterized in that, The storage medium stores instructions that, when read by a computer, cause the computer to execute a speech recognition method as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the computer to perform a speech recognition method as described in any one of claims 1 to 6.

Images