Speech recognition method, system, device, storage medium

Abstract

The application discloses a speech recognition method, system, device and storage medium, which is used for obtaining original speech, and conveying the original speech to an Adam-BP neural network which has been trained to output a speech recognition result; wherein the training process of the Adam-BP neural network which has been constructed is as follows: S1, initializing and setting the Adam-BP neural network model; S2, collecting a training sample set and inputting the training sample set into the Adam-BP neural network; S3, defining an objective function of the Adam-BP neural network; S4, updating a parameter vector by using the objective function and applying the parameter vector to the Adam-BP neural network, and then entering the step S2, and iteratively updating the parameter vector until a convergence condition is met; the speech recognition method can realize intelligent speech recognition of a meeting, including dialects and slang which are difficult for interpreters to distinguish, and can assist the interpreters in simultaneous interpretation, thereby improving the efficiency and quality of the simultaneous interpretation of the interpreters.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and more particularly to speech recognition methods, systems, devices, and storage media. Background Technology

[0002] Simultaneous interpreting, as a real-time translation method, requires interpreters to accurately translate the speaker's content to the audience without interrupting the speaker's speech or disrupting their train of thought, ensuring the audience's overall comprehension. Simultaneous interpreting boasts advantages such as real-time translation, high efficiency, and strong accuracy, and is widely used in large-scale lectures, seminars, and international conferences. The interpreter's booth, the primary workspace for simultaneous interpreting, typically consists of a cabinet, ventilation system, and related interpreting equipment. During simultaneous interpreting, the interpreter observes the meeting room through plexiglass while simultaneously translating the content heard through headphones into the target language. The translated content is then output through the interpreter's receiver, and attendees receive the translated language information in real-time through headphones, ensuring the smooth running of the meeting.

[0003] As the core of simultaneous interpreting, the interpreter's work generally consists of three stages. The first stage is language recognition, requiring the interpreter to accurately listen to and identify the language used in the meeting room to ensure the smooth progress of subsequent interpreting work. The second stage is comprehension and translation, requiring the interpreter to quickly memorize, understand, and translate the source language using their professional knowledge within a very short time. The third stage is organization and expression, requiring the interpreter to organize, revise, and express the translated target language. Simultaneous interpreting places extremely high demands on the interpreter's translation and expression abilities. Only interpreters with solid professional knowledge, flexible thinking, and fluent organizational and expression skills can be competent for the position. However, even interpreters with extensive professional knowledge and rich experience still face significant challenges in simultaneous interpreting tasks such as meetings, conversations, and business negotiations.

[0004] Unlike large international conferences and seminars, in some business negotiations and meetings, interpreters cannot obtain the speech in advance to understand the content and plan their interpretation. Moreover, the speakers' pronunciation and speaking speed are not as clear and deliberate as the moderators of large conferences. There may even be speakers who mix regional accents or dialects, which will pose a huge challenge to interpreters' simultaneous interpretation work. Summary of the Invention

[0005] Based on the technical problems existing in the background technology, the present invention proposes a speech recognition method, system, device and storage medium. This speech recognition method has accurate recognition results for slang and dialects of various languages, thus reducing the difficulty of the interpreter's work in the original language listening and recognition stage and improving the efficiency and quality of the interpreter's simultaneous interpretation.

[0006] The speech recognition method proposed in this invention includes the following steps:

[0007] The original speech is acquired and fed into a pre-trained Adam-BP neural network to output the speech recognition result.

[0008] The training process for the pre-constructed Adam-BP neural network is as follows:

[0009] S1: Initialize the neural network structure parameters and hyperparameters in the Adam-BP neural network model;

[0010] S2: Collect a training sample set, and input the k-th sample in the training sample set into the Adam-BP neural network, and calculate the output of the Adam-BP neural network. in b represents the output layer weights. k The threshold value is denoted by h, where m is the number of nodes in the output layer of the neural network, and h is the threshold value. j This is the output of hidden layer node j, where j = 1, 2, ..., l, and l is the number of hidden layer nodes;

[0011] S3: Define the objective function of the Adam-BP neural network Where y i Let o be the true value of the i-th sample. i Let θ be the predicted value of the i-th sample in the Adam-BP neural network, and θ be the parameter vector.

[0012] S4: Update the parameter vector θ using backpropagation of the objective function E(θ), and apply the parameter vector θ to the Adam-BP neural network. If the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not reached the maximum value, then loop back to step S2 to iteratively update the parameter vector θ until the convergence condition is met.

[0013] Furthermore, for h in S2 j The calculation formula is as follows:

[0014]

[0015] Where f(·) is the hidden layer activation function, x i For the i-th input node variable, ijHidden layer weights, a j is the threshold of the hidden layer, and n is the number of input layer nodes in the neural network.

[0016] Furthermore, step S4 specifically includes:

[0017] Calculate the correction of the first and second moment estimates of the gradient of the Adam-BP neural network, and the first and second moment estimates of the updated gradient;

[0018] The parameter vector θ is updated based on the first-order moment estimate and second-order moment estimate of the corrected gradient, and the parameter vector θ is applied to the Adam-BP neural network;

[0019] If the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not yet reached the maximum value, then the loop continues into step S2 to iteratively update the parameter vector θ until the convergence condition is met.

[0020] Furthermore, the update iteration expression for the parameter vector θ is as follows:

[0021]

[0022] in: The gradient is the distance the i-th parameter of the neural network descends along the gradient direction during the k-th iteration. and These refer to the first-order moment estimate and the second-order moment estimate of the current gradient for the i-th parameter at the k-th iteration. and They are and The bias correction is given by η, where η is the learning rate and ε is a constant.

[0023] Furthermore, the bias correction for the first and second moment estimates of the gradient in the Adam-BP neural network, as well as the updated first and second moment estimates, are specifically formulated as follows:

[0024]

[0025] Where β1 and β2 are the exponential decay rates of the first and second moment estimates of the gradient, respectively; in addition, and These refer to the first-order moment estimate and the second-order moment estimate of the current gradient for the i-th parameter at the k-th iteration. and These refer to the first-order moment estimate and the second-order moment estimate of the current gradient for the i-th parameter at the (k+1)-th iteration. and They are and Deviation correction.

[0026] Furthermore, the Adam-BP neural network includes an input layer, a hidden layer, and an output layer, and the training sample set includes a collection of speech under different environmental conditions.

[0027] A speech recognition system is used to acquire raw speech and feed the raw speech into a pre-trained Adam-BP neural network to output speech recognition results. The speech recognition system includes a parameter initialization module, a sample set collection and input module, an objective function definition module, and a backpropagation judgment module.

[0028] The parameter initialization module is used to initialize the neural network structure parameters and hyperparameters in the Adam-BP neural network model;

[0029] The sample set collection input module is used to collect the training sample set and input the k-th sample in the training sample set into the Adam-BP neural network to calculate the output of the Adam-BP neural network. in b represents the output layer weights. k The threshold value is denoted by h, where m is the number of nodes in the output layer of the neural network, and h is the threshold value. j This is the output of hidden layer node j, where j = 1, 2, ..., l, and l is the number of hidden layer nodes;

[0030] The objective function definition module is used to define the objective function of the Adam-BP neural network. Where y i Let o be the true value of the i-th sample. i Let θ be the predicted value of the i-th sample in the Adam-BP neural network, and θ be the parameter vector.

[0031] The backpropagation judgment module is used to update the parameter vector θ by backpropagating the objective function E(θ), and apply the parameter vector θ to the Adam-BP neural network. If the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not reached the maximum value, then the loop enters step S2 to iteratively update the parameter vector θ until the convergence condition is met.

[0032] A speech recognition device includes a housing containing a simultaneous interpretation receiver and a speech recognition unit. The receiver acquires raw speech and transmits it to the speech recognition unit, which processes the speech to output a speech recognition result. This result is used to assist interpreters in their simultaneous interpretation work, improving efficiency and accuracy. The speech recognition unit is configured with a program to implement the aforementioned speech recognition method.

[0033] A computer storage medium storing a plurality of classification programs, the plurality of classification programs being invoked by a processor and executed as described above in the speech recognition method.

[0034] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0035] The advantages of the speech recognition method, system, device, and storage medium provided by this invention are as follows: This invention provides a speech recognition method, system, device, and storage medium based on an Adam-BP neural network. The speech recognition device configured with an Adam-BP neural network can accurately recognize slang and dialects of various languages, thus reducing the difficulty of the interpreter's work in the source language listening and recognition stage, allowing the interpreter more time to focus on the translation and understanding of the source language and the organization and expression of the target language. This is of great significance for improving the efficiency and quality of simultaneous interpretation. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the speech recognition method of the present invention;

[0037] Figure 2 This is a schematic diagram of the Adam-BP neural network structure;

[0038] Figure 3 A schematic diagram of the neurons in an Adam-BP neural network;

[0039] Figure 4 This is a structural diagram of a speech recognition device;

[0040] Figure 5 This is a diagram of the internal structure of the box.

[0041] Among them, 1-box body, 2-box body support, 3-workbench, 4-transparent glass, 5-noise reduction and sound insulation board, 6-ventilation mechanism, 7-simultaneous interpretation receiver, 8-simultaneous interpretation machine, 9-voice recognition device. Detailed Implementation

[0042] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] like Figures 1 to 5 As shown, the speech recognition method proposed in this invention is used to input the acquired raw speech into a pre-trained Adam-BP neural network to output the speech recognition result.

[0044] Backpropagation (BP) neural networks, as a typical multilayer feedforward network, possess strong nonlinear mapping capabilities. They can obtain nonlinear mapping relationships between data by training the input and output data of the BP neural network. BP neural networks also exhibit good adaptability, generalization ability, and fault tolerance. During data training, BP neural networks can adaptively approach the global optimum through continuous optimization and learning, and even if local neurons are damaged, it will not affect the training results. However, the optimization algorithm of BP neural networks uses gradient descent, which typically maintains a single learning rate to update all weights of the network. This leads to problems such as slow convergence speed and susceptibility to local optima in applications, limiting its application in speech recognition. To address the shortcomings of gradient descent, this embodiment uses the Adam algorithm as the optimization algorithm for BP neural networks. The Adam algorithm, or adaptive moment estimation algorithm, can design independent adaptive learning rates for different parameters by calculating the first and second moment estimates of the gradient. After bias correction, the learning rate has a defined range for each iteration, making parameter changes relatively stable. Because the Adam algorithm combines the advantages of the momentum method and the RMSprop algorithm, it has the ability to converge quickly and is very robust to the selection of hyperparameters.

[0045] This embodiment provides a speech recognition device, which includes a housing containing a simultaneous interpretation receiver and a speech recognition device. The interpreter acquires the original speech through the simultaneous interpretation receiver, and the speech recognition device processes the original speech to output a speech recognition result. The speech recognition device is configured with a program to implement speech recognition. To achieve better translation recognition performance, the configured Adam-BP neural network needs to be trained. First, hyperparameters, such as the number of hidden layers and the initial learning rate, need to be initialized using empirical formulas or extensive experiments. Then, a large amount of relevant language data from a database is used as training samples and imported into the Adam-BP neural network for training. A system objective function is set to reduce the error between the network output value and the expected value. The error is backpropagated, and the weights and thresholds are iteratively updated. This process is repeated until the objective function meets the accuracy requirements, resulting in the optimal trained Adam-BP neural network model. The Adam-BP neural network model includes an input layer, hidden layers, and an output layer.

[0046] like Figure 2 and3 As shown, the training steps for the constructed Adam-BP neural network are as follows:

[0047] S1: Initialize the neural network structure parameters and hyperparameters in the Adam-BP neural network model; see Adam-BP neural network structure and neuron structure for details. Figure 2 and 3 ;

[0048] S2: Collect the speech database, sample and label the language data used for training in the speech database, select N training samples from it to form a training sample set, and input the training sample set into the pre-constructed Adam-BP neural network.

[0049] Calculate the output h of hidden layer node i in the Adam-BP neural network j :

[0050]

[0051] Where f(·) is the hidden layer activation function, x i For the i-th input node variable, ij Hidden layer weights, a j is the hidden layer threshold, and n is the number of input layer nodes in the neural network;

[0052] Calculate the output o of the Adam-BP neural network k (θ):

[0053]

[0054] in, b represents the output layer weights. k is the output layer threshold, m is the number of output layer nodes of the neural network, = 1, 2, ..., l, l is the number of hidden layer nodes;

[0055] S3: Define the objective function E(θ) of the Adam-BP neural network:

[0056]

[0057] Where y i Let o be the true value of the i-th sample. i Let θ be the predicted value of the i-th sample in the Adam-BP neural network, and let θ be the parameter vector. The expression for θ is as follows:

[0058]

[0059] S4: Optimize the neural network using the Adam optimization algorithm. The optimization problem of the Adam-BP neural network is to find the minimum value of the objective function E(θ). The objective function E(θ) is backpropagated to update the parameter vector θ. The iterative expression for the parameter vector θ is:

[0060]

[0061] in: The gradient is the distance the i-th parameter of the neural network descends along the gradient direction during the k-th iteration. and These refer to the first-order moment estimate and the second-order moment estimate of the current gradient for the i-th parameter at the k-th iteration. and They are and The bias correction is given by η, where η is the learning rate and ε is a constant.

[0062] Initialize the hyperparameters of the Adam optimization algorithm, calculate the correction of the first and second moment estimates of the gradient of the Adam-BP neural network, and the first and second moment estimates of the updated gradient;

[0063]

[0064] Where β1 and β2 are the exponential decay rates of the first and second moment estimates of the gradient, respectively; in addition, and These refer to the first-order moment estimate and the second-order moment estimate of the current gradient for the i-th parameter at the k-th iteration. and These refer to the first-order moment estimate and the second-order moment estimate of the current gradient for the i-th parameter at the (k+1)-th iteration. and They are and Deviation correction;

[0065] After each iteration, the optimal parameter vector θ is updated and applied to the Adam-BP neural network. If the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not reached the maximum value, then the loop enters step S2, selects a new set of sampled data, and iterates and updates the parameter vector θ again until the convergence condition is met.

[0066] Speech recognition devices equipped with Adam-BP neural networks can accurately recognize slang and dialects of various languages, thus reducing the difficulty of interpreters' work in the source language listening and recognition stage. This allows interpreters to have more time to translate and understand the source language and organize and express the target language, which is of great significance for improving the efficiency and quality of interpreters' simultaneous interpretation.

[0067] As an example, such as Figure 4 and 5 As shown, this embodiment provides a voice recognition device, which includes a housing 1. The housing 1 includes a housing support 2, a workbench 3, a transparent glass 4, a noise reduction and sound insulation panel 5, and a ventilation mechanism 6. The housing support 22 is an aluminum alloy structure with grooves on all four sides, which are supported and connected to each other by hexagonal bolts. The workbench 3, the transparent glass 4, and the noise reduction and sound insulation panel 5 are embedded in the grooves of the housing support and are tightly fixed to the housing support by rubber sealing strips. The noise reduction and sound insulation panel 5 is a polyester fiber sound-absorbing board to achieve a greater degree of sound insulation effect for the housing 1. The housing support 2 is made of aluminum alloy, which has the advantages of being lightweight and high-strength, easy to install and disassemble, and has small deformation and high safety. The ventilation mechanism 6 is fixedly connected to the upper part of the housing support 2 by bolts and connected to the inside of the housing 1 through air ducts, thereby realizing the air exchange between the inside and outside of the housing 1. The ventilation mechanism 6 can be a full-pressure ventilator.

[0068] Workbench 3 is equipped with a simultaneous interpretation receiver 7, a simultaneous interpretation machine 8, and a speech recognition device 9. The interpreter obtains the sound of the venue through the simultaneous interpretation receiver 7, and translates the original language into the target language through listening, interpretation, and expression. The speech recognition device 9 can perform intelligent speech recognition on the sound obtained by the simultaneous interpretation receiver 7 and display the speech recognition result on the display of the speech recognition device 9 to assist the interpreter in the simultaneous interpretation work. The simultaneous interpretation machine 8 outputs the translated language to the headphones of the participants in the venue.

[0069] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A speech recognition method, characterized in that, Includes the following steps: The original speech is acquired and fed into a pre-trained Adam-BP neural network to output the speech recognition result. The training process for the pre-constructed Adam-BP neural network is as follows: S1: Initialize the neural network structure parameters and hyperparameters in the Adam-BP neural network model. The structure parameters and hyperparameters include the number of nodes, the number of network layers, the learning rate, and the exponential decay rate parameters of the Adam-BP neural network. S2: Collect a training sample set and input the training sample set into the Adam-BP neural network, then calculate the output of the Adam-BP neural network. ,in These are the weights for the output layer. The threshold value for the output layer. m This represents the number of nodes in the output layer of the neural network. Hidden layer node The output, =1,2··· , This represents the number of hidden layer nodes. S3: Define the objective function of the Adam-BP neural network ,in For the first The true value of each sample For the Adam-BP neural network, the first The predicted value for each sample, For parameter vectors; S4: Using the objective function Backpropagation updates parameter vector and the parameter vector When applied to the Adam-BP neural network, if the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not yet reached the maximum value, then the loop continues into step S2 to iteratively update the parameter vector. Continue until the convergence condition is met; Step S4 specifically includes: Calculate the correction of the first and second moment estimates of the gradient of the Adam-BP neural network, and the first and second moment estimates of the updated gradient; Update the parameter vector based on the first-order moment estimate and second-order moment estimate of the corrected gradient. and the parameter vector Applied to the Adam-BP neural network; If the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not yet reached the maximum value, then the loop continues into step S2 to iteratively update the parameter vector. Continue until the convergence condition is met; The specific formulas for the bias correction of the first and second moment estimates of the gradient in the Adam-BP neural network, as well as the updated first and second moment estimates of the gradient, are as follows: in: and These are the exponential decay rates of the first and second moment estimates of the gradient, respectively; additionally... and They refer to the first i The parameter in the first... k At the next iteration, the first and second moment estimates of the current gradient are... and They refer to the first i The parameter in the first... k At iteration +1, the first and second moment estimates of the current gradient. and They are and Deviation correction.

2. The speech recognition method according to claim 1, characterized in that, For S2 The calculation formula is as follows: in, Here is the activation function for the hidden layer. For the first One input node variable, Hidden layer weights, The hidden layer threshold, This represents the number of nodes in the input layer of the neural network.

3. The speech recognition method according to claim 1, characterized in that, parameter vector The update iteration expression is as follows: in: It is the first During the nth iteration, the neural network... The current gradient is the distance that each parameter descends along the gradient direction. and They refer to the first The parameter in the first... k At the next iteration, the first and second moment estimates of the current gradient are... and They are and Deviation correction, Let ε be the learning rate, and ε be a constant.

4. The speech recognition method according to claim 1, characterized in that, The Adam-BP neural network includes an input layer, a hidden layer, and an output layer, and the training sample set includes a collection of speech under different environmental conditions.

5. A speech recognition system, characterized in that, The system is used to acquire raw speech and feed the raw speech into a pre-trained Adam-BP neural network to output speech recognition results. The speech recognition system includes a parameter initialization module, a sample set collection and input module, an objective function definition module, and a backpropagation judgment module. The parameter initialization module is used to initialize the neural network structure parameters and hyperparameters in the Adam-BP neural network model. The structure parameters and hyperparameters include the number of nodes, the number of network layers, the learning rate, and the exponential decay rate parameters of the Adam-BP neural network. The sample set collection input module is used to collect the training sample set and input the k-th sample in the training sample set into the Adam-BP neural network to calculate the output of the Adam-BP neural network. ,in These are the weights for the output layer. The threshold value for the output layer. m This represents the number of nodes in the output layer of the neural network. Hidden layer node The output, =1,2··· , This represents the number of hidden layer nodes. The objective function definition module is used to define the objective function of the Adam-BP neural network. ,in For the first The true value of each sample For the Adam-BP neural network, the first The predicted value for each sample, For parameter vectors; The backpropagation decision module is used to utilize the objective function Backpropagation updates parameter vector and the parameter vector When applied to the Adam-BP neural network, if the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not yet reached the maximum value, then the loop continues into step S2 to iteratively update the parameter vector. Continue until the convergence condition is met; The backpropagation judgment module specifically includes: Calculate the correction of the first and second moment estimates of the gradient of the Adam-BP neural network, and the first and second moment estimates of the updated gradient; Update the parameter vector based on the first-order moment estimate and second-order moment estimate of the corrected gradient. and the parameter vector Applied to the Adam-BP neural network; If the value of the loss function in the Adam-BP neural network is still greater than the pre-set threshold and the number of iterations has not yet reached the maximum value, then the loop continues into step S2 to iteratively update the parameter vector. Continue until the convergence condition is met; The specific formulas for the bias correction of the first and second moment estimates of the gradient in the Adam-BP neural network, as well as the updated first and second moment estimates of the gradient, are as follows: in: and These are the exponential decay rates of the first and second moment estimates of the gradient, respectively; additionally... and They refer to the first i The parameter in the first... k At the next iteration, the first and second moment estimates of the current gradient are... and They refer to the first i The parameter in the first... k At iteration +1, the first and second moment estimates of the current gradient. and They are and Deviation correction.

6. A voice recognition device, characterized in that, The device includes a housing, in which a simultaneous interpretation receiver and a speech recognition device are installed. After acquiring the original speech, the simultaneous interpretation receiver transmits the original speech to the speech recognition device. The speech recognition device processes the original speech to output a speech recognition result. The speech recognition device is configured with a program to implement the speech recognition method as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, The computer storage medium stores a plurality of classification programs, which are used by the processor to call and execute the speech recognition method as described in any one of claims 1 to 4.

Images