Voice training, recognition method and device, electronic device, storage medium

Abstract

The present disclosure relates to a speech training method and device, a speech recognition method and device, an electronic device, and a storage medium. The speech training method includes: obtaining a first data set and a first speech recognition model; the first speech recognition model is a model completed by neural network training based on a second data set; the first data set includes a plurality of first data sample combinations, wherein the first data sample combination includes: first category speech data and labels corresponding to the first category speech data; the second data set includes a plurality of second data sample combinations, wherein the second data sample combination includes: second category speech data and labels corresponding to the second category speech data; using the first data set, the second speech recognition model is trained until the recognition rate of the second speech recognition model meets a preset condition, thereby obtaining a trained second speech recognition model; wherein the second recognition model includes: a fully connected layer and the first speech recognition model.

Description

Technical Field

[0001] The present disclosure relates to speech recognition technology, and in particular to a speech training method and device, a speech recognition method and device, an electronic device, and a storage medium. Background Art

[0002] Current speech recognition technology is relatively mature, boasting high accuracy for normal speech input and commercially available. However, for individuals with speech impairments, such as those with cerebral palsy or lisping, speech recognition accuracy is extremely low, rendering current speech recognition technology virtually unusable. These individuals, whether out and about or at home, have a significant need for communication, making the demand for speech recognition even greater. Summary of the Invention

[0003] The present disclosure provides a speech training method and device, a speech recognition method and device, an electronic device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a speech training method, comprising:

[0005] Obtain a first data set and a first speech recognition model; the first speech recognition model is a model trained by a neural network based on the second data set; the first data set includes multiple first data sample combinations, wherein the first data sample combinations include: first category speech data and labels corresponding to the first category speech data; the second data set includes multiple second data sample combinations, wherein the second data sample combinations include: second category speech data and labels corresponding to the second category speech data;

[0006] The first data set is used to train a second speech recognition model until the recognition rate of the second speech recognition model meets a preset condition, thereby obtaining a trained second speech recognition model; wherein the second recognition model includes: a fully connected layer and the first speech recognition model.

[0007] Optionally, during the training process, the first weight parameter in the first speech recognition model is kept unchanged, and the second weight in the fully connected layer is continuously adjusted.

[0008] Optionally, the first data set includes multiple batches of training data, wherein each batch of training data includes: multiple combinations of first data samples; using the first data set, the second speech recognition model is trained until the recognition rate of the second speech recognition model meets a preset condition, thereby obtaining a trained second speech recognition model:

[0009] The first batch of training data among the multiple batches of training data is used as current training data, and the following steps are executed in a loop until the recognition rate of the second speech recognition model meets a preset condition, including:

[0010] Training the second speech recognition model using the current training data to obtain a second speech recognition model after the first training;

[0011] Determining whether the recognition rate of the second speech recognition model after the first training meets the preset condition;

[0012] If the preset condition is not met, obtaining a second batch of training data from the multiple batches of training data; wherein the second batch of training data is different from the current training data;

[0013] The second batch of training data is used as the current training data.

[0014] Optionally, training the second speech recognition model using the current training data to obtain the second speech recognition model after the first training includes:

[0015] The first first data sample combination in the first batch of training data is used as the current first data training combination, and the following steps are performed in a loop until the data in the first batch of training data is exhausted:

[0016] Inputting the first category of speech data in the current first data training combination into the fully connected layer, and outputting an intermediate result;

[0017] Inputting the intermediate result into the first speech recognition model;

[0018] Adjusting the second weight of the fully connected layer according to a result output by the first speech recognition model and a label corresponding to the first category of speech data;

[0019] Obtaining a second first data sample combination in the first batch of training data; wherein the second first data sample combination is different from the current first data training combination;

[0020] Using the second first data sample combination as the current first data training combination;

[0021] The second speech recognition model trained by the last second data sample combination in the first batch of training data is used as the second speech recognition model after the first training.

[0022] Optionally, the preset condition includes: the recognition rate of the second speech recognition model is greater than or equal to a preset threshold.

[0023] Optionally, the first voice category data is voice data of a person having a first physiological characteristic; and the label is text data corresponding to the first semantic category data.

[0024] Optionally, before inputting the first category speech data in the current first data training set into the fully connected layer, the method further includes:

[0025] Set an activation function for the fully connected layer.

[0026] According to a second aspect of an embodiment of the present disclosure, there is provided a speech recognition method, including:

[0027] Acquiring a first voice of a person having a first physiological characteristic;

[0028] The first speech is input into a second speech recognition model trained according to the speech training method; and a recognition result of the first speech is output.

[0029] According to a third aspect of an embodiment of the present disclosure, there is provided a speech training device, comprising:

[0030] An acquisition unit is configured to acquire a first data set and a first speech recognition model; the first speech recognition model is a model trained by a neural network based on the second data set; the first data set includes a plurality of first data sample combinations, wherein the first data sample combinations include: first category speech data and labels corresponding to the first category speech data; the second data set includes a plurality of second data sample combinations, wherein the second data sample combinations include: second category speech data and labels corresponding to the second category speech data;

[0031] A training unit is used to train a second speech recognition model using the first data set until the recognition rate of the second speech recognition model meets a preset condition, thereby obtaining a trained second speech recognition model; wherein the second recognition model includes: a fully connected layer and the first speech recognition model.

[0032] Optionally, the training unit is further configured to:

[0033] During the training process, the first weight parameter in the first speech recognition model is kept unchanged, and the second weight in the fully connected layer is continuously adjusted.

[0034] Optionally, the first data set includes multiple batches of training data, wherein each batch of training data includes: multiple first data sample combinations;

[0035] Correspondingly, the training unit is further used to:

[0036] Using the first batch of training data from the multiple batches of training data as current training data, the following steps are performed in a loop until the recognition rate of the second speech recognition model meets a preset condition:

[0037] Training the second speech recognition model using the current training data to obtain a second speech recognition model after the first training;

[0038] Determining whether the recognition rate of the second speech recognition model after the first training meets the preset condition;

[0039] If the preset condition is not met, obtaining a second batch of training data from the multiple batches of training data; wherein the second batch of training data is different from the current training data;

[0040] The second batch of training data is used as the current training data.

[0041] Optionally, the training unit is further configured to:

[0042] The first first data sample combination in the first batch of training data is used as the current first data training combination, and the following steps are performed in a loop until the data in the first batch of training data is exhausted:

[0043] Inputting the first category of speech data in the current first data training combination into the fully connected layer, and outputting an intermediate result;

[0044] Inputting the intermediate result into the first speech recognition model;

[0045] Adjusting the second weight of the fully connected layer according to a result output by the first speech recognition model and a label corresponding to the first category of speech data;

[0046] Obtaining a second first data sample combination in the first batch of training data; wherein the second first data sample combination is different from the current first data training combination;

[0047] Using the second first data sample combination as the current first data training combination;

[0048] The second speech recognition model trained by the last second data sample combination in the first batch of training data is used as the second speech recognition model after the first training.

[0049] Optionally, the preset condition includes: the recognition rate of the second speech recognition model is greater than or equal to a preset threshold.

[0050] Optionally, the first voice category data is voice data of a person having a first physiological characteristic; and the label is text data corresponding to the first semantic category data.

[0051] Optionally, the device further includes:

[0052] The setting unit is used to set an activation function for the fully connected layer; the training unit is also used to input the first category speech data in the current first data training combination into the fully connected layer and output an intermediate result.

[0053] According to a fourth aspect of an embodiment of the present disclosure, there is provided a speech recognition device, including:

[0054] A collection unit, configured to collect a first voice of a person having a first physiological characteristic;

[0055] an input unit, configured to input the first speech into a second speech recognition model trained according to the speech training method;

[0056] an acquiring unit, configured to acquire a recognition result of the second speech recognition model based on the first speech;

[0057] An output unit is used to output the recognition result.

[0058] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor and a memory for storing processor-executable instructions, wherein the processor is configured to execute the steps of the above-mentioned voice training method when calling the executable instructions in the memory.

[0059] According to the sixth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor and a memory for storing processor executable instructions, wherein the processor is configured to execute the steps of the above-mentioned speech recognition method when calling the executable instructions in the memory.

[0060] According to the seventh aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the steps of the above-mentioned voice training method or the steps of the above-mentioned voice recognition method.

[0061] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0062] In an embodiment of the present disclosure, a second speech recognition model is formed by collecting limited segments of speech from a person with a speech impairment and obtaining a first speech recognition model based on a neural network that has been trained in advance with the speech of an ordinary person. A fully connected layer with a linear transformation is added to the input of the first speech recognition model. The weights determined in the first speech recognition model are maintained, and the second speech recognition model is iteratively trained using a backpropagation algorithm using the collected speech of the person with a speech impairment to determine the weights of the fully connected layer that meet the training requirements, thereby completing the training of the second speech recognition model. The embodiment of the present disclosure uses the second speech recognition model of the person with a speech impairment to recognize their speech, and the recognition accuracy is greatly improved, enabling the person with a speech impairment to communicate with others without obstacles. By collecting limited speech training data, the second speech recognition model of the embodiment of the present disclosure has increased the speech recognition accuracy of the person with a speech impairment by nearly four times, greatly improving the accuracy of speech recognition.

[0063] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0065] Figure 1 A flowchart of a speech training method according to an embodiment of the present disclosure is shown;

[0066] Figure 2 This is a schematic diagram of a speech training network architecture according to an embodiment of the present disclosure;

[0067] Figure 3 A flowchart of a speech recognition method according to an embodiment of the present disclosure is shown;

[0068] Figure 4 A schematic diagram of the structure of a speech training device according to an embodiment of the present disclosure is shown;

[0069] Figure 5 A schematic diagram of the structure of a speech recognition device according to an embodiment of the present disclosure;

[0070] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0071] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of devices and apparatuses consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0072] Figure 1 A flow chart of a speech training method according to an embodiment of the present disclosure is shown as follows. Figure 1 As shown, the speech training method of the embodiment of the present disclosure includes the following steps:

[0073] S11, obtaining a first data set and a first speech recognition model.

[0074] In an embodiment of the present disclosure, the first speech recognition model is a model trained through a neural network based on a second data set; the first data set includes multiple first data sample combinations, wherein the first data sample combinations include: first category speech data and labels corresponding to the first category speech data; the second data set includes multiple second data sample combinations, wherein the second data sample combinations include: second category speech data and labels corresponding to the second category speech data.

[0075] In the embodiment of the present disclosure, the first category of voice data mainly refers to voice data collected from people with slurred speech, such as people with speech disorders. The pronunciation of this group of people is difficult for others to understand; therefore, it is difficult for normal voice recognition models to accurately recognize the voice of this group of people. Generally speaking, when the first voice recognition model obtained based on voice training of normal people recognizes the voice of this group of people, the recognition accuracy is extremely low, and generally only the voice that is more consistent with normal pronunciation can be recognized. The second category of voice data in the embodiment of the present disclosure refers to voice data collected from people with normal speech. The label of the first category of voice data can be text data transcribed by the person with speech disorders themselves from the collected first category of voice data; this may make the labeling more accurate, thereby improving the recognition rate of the trained voice recognition model.

[0076] In the embodiment of the present disclosure, the first speech recognition model is a model trained by a neural network based on the second category of speech. That is, in the embodiment of the present disclosure, the first speech recognition model is a speech recognition model trained by a neural network using the speech of an ordinary person in advance. The first speech recognition model uses a conventional speech recognition training method. Various common network structures suitable for speech recognition can be used to generate the first speech recognition model. As an example, the neural network can adopt an Alex-Net network model, a VGG-Nets network model, a residual network model, etc.

[0077] S12. Use the first data set to train a second speech recognition model until the recognition rate of the second speech recognition model meets a preset condition, thereby obtaining a trained second speech recognition model.

[0078] In the embodiment of the present disclosure, the second recognition model includes: a fully connected layer and the first speech recognition model.

[0079] Figure 2 This is a schematic diagram of a speech training network architecture shown in an embodiment of the present disclosure, such as Figure 2 As shown in FIG, the rectangular box with circular nodes represents the first speech recognition model, the fully connected layer (Input Transform) is connected to the front end of the first speech recognition model, and the output end of the fully connected layer serves as the input end of the first speech recognition model. Figure 2 The fully connected layer and the first speech recognition model in the training data set are used as the training objects (i.e., the second speech recognition model), and the fully connected layer and the first speech recognition model are trained using the first dataset. During the training process, the first weight parameter in the first speech recognition model is kept unchanged (i.e., the learning rate of the first speech recognition model is set to 0), and the second weight in the fully connected layer is continuously adjusted.

[0080] It should be noted that the fully connected layer can be a fully connected layer of a linear transformation, and the activation function is a linear function.

[0081] In an embodiment of the present disclosure, the first data set includes multiple batches of training data, wherein each batch of training data includes: multiple combinations of first data samples; using the first data set, a second speech recognition model is trained until the recognition rate of the second speech recognition model meets a preset condition, thereby obtaining a trained second speech recognition model:

[0082] Using the first batch of training data from the multiple batches of training data as current training data, the following steps are performed in a loop until the recognition rate of the second speech recognition model meets a preset condition:

[0083] The second speech recognition model is trained using the current training data to obtain a second speech recognition model after the first training; whether the recognition rate of the second speech recognition model after the first training meets the preset conditions is determined; if the preset conditions are not met, a second batch of training data from the multiple batches of training data is obtained; wherein the second batch of training data is different from the current training data; and the second batch of training data is used as the current training data.

[0084] In the embodiment of the present disclosure, training the second speech recognition model using the current training data to obtain the second speech recognition model after the first training specifically includes:

[0085] The first first data sample combination in the first batch of training data is used as the current first data training combination, and the following steps are performed in a loop until the data in the first batch of training data is exhausted:

[0086] Input the first category speech data in the current first data training combination into the fully connected layer and output an intermediate result; input the intermediate result into the first speech recognition model; adjust the second weight of the fully connected layer according to the result output by the first speech recognition model and the label corresponding to the first category speech data; obtain the second first data sample combination in the first batch of training data; wherein the second first data sample combination is different from the current first data training combination; use the second first data sample combination as the current first data training combination; use the second speech recognition model trained with the last second data sample combination in the first batch of training data as the second speech recognition model after the first training.

[0087] It should be noted that the preset condition may include the recognition rate of the second speech recognition model being greater than or equal to a preset threshold. If the recognition rate of the second speech recognition model is expected to reach 90%, the preset threshold may be 90%. Of course, the preset threshold may also be set based on the recognition rate requirements of the recognition model, such as 95%, 80%, etc.

[0088] In the disclosed embodiment, a fully connected layer with a linear transformation is added to the input of the first speech recognition model. A corresponding activation function may also be configured for the fully connected layer as needed. It should be noted that the fully connected layer in the disclosed embodiment is a common training layer in a neural network, and it only needs to perform a pre-linear transformation on the input first category speech data.

[0089] In the disclosed embodiment, it may be difficult to collect the voice of a person with a speech impairment. Therefore, a smaller amount of voice of the person with a speech impairment may be collected, and the person with the speech impairment may transcribe the collected voice into text data themselves, and use the text data as a label for the first category of voice data.

[0090] In the embodiment of the present disclosure, after the speech of a person with a speech disorder is collected, a first data set is generated, that is, the collected speech of the person with a speech disorder and the corresponding labels are divided into multiple batches of training data, the multiple batches of training data are used to train the second speech recognition model, and the weights of the fully connected layers in the second speech recognition model are adjusted, the recognition rate of the trained second speech recognition model is tested using the verification data set, and when it is determined that the requirements are not met, another batch of training data is used to continue training and verifying the second speech recognition model until the speech recognition rate of the second speech recognition model meets the recognition requirements.

[0091] It should be noted that the validation data set may include: first category speech data and labels corresponding to the first category speech data; the validation data set may be considered as a data set for verifying the performance of the trained model.

[0092] In the embodiment of the present disclosure, using Figure 2 When training the first category of speech data, the illustrated speech training structure locks the weights of the first speech recognition model, already determined by the original neural network training, by setting the learning rate to 0. The added fully connected layer is then iteratively trained using a backpropagation algorithm, updating only the weights of the newly added linearly transformed fully connected layer at the input. This training continues through multiple iterations until the speech recognition rate of the second speech recognition model on the development set is greater than or equal to a preset threshold. This threshold can be 80%, 85%, 93%, 95%, or other thresholds.

[0093] In the embodiment of the present disclosure, in the aforementioned iterative training, the second speech recognition model that meets the recognition rate requirement is used as the final speech recognition model.

[0094] After adopting the speech training method of the embodiment of the present disclosure, only a speech of about 5 minutes of speech by a person with speech impairment was used, and the speech was used as a label. Figure 2 The training process shown improves the recognition rate of the first speech recognition model from about 25% to 95%, and the recognition rate of the first category of speech is increased by nearly four times. Obviously, the second speech recognition model obtained by the speech training method of the embodiment of the present disclosure can provide a convenient communication method for people with speech disorders and can solve the communication barriers of this group of people.

[0095] Figure 3 A flow chart of a speech recognition method according to an embodiment of the present disclosure is shown as follows. Figure 3As shown, the speech recognition method of the embodiment of the present disclosure includes the following processing steps:

[0096] S31, collecting a first voice of a person having a first physiological characteristic.

[0097] In the disclosed embodiment, the first physiological feature is slurred speech; the first voice mainly refers to the voice of people with slurred speech, such as people with speech disorders. The pronunciation of this group of people is difficult for others to understand, and there is an urgent need for a voice recognition model for this group of people to better express their voice accurately to others.

[0098] S32: Input the first speech into a second speech recognition model trained according to the speech training method, and output a recognition result of the first speech.

[0099] In the embodiment of the present disclosure, the second speech recognition model is the second speech recognition model trained by the speech training method of the aforementioned embodiment. The second speech recognition model can be used to accurately recognize the first speech of a person with the first physiological characteristics.

[0100] The speech recognition method of the embodiment of the present disclosure can be applied to mobile phones, portable electronic devices, automatic translators, PADs, etc., and can be used by people with speech impairments as long as the corresponding training speech is input.

[0101] Figure 4 This is a schematic diagram of the structure of a speech training device according to an embodiment of the present disclosure. Figure 4 As shown, the speech training device of the embodiment of the present disclosure includes:

[0102] An acquisition unit 40 is configured to acquire a first data set and a first speech recognition model; the first speech recognition model is a model trained using a neural network based on a second data set; the first data set includes a plurality of first data sample combinations, wherein the first data sample combinations include: first category speech data and labels corresponding to the first category speech data; the second data set includes a plurality of second data sample combinations, wherein the second data sample combinations include: second category speech data and labels corresponding to the second category speech data;

[0103] The training unit 41 is used to train the second speech recognition model using the first data set until the recognition rate of the second speech recognition model meets the preset conditions, thereby obtaining a trained second speech recognition model; wherein the second recognition model includes: a fully connected layer and the first speech recognition model.

[0104] As an implementation manner, the training unit 40 is further configured to:

[0105] During the training process, the first weight parameter in the first speech recognition model is kept unchanged, and the second weight in the fully connected layer is continuously adjusted.

[0106] As an implementation, the first data set includes multiple batches of training data, wherein each batch of training data includes: multiple first data sample combinations;

[0107] Correspondingly, the training unit 41 is further configured to:

[0108] Using the first batch of training data from the multiple batches of training data as current training data, the following steps are performed in a loop until the recognition rate of the second speech recognition model meets a preset condition:

[0109] Training the second speech recognition model using the current training data to obtain a second speech recognition model after the first training;

[0110] Determining whether the recognition rate of the second speech recognition model after the first training meets the preset condition;

[0111] If the preset condition is not met, obtaining a second batch of training data from the multiple batches of training data; wherein the second batch of training data is different from the current training data;

[0112] The second batch of training data is used as the current training data.

[0113] As an implementation manner, the training unit 41 is further configured to:

[0114] The first first data sample combination in the first batch of training data is used as the current first data training combination, and the following steps are performed in a loop until the data in the first batch of training data is exhausted:

[0115] Inputting the first category of speech data in the current first data training combination into the fully connected layer, and outputting an intermediate result;

[0116] Inputting the intermediate result into the first speech recognition model;

[0117] Adjusting the second weight of the fully connected layer according to a result output by the first speech recognition model and a label corresponding to the first category of speech data;

[0118] Obtaining a second first data sample combination in the first batch of training data; wherein the second first data sample combination is different from the current first data training combination;

[0119] Using the second first data sample combination as the current first data training combination;

[0120] The second speech recognition model trained by the last second data sample combination in the first batch of training data is used as the second speech recognition model after the first training.

[0121] As an implementation method, the preset condition includes: the recognition rate of the second speech recognition model is greater than or equal to a preset threshold.

[0122] As an implementation manner, the first voice category data is voice data of a person having a first physiological characteristic; and the label is text data corresponding to the first semantic category data.

[0123] As a way to implement Figure 4 Based on the voice training device shown in FIG, the voice training device of the embodiment of the present disclosure further includes:

[0124] Set the unit ( Figure 4 (not shown) for setting an activation function for the fully connected layer; the training unit 41 is further used to input the first category speech data in the current first data training combination into the fully connected layer and output an intermediate result.

[0125] In an exemplary embodiment, the acquisition unit 40, the training unit 41, the setting unit, etc. can be implemented by one or more central processing units (CPU), graphics processing units (GPU), baseband processors (BP), application-specific integrated circuits (ASIC), digital signal processors (DSP), programmable logic devices (PLD), complex programmable logic devices (CPLD), field programmable gate arrays (FPGA), general-purpose processors, controllers, microcontrollers (MCU), microprocessors, or other electronic components.

[0126] Regarding the speech training device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0127] Figure 5 This is a schematic diagram of the structure of a speech recognition device according to an embodiment of the present disclosure. Figure 5 As shown, the speech recognition device of the embodiment of the present disclosure includes:

[0128] The collecting unit 50 is configured to collect a first speech of a person having a first physiological characteristic;

[0129] An input unit 51, configured to input the first speech into a second speech recognition model trained according to the speech training method;

[0130] The output unit 52 is configured to output the recognition result of the first speech.

[0131] In an exemplary embodiment, the acquisition unit 50, the input unit 51, the output unit 52, etc. can be implemented by one or more central processing units (CPUs), graphics processing units (GPUs), baseband processors (BPs), application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components.

[0132] Regarding the speech recognition device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0133] Figure 6 FIG. 8 is a block diagram of an electronic device 800 according to an exemplary embodiment. Figure 6 As shown, the electronic device 800 supports multi-screen output, and the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0134] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned speech training and recognition methods. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0135] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0136] The power component 806 provides power to the various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0137] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0138] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0139] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0140] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor assembly 814 can also detect changes in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and temperature changes of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0141] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0142] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing steps of the above-described embodiments.

[0143] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions. The instructions can be executed by the processor 820 of the electronic device 800 to complete the steps of the speech training and recognition method of the above embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0144] The embodiments of the present disclosure also describe a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can perform the speech training and recognition methods of the aforementioned embodiments.

[0145] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0146] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A speech training method, characterized in that: The method comprises: Obtain a first data set and a first speech recognition model; the first speech recognition model is a model trained by a neural network based on the second data set; the first data set includes multiple first data sample combinations, wherein the first data sample combinations include: first category speech data and labels corresponding to the first category speech data; the second data set includes multiple second data sample combinations, wherein the second data sample combinations include: second category speech data and labels corresponding to the second category speech data; The first data set is used to train the second speech recognition model until the recognition rate of the second speech recognition model meets the preset conditions, thereby obtaining the trained second speech recognition model; wherein, the second speech recognition model includes: a fully connected layer and the first speech recognition model; the fully connected layer is located at the front end of the first speech recognition model; during the training process, the first weight parameter in the first speech recognition model is kept unchanged, and the second weight in the fully connected layer is continuously adjusted.

2. The method according to claim 1, characterized in that The first data set includes multiple batches of training data, wherein each batch of training data includes: multiple combinations of first data samples; using the first data set, training the second speech recognition model until the recognition rate of the second speech recognition model meets a preset condition, and the trained second speech recognition model includes: Using the first batch of training data from the multiple batches of training data as current training data, the following steps are performed in a loop until the recognition rate of the second speech recognition model meets a preset condition: Training the second speech recognition model using the current training data to obtain a second speech recognition model after the first training; Determining whether the recognition rate of the second speech recognition model after the first training meets the preset condition; If the preset condition is not met, obtaining a second batch of training data from the multiple batches of training data; wherein the second batch of training data is different from the current training data; The second batch of training data is used as the current training data.

3. The method according to claim 2, characterized in that The second speech recognition model is trained using the current training data to obtain a second speech recognition model after the first training, comprising: The first first data sample combination in the first batch of training data is used as the current first data training combination, and the following steps are performed in a loop until the data in the first batch of training data is exhausted: Inputting the first category of speech data in the current first data training combination into the fully connected layer, and outputting an intermediate result; Inputting the intermediate result into the first speech recognition model; Adjusting the second weight of the fully connected layer according to a result output by the first speech recognition model and a label corresponding to the first category of speech data; Obtaining a second first data sample combination in the first batch of training data; wherein the second first data sample combination is different from the current first data training combination; Using the second first data sample combination as the current first data training combination; The second speech recognition model trained by the last second data sample combination in the first batch of training data is used as the second speech recognition model after the first training.

4. The method according to claim 1, wherein The preset condition includes: the recognition rate of the second speech recognition model is greater than or equal to a preset threshold.

5. The method according to claim 1, wherein The first category of voice data is voice data of a person having a first physiological characteristic; the label is text data corresponding to the first category of voice data.

6. The method according to claim 3, characterized in that Before inputting the first category speech data in the current first data training set into the fully connected layer, the method further includes: Set an activation function for the fully connected layer.

7. A speech recognition method, characterized in that: The method comprises: Acquiring a first voice of a person having a first physiological characteristic; Inputting the first speech into a second speech recognition model trained by the speech training method according to any one of claims 1 to 6; Output the recognition result of the first speech.

8. A speech training device, characterized in that: The device comprises: An acquisition unit is configured to acquire a first data set and a first speech recognition model; the first speech recognition model is a model trained by a neural network based on the second data set; the first data set includes a plurality of first data sample combinations, wherein the first data sample combinations include: first category speech data and labels corresponding to the first category speech data; the second data set includes a plurality of second data sample combinations, wherein the second data sample combinations include: second category speech data and labels corresponding to the second category speech data; a training unit, configured to train a second speech recognition model using the first data set until a recognition rate of the second speech recognition model satisfies a preset condition, thereby obtaining a trained second speech recognition model; wherein the second speech recognition model comprises: a fully connected layer and the first speech recognition model; the fully connected layer is located at the front end of the first speech recognition model; The training unit is further used to: during the training process, keep the first weight parameter in the first speech recognition model unchanged and continuously adjust the second weight in the fully connected layer.

9. An electronic device, characterized in that: The electronic device includes: a processor and a memory for storing processor-executable instructions, wherein the processor is configured to execute the steps of the speech training method according to any one of claims 1 to 6 when calling the executable instructions in the memory.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory for storing processor-executable instructions, wherein the processor is configured to execute the steps of the speech recognition method according to claim 7 when calling the executable instructions in the memory.

11. A non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the steps of the speech training method according to any one of claims 1 to 6, or the steps of the speech recognition method according to claim 7.

Images