Speech recognition method, electronic equipment, storage medium and chip system

Abstract

The embodiment of the invention provides a voice recognition method, electronic equipment, a storage medium and a chip system, and relates to the technical field of terminals. In the method, a first voice recognition model can be preset in the electronic equipment, the first voice recognition model can be a lightweight voice recognition model used for recognizing non-standard voice data, and when the electronic equipment receives first voice data, the electronic equipment can input the first voice data into the first voice recognition model. The standard text corresponding to the first voice data is obtained, and the electronic equipment can execute an instruction corresponding to the standard text. Since the electronic equipment does not need to interact with the cloud server, the standard text corresponding to the first voice data can be directly obtained, the time delay can be reduced, and the user experience is improved. In addition, the first speech recognition model is a lightweight speech recognition model, so that the first speech recognition model can be conveniently deployed in the electronic equipment.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a speech recognition method, electronic equipment, storage medium and chip system. Background Art

[0002] With the rapid development of science and technology, voice recognition technology has penetrated into users’ lives. For example, users can control electronic devices through voice commands without manual operation, which can simplify the interaction process between users and electronic devices and improve interaction efficiency.

[0003] However, due to the user's pronunciation habits, such as repeated, prolonged or paused pronunciation, the non-standard voice commands received by the electronic device from the user do not match the standard voice commands in the voice command library preset by the electronic device. Therefore, the electronic device cannot recognize the non-standard voice commands issued by the user, affecting the user experience. Summary of the invention

[0004] The embodiments of the present application provide a voice recognition method, an electronic device, a storage medium, and a chip system, which are applied to the field of terminal technology and can enable the electronic device to recognize non-standard voice commands issued by the user and improve the user experience.

[0005] In the first aspect, an embodiment of the present application proposes a speech recognition method, and the subject executing the method may be an electronic device or a chip in an electronic device. The following is an explanation taking an electronic device as an example. The method may include: the electronic device receives first voice data, the first voice data may include non-standard voice data, and the electronic device may input the first voice data into a first voice recognition model to obtain a standard text corresponding to the first voice data. Among them, the first voice recognition model is a lightweight speech recognition model preset in the electronic device for identifying non-standard voice data. After obtaining the standard text corresponding to the first voice data, the electronic device may execute the instruction corresponding to the standard text.

[0006] It should be understood that the first voice data may be a voice command from the user, such as the audio corresponding to “turn up the volume”.

[0007] In the embodiment of the present application, since the first speech recognition model is a lightweight speech recognition model, it is easy to deploy in an electronic device. And since the electronic device can obtain the standard text corresponding to the first voice data based on the first speech recognition model without interacting with the cloud server, the latency can be reduced, thereby improving the recognition efficiency of the electronic device and enhancing the user experience.

[0008] In one possible implementation, the first speech recognition model may include a feature extraction module, an acoustic module and a feature matching module connected in sequence; wherein the feature extraction module can be used to extract audio features of speech data, the acoustic module can be used to extract acoustic features of speech data, and the feature matching module may include a preset feature database, the feature database includes acoustic features of multiple non-standard speech samples, and standard text corresponding to the acoustic features of each non-standard speech sample.

[0009] The feature extraction module, acoustic module, and feature matching module connected in sequence can be understood as: the output end of the feature extraction module is connected to the input end of the acoustic module, and the output end of the acoustic module is connected to the input end of the feature matching module.

[0010] In other words, the output of the feature extraction module is the input of the acoustic module, and the output of the acoustic module is the input of the feature matching module.

[0011] In this implementation, since a standard text in the feature data can correspond to the acoustic features of multiple non-standard voice samples, the electronic device can obtain the standard text corresponding to the acoustic features of the non-standard voice samples based on the acoustic features of the non-standard voice samples, and execute instructions corresponding to the standard text.

[0012] In a possible implementation, the electronic device may input the first voice data into a feature extraction module in the first voice recognition model, and obtain the first audio feature after processing by the feature extraction module. The first audio feature may include time domain information and frequency domain information of the first voice data. After obtaining the first audio feature, the electronic device may input the first audio feature into an acoustic module in the first voice recognition model, and obtain the first acoustic feature after processing by the acoustic module. The first acoustic feature is used to indicate the semantic information of the first voice data. After obtaining the first acoustic feature, the electronic device may input the first acoustic feature into a feature matching module in the first voice recognition model, and obtain the standard text corresponding to the first voice data after processing by the feature matching module.

[0013] Exemplarily, the first audio feature may be spectrum energy.

[0014] In this implementation, the electronic device processes the first voice data in stages (feature extraction module-acoustic module-feature matching module), and finally maps the first voice data to a standard text. Since the electronic device can recognize the standard text, the electronic device can execute instructions corresponding to the standard text.

[0015] In a possible implementation, the electronic device may traverse the feature database through the feature matching module, and obtain a target acoustic feature whose similarity with the first acoustic feature is greater than a similarity threshold from the feature database. After obtaining the target acoustic feature, the electronic device may use the standard text corresponding to the target acoustic feature as the standard text corresponding to the first voice data.

[0016] Exemplarily, the feature database contains acoustic features of multiple non-standard speech samples. The electronic device can calculate the similarity between the acoustic feature of each non-standard speech sample and the first acoustic feature, thereby determining multiple similarities. The electronic device can select the largest similarity among the multiple similarities and determine whether the largest similarity is greater than a similarity threshold. If it is greater than the similarity threshold, the electronic device can use the acoustic feature of the non-standard speech sample corresponding to the largest similarity as the target acoustic feature.

[0017] In this implementation, since the target acoustic feature determined by the electronic device is: an acoustic feature of a non-standard speech sample whose similarity to the first acoustic feature is greater than a similarity threshold, the accuracy of the electronic device's recognition can be improved.

[0018] In a possible implementation manner, the first voice data may include at least one of a repeated syllable, a dragged syllable, or a paused syllable.

[0019] In a possible implementation, the electronic device may further include a non-standard speech generation model, and the electronic device may obtain a first standard speech sample, a first standard text corresponding to the first standard speech sample, and a first non-standard text corresponding to the first standard speech sample. The electronic device may input the first standard speech sample and the first non-standard text into the non-standard speech generation model at the same time to obtain the first non-standard speech sample. The electronic device may input the first non-standard speech sample into the first speech recognition model, and obtain the acoustic features of the first non-standard speech sample after being processed by the feature extraction module and the acoustic module in the first speech recognition model. The electronic device may store the acoustic features of the first non-standard speech sample and the first standard text as a set of data in a feature database. Based on the above steps, the electronic device may construct other sets of data in the feature database.

[0020] In this implementation, the electronic device can use a non-standard speech generation model to obtain a first non-standard speech sample, and input the first non-standard speech sample into a first speech recognition model to obtain the acoustic features of the first non-standard speech sample. The acoustic features of the first non-standard speech sample can be used for subsequent matching, thereby saving processing resources of the electronic device and improving recognition efficiency.

[0021] In a possible implementation, the non-standard speech generation model may include an audio encoding module, a text encoding module, a feature fusion module, and an audio decoding module, wherein the output ends of the audio encoding module and the text encoding module are both connected to the input end of the feature fusion module, and the output end of the feature fusion module is connected to the input end of the audio decoding module.

[0022] In this implementation, a first non-standard speech sample can be generated through an audio encoding module, a text encoding module, a feature fusion module, and an audio decoding module in a non-standard speech generation model.

[0023] In a possible implementation, the electronic device may input the first standard voice sample into the audio encoding module of the non-standard voice generation model to obtain the audio features of the first standard voice sample, and input the first non-standard text into the text encoding module of the non-standard voice generation model to obtain the text features of the first non-standard text. The electronic device may input the audio features of the first standard voice sample and the text features of the first non-standard sample into the feature fusion module of the non-standard voice generation model at the same time to obtain the audio features of the first non-standard voice sample. The electronic device may input the audio features of the first non-standard voice sample into the audio decoding module of the non-standard voice generation model to obtain the first non-standard voice sample.

[0024] In this implementation, a first non-standard voice sample can be obtained after processing by an audio encoding module, a text encoding module, a feature fusion module, and an audio decoding module, so that the electronic device can generate acoustic features of the first non-standard voice sample based on the first non-standard voice sample.

[0025] In a possible implementation, the non-standard speech generation model may further include a random variable generation module, the output end of the random variable generation module is connected to the input end of the feature fusion module, and the random variable generation module is used to generate audio features of random noise.

[0026] Exemplarily, the input of the random variable generation module may be a standard normal distribution, and the random variable generation module may sample from the standard normal distribution to obtain a latent variable in the standard normal distribution, and the latent variable may be used as an audio feature of the random noise.

[0027] In this implementation, the audio features of the random noise generated by the random variable generation module can be used to guide the fusion process of the feature fusion module, so that the audio features of the first non-standard speech sample generated by the feature fusion module have diversity and randomness, thereby avoiding the audio features of the first non-standard speech sample being single.

[0028] In a possible implementation, the electronic device inputs the audio features of the first standard speech sample, the text features of the first non-standard sample, and the audio features of the random noise into a feature fusion module to obtain the audio features of the first non-standard speech sample.

[0029] In this implementation, the authenticity of the audio features of the obtained first non-standard speech sample can be improved by adding the audio features of the audio random noise to the feature fusion module.

[0030] In a second aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store computer-executable instructions, and the processor being used to run the computer-executable instructions stored in the memory to execute the method described in the first aspect or any possible implementation of the first aspect.

[0031] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0032] In a fourth aspect, an embodiment of the present application provides a computer program product comprising a computer program, which, when executed, enables a computer to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0033] In a fifth aspect, the present application provides a chip or a chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0034] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0035] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1A A schematic diagram of a mobile phone interface change;

[0037] Figure 1B It is a schematic diagram of a scenario where a mobile phone controls an air conditioner;

[0038] Figure 2 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0039] Figure 3 It is a schematic flowchart of a speech recognition method provided by an embodiment of the present application;

[0040] Figure 4 It is a schematic flowchart of another speech recognition method provided by an embodiment of the present application;

[0041] Figure 5 It is a schematic structural diagram of a first speech recognition model provided by an embodiment of the present application;

[0042] Figure 6 It is a schematic flowchart of another speech recognition method provided by an embodiment of the present application;

[0043] Figure 7 It is a schematic structural diagram of an acoustic module provided by an embodiment of the present application;

[0044] Figure 8 It is a schematic flowchart of another speech recognition method provided by an embodiment of the present application;

[0045] Fig. 9 It is a schematic structural diagram of another first speech recognition model provided by an embodiment of the present application. Detailed implementation manners

[0046] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0047] 1. Speech disorder: It can be manifested as disfluency in the speech expression process. Speech disorders can include, but are not limited to, at least one of the following: pronunciation repetition, prolongation of sound, and pauses, etc.

[0048] Among them, pronunciation repetition can be the repetition of a certain character or phoneme. Exemplarily, "kong kong" is the repetition of the character "kong", and "K-K-KONG" is the repetition of the phoneme "K".

[0049] Prolongation of sound refers to prolonging the duration of a certain character or phoneme during pronunciation. Exemplarily, "Open ~~ the air conditioner" is the prolongation of the sound of the character "kai". Among them, "~~" is used to indicate the prolongation of sound.

[0050] A pause refers to suddenly getting stuck during the speaking process and being unable to smoothly pronounce the next character or phoneme, which can be accompanied by filler words (such as "um" or "ah", etc.). Exemplarily, "Open um the air conditioner".

[0051] In some embodiments, the speech disorder may be referred to as stuttering.

[0052] In some embodiments, taking a voice interaction scenario as an example, and when the voice interaction function is turned on in an electronic device, the content of a user's speech, or the content of a verbal expression, or the content of a pronunciation can all be referred to as a voice command.

[0053] Taking the electronic device as a mobile phone as an example, the voice command may include but is not limited to at least one of the following: "turn up the volume", "turn down the volume", "pause", or "put down a song, etc.".

[0054] Taking the electronic device as a vehicle-mounted system as an example, the voice command may include but is not limited to at least one of the following: "open the map", or "turn on the air conditioner", etc.

[0055] In some embodiments, voice commands may be divided into the following types: standard voice commands and non-standard voice commands.

[0056] Among them, standard voice commands refer to voice commands issued when the user has no speech impairment, and non-standard voice commands refer to voice commands issued when the user has speech impairment.

[0057] For example, a standard voice command may be: “Turn on the air conditioner”, or “Turn up the volume”, etc., and a non-standard voice command may be: “Turn on the air conditioner”, or “Turn up the volume”, etc.

[0058] 2. Other terms

[0059] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0060] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0061] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0062] 3. Electronic equipment

[0063] The electronic device of the embodiment of the present application may include a handheld device, a vehicle-mounted device, etc. with a voice recognition function. For example, some electronic devices include: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MID), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication functions, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, terminal devices in 5G networks, or terminal devices in future evolved public land mobile communication networks (PLMNs), etc., and the embodiments of the present application are not limited to this.

[0064] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud server interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as mobile phones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0065] In addition, in the embodiments of the present application, the electronic device may also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0066] The electronic device in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0067] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0068] Speech recognition technology is widely used in various scenarios, which may include but are not limited to any of the following: smart home scenarios, car scenarios, mobile phone voice assistant scenarios, or medical and health scenarios, etc.

[0069] For example, taking the smart home scenario as an example, users can control smart devices in their homes, such as speakers, TVs, or air conditioners, through voice commands, so that they can play music, check the weather, or adjust the temperature, which can improve the portability of users' lives.

[0070] For example, taking the in-vehicle scenario as an example, the vehicle is equipped with a voice control function. When the user is driving the vehicle and the user turns on the voice control function in the vehicle, the user can control the in-vehicle system through voice commands, such as controlling the in-vehicle system to make calls, send messages, or query navigation routes, etc., without the need for manual operation by the user, which can reduce user distraction and thus improve driving safety.

[0071] For example, taking the mobile phone voice assistant scenario, the mobile phone is equipped with a voice control function. When the user turns on the voice control function, the user can control the mobile phone through voice commands, such as setting alarm reminders, playing music, or adjusting the volume, etc., without the user manually touching the screen or buttons, saving time.

[0072] Taking a mobile phone as an example, the mobile phone is equipped with a voice command library and an automatic speech recognition (ASR) model, and the mobile phone is equipped with a voice control function. The voice command library may include standard texts corresponding to multiple standard voice commands. For example, the standard texts corresponding to the standard voice commands may be, for example, "play music", "turn up the volume", etc.

[0073] Exemplarily, in response to a voice command issued by a user (a standard voice command or a non-standard voice command), such as "turn up the volume" or "turn up the volume very high", the ASR model configured in the mobile phone can convert the voice command received from the user into corresponding text.

[0074] The ASR model may include, but is not limited to, at least one of the following: a Gaussian mixture model-hidden Markov model (GMM-HMM), a deep neural network-hidden Markov model, or a convolution neural network model.

[0075] It should be understood that the ASR model configured in the mobile phone is a pre-trained ASR model. In addition, this application does not elaborate on the structure of the above-mentioned ASR model, and reference can be made to the description in the current prior art.

[0076] In a possible scenario, a user uses a mobile phone to play a TV series, and the user controls the mobile phone to adjust the volume through voice commands. Figure 1A This is a schematic diagram of the changes in a mobile phone interface, refer to Figure 1A In (a), the interface 10 is in response to a voice command input by the user, such as "turn up the volume", and the mobile phone can display a page related to the voice control function, such as the mobile phone displays a page for waking up the voice control function. Figure 1A (a) may include an icon 101 for waking up the voice control function. The voice command input by the user may be understood as: the user says to the microphone of the mobile phone: "Turn up the volume".

[0077] Accordingly, the microphone of the mobile phone can capture the voice command "turn up the volume" from the user, and the mobile phone can use the preset ASR model to convert the voice command "turn up the volume" into corresponding text, referring to Figure 1A In (b), the mobile phone can display interface 11, which may include text corresponding to the voice command, such as "turn up the volume". In addition, the mobile phone can search for the text corresponding to the voice command in a preset voice command library. Since the voice command "turn up the volume" is a standard voice command, there is a standard text in the voice command library that is the same as the text. Therefore, the mobile phone can execute the standard voice command corresponding to the standard text, such as the mobile phone can control the speaker to increase the volume by 10 decibels (DB), and refer to Figure 1A In (c), the mobile phone may display an interface 12, which may include text corresponding to: "The volume has been increased by 10DB".

[0078] In this scenario, when the user has a speech disorder, for example, the user inputs a voice command of "turn up the volume". After the microphone of the mobile phone captures the voice command "turn up the volume" from the user, the mobile phone can convert the voice command "turn up the volume" into corresponding text through the preset ASR model, and the mobile phone can search for the text corresponding to the voice command "turn up the volume" in the preset voice command library. Since the voice command "turn up the volume" is a non-standard voice command, there is no standard text in the voice command library that is the same as the text, so the mobile phone cannot recognize the voice command "turn up the volume", and the mobile phone cannot execute the voice command "turn up the volume", that is, the mobile phone will not control the speaker to increase the volume by 10DB.

[0079] In some embodiments, in order to solve the problem that when a user has a speech disorder such as repeated pronunciation, the electronic device is unable to recognize non-standard voice commands from the user and is therefore unable to execute the non-standard voice commands, after the electronic device converts the non-standard voice command into corresponding text through an ASR model, the electronic device will delete a portion of the text. For example, the electronic device may retain one of multiple repeated words or phonemes in the text corresponding to the non-standard voice command, and delete the remaining repeated words or phonemes to obtain the deleted text.

[0080] Since the deleted text is the same as a standard text in a preset voice instruction library of the electronic device, the electronic device can recognize the non-standard voice instruction, and the electronic device can execute the voice instruction corresponding to the deleted text.

[0081] For example, when a user has a speech disorder with repeated pronunciation, the non-standard voice command is "Turn up the volume". After the electronic device converts the non-standard voice command "Turn up the volume" into corresponding text through the ASR model, the electronic device can retain one "big" character in the text "Turn up the volume" corresponding to the non-standard voice command, and delete the remaining "big" characters to obtain the deleted text "Turn up the volume". Since the standard text "Turn up the volume" in the voice command library is the same as the deleted text "Turn up the volume", the electronic device can recognize the non-standard voice command, and the electronic device can control the speaker to increase the volume by 10DB.

[0082] However, when the user wakes up the voice control function of the electronic device through a voice command, for example, the voice command used by the user to wake up the voice control function of the electronic device is "Hello Youyou", in response to the voice command "Hello Youyou" from the user, the electronic device deletes a part of the text after converting the voice command into corresponding text, and obtains the deleted text "Hello You". Since the text corresponding to the voice command for waking up the electronic device in the preset voice command library is "Hello Youyou", and the text "Hello Youyou" is different from "Hello You", the electronic device cannot recognize the voice command "Hello Youyou" from the user, and thus cannot wake up the voice control function of the electronic device.

[0083] It should be understood that the voice command "Hello Youyou" used by the user to wake up the voice control function of the electronic device is a standard voice command. However, since the text converted from the command contains repeated words "You", the electronic device will process the text as text corresponding to a non-standard voice command, resulting in the electronic device being unable to recognize the voice command "Hello Youyou".

[0084] In addition, in this method, the electronic device is also unable to recognize the user's voice commands (non-standard voice commands) when there are speech disorders such as dragging sounds and pauses.

[0085] In order to solve the problem that the electronic device cannot recognize the non-standard voice commands when the user has speech disorders, in some embodiments, a pre-trained ASR model and a pre-trained natural language understanding (NLU) model can be pre-deployed in the cloud server.

[0086] Among them, the cloud server refers to a server built based on cloud computing technology, which can provide remote computing and storage services. When an electronic device is connected to the Internet, the electronic device can access the cloud server to achieve remote computing and storage. For example, the electronic device can upload a voice command to the cloud server, and the cloud server can process the voice command, obtain a processing result, and return the processing result to the electronic device.

[0087] It should be understood that the embodiments of the present application do not elaborate on the process of ASR model and NLU model training, and reference may be made to the description in the current prior art.

[0088] When the electronic device is connected to the Internet (network), in response to the user inputting a non-standard voice command, the microphone of the electronic device can capture the non-standard voice command from the user and upload the non-standard voice command to the cloud server. In response to the non-standard voice command from the electronic device, the cloud server can convert the non-standard voice command into the corresponding text (non-standard text) through the pre-trained ASR model. After obtaining the text corresponding to the non-standard voice command, the cloud server can analyze the text corresponding to the non-standard voice command through the pre-trained NLU model to obtain the user's true intention or demand. Among them, the true intention or demand is the standard text corresponding to the non-standard voice command.

[0089] After obtaining the user's true intention or demand, the cloud server can send the true intention or demand to the electronic device. Accordingly, in response to the user's true intention or demand from the cloud server, the electronic device can search in the preset voice command library. Since the standard text is stored in the preset voice command library of the electronic device, the electronic device can recognize non-standard voice commands, and the electronic device can execute the standard voice commands corresponding to the standard text. Therefore, the problem that the electronic device cannot recognize the voice commands when the user has a speech disorder can be solved.

[0090] For example, Figure 1B This is a schematic diagram of a scenario where a mobile phone controls an air conditioner. Figure 1B The scene may include: a mobile phone, an air conditioner, and a cloud server. The air conditioner may be an air conditioner with a Bluetooth function. When a Bluetooth connection is established between the mobile phone and the air conditioner, the mobile phone may control the air conditioner to turn on or off.

[0091] In this scenario, in response to a non-standard voice command from the user, such as "turn on the air conditioner", the microphone of the mobile phone can capture the non-standard voice command from the user, and the mobile phone can send the non-standard voice command "turn on the air conditioner" to the cloud server. The cloud server uses the ASR model to convert the non-standard voice command into corresponding text (non-standard text), and uses the text as the input of the NLU model. The NLU model obtains the user's real intention or demand as "turn on the air conditioner". The cloud server can send the real intention or demand "turn on the air conditioner" to the mobile phone.

[0092] Accordingly, the mobile phone can receive the real intention or demand "turn on the air conditioner" from the cloud server, and since "turn on the air conditioner" is a standard text, the mobile phone can find the standard text "turn on the air conditioner" in the preset voice command library, and the mobile phone can send the command "turn on the air conditioner" to the air conditioner. Accordingly, when the air conditioner receives the command "turn on the air conditioner" from the mobile phone, the air conditioner can control the air conditioner to turn on.

[0093] In some embodiments, reference Figure 1B , after the air conditioner is controlled to turn on, the air conditioner can send a "turn on successfully" message to the mobile phone.

[0094] However, in this example, the pre-trained ASR model and the pre-trained NLU model need to be deployed in the cloud server in advance, and when the electronic device receives a non-standard voice command from the user, the electronic device needs to interact with the cloud server to obtain the user's true intention or demand, which has the problem of low response efficiency. For example, when the network signal is unstable, it takes a long time for the electronic device to obtain the user's true intention or demand from the cloud server, affecting the user experience.

[0095] In addition, every time an electronic device receives a non-standard voice command from a user, it needs to call the ASR model and NLU model in the cloud server. The operation of the ASR model and NLU model requires a large amount of computing resources, which will not only increase the power consumption of the cloud server, but also accelerate the power consumption of the electronic device.

[0096] If the purpose is to reduce the latency of interaction between electronic devices and cloud servers, pre-trained ASR models and pre-trained NLU models are directly deployed in electronic devices. However, it is difficult to deploy ASR models and NLU models in electronic devices because the operation of ASR models and NLU models consumes a lot of computing resources and the data volume of ASR models and NLU models is large (such as tens of megabytes), while the storage space and computing resources of electronic devices are limited.

[0097] Accordingly, the embodiment of the present application provides a speech recognition method, in which a first speech recognition model can be pre-set in the electronic device (the first speech recognition model can refer to the description in the following embodiment), and the electronic device can use the feature extraction module and the acoustic module in the first speech recognition model to pre-generate acoustic features of multiple non-standard voice instructions, and store the acoustic features of the multiple non-standard voice instructions and the standard text corresponding to the acoustic features of each non-standard voice sample in a feature database. When the user inputs a non-standard voice instruction, the electronic device can use the first speech recognition model to obtain the acoustic features of the non-standard voice instruction, and match them with the acoustic features of multiple non-standard voice samples in the feature database to obtain the target acoustic features and the standard text corresponding to the target acoustic features, so that the electronic device can execute the voice instruction corresponding to the standard text.

[0098] In this method, since the electronic device does not need to interact with the cloud server, the latency can be reduced, thereby improving the recognition efficiency of the electronic device. Moreover, since the electronic device can perform matching based on the acoustic features of multiple pre-generated non-standard voice samples, the computing resources of the electronic device can be saved, thereby reducing the power consumption of the electronic device and improving the user experience.

[0099] Before introducing the speech recognition method provided in the embodiment of the present application, the structure of the electronic device provided in the embodiment of the present application is first introduced:

[0100] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, referring to Figure 2 The electronic device 200 may include a processor 210, an audio module 211, a speaker 211A, a receiver 211B, a microphone 211C, an earphone interface 211D, a sensor 212, a display screen 213, an internal memory 214, a wireless communication module 215, and an antenna 1. It is understood that the structure shown in this embodiment does not constitute a specific limitation on the electronic device 200.

[0101] In the embodiment of the present application, the processor 210 can be used to execute the speech recognition method provided in the embodiment of the present application.

[0102] Microphone 211C can be used to capture non-standard voice commands from the user.

[0103] The display screen 213 can be used to display an interface. For example, when a user wakes up the voice control function of a mobile phone, the display screen 213 can display a wake-up interface.

[0104] The internal memory 214 can be used to store relevant instructions so as to implement the speech recognition method provided in the embodiment of the present application when the processor 210 calls the relevant instructions.

[0105] In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0106] The speech recognition method provided by the embodiments of the present application is described below in conjunction with specific embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0107] The following describes the speech recognition method provided by the embodiment of the present application. Figure 3 A flow chart of a speech recognition method provided in an embodiment of the present application. Figure 3 , the speech recognition method provided in the embodiment of the present application may include:

[0108] S301, the electronic device obtains a first standard voice sample, a first standard text corresponding to the first standard voice sample, and a first non-standard text corresponding to the first standard voice sample.

[0109] The first standard voice sample may be a standard voice instruction, the first standard text may be a text corresponding to the standard voice instruction, and the first non-standard text may be a text corresponding to the non-standard voice instruction.

[0110] Exemplarily, the first standard voice sample may be the audio corresponding to "turn up the volume", the first standard text corresponding to the first standard voice sample may be the text corresponding to "turn up the volume", and the first non-standard text corresponding to the first standard voice sample may be the text corresponding to "turn up the volume", or the text corresponding to "turn up the volume very high", or the text corresponding to "turn up the volume~", etc.

[0111] In some embodiments, when the electronic device is started for the first time, or when the user registers the voice control function in the electronic device for the first time, the electronic device can obtain the first standard text corresponding to the first standard voice sample from the preset voice instruction library. Exemplarily, the first standard text corresponding to the first standard voice sample can be: "Turn up the volume".

[0112] In this example, when a user registers the voice control function in an electronic device for the first time, the electronic device can record the user's voiceprint and information about the user's surrounding environment.

[0113] Among them, voiceprint is the specific voice feature of each user. The user-specific voice features may include pronunciation habits, timbre, intonation, or speaking speed. Voiceprint can be used to identify users like fingerprints. By recording the user's voiceprint, the electronic device can distinguish the voice of the user who has registered the voice control function from the voices of other users.

[0114] In some embodiments, a text-to-speech (TTS) model is pre-set in the electronic device, and the TTS model is used to convert the first standard text into a first standard voice sample that conforms to the user's voiceprint and surrounding environment information.

[0115] Exemplarily, the electronic device may input the first standard text, the recorded voiceprint of the user, and the recorded surrounding environment information into the TTS model, and the output of the TTS model is a first standard voice sample that conforms to the user's voiceprint and the surrounding environment information.

[0116] In some embodiments, the electronic device may add text to the first standard text based on a preset rule to obtain a first non-standard text. Each rule may correspond to a different type of speech disorder. Exemplarily, the first standard text is the text corresponding to "turn up the volume", and the first non-standard text may be the text corresponding to "turn up the volume", or the text corresponding to "turn up the volume", or the text corresponding to "turn up the volume".

[0117] S302, the electronic device simultaneously inputs a first standard voice sample and a first non-standard text into a non-standard voice generation model to obtain a first non-standard voice sample.

[0118] Among them, the non-standard speech generation model is used to generate non-standard speech samples.

[0119] In some embodiments, reference Figure 4 The non-standard speech generation model may include an audio encoding module, a text encoding module, a feature fusion module, and an audio decoding module. The audio encoding module is used to generate audio features of the first standard speech sample, the text encoding module is used to generate text features of the first non-standard text, the feature fusion module is used to generate audio features of the first non-standard speech sample, and the audio decoding module is used to generate the first non-standard speech sample.

[0120] In this example, the output ends of the audio encoding module and the text encoding module are both connected to the input end of the feature fusion module, and the output end of the feature fusion module is connected to the input end of the audio decoding module.

[0121] For example, Figure 4 As shown, the input of the audio encoding module is a first standard voice sample, such as the audio corresponding to "turn up the volume", the input of the text encoding module is a first non-standard text, such as the text corresponding to "turn up the volume", and the output of the audio decoding module is a first non-standard voice sample, such as the audio corresponding to "turn up the volume".

[0122] In some embodiments, the audio encoding module may include: a variational auto-encoder (VAE), which can capture the audio features of a first standard speech sample by learning the distribution of audio data. The audio features of the first standard speech sample can be represented by a vector in a latent space.

[0123] Reference Figure 4 , the audio features of the first standard speech sample can be represented by vector 1, the text features of the first non-standard text can be represented by vector 2, and the audio features of the first non-standard speech sample can be represented by vector 3.

[0124] In some embodiments, the feature fusion module may include a pre-trained generative model, which is a machine learning model that aims to learn the underlying distribution patterns of the original data and generate new data samples that are similar to the original data.

[0125] The pre-trained generative model may include but is not limited to any one of the following: a generative adversarial network (GAN) model, a diffusion model, etc. The original data may include: text features of the first non-standard text and audio features of the first standard voice sample, and the new data sample may include: audio features of the first non-standard voice sample.

[0126] Taking the generative model in the feature fusion module as a diffusion model as an example, in some embodiments, referring to Figure 4 The non-standard speech generation model may further include: a random variable generation module, which may be used to generate audio features of random noise.

[0127] like Figure 4 As shown, the output end of the random variable generation module is connected to the input end of the feature fusion module. Exemplarily, the random variable generation module can sample from a standard normal distribution to obtain the audio features of random noise.

[0128] It should be understood that since there are multiple first non-standard texts, the electronic device can obtain a first non-standard voice sample by inputting each first non-standard text and the first standard voice sample into the non-standard voice generation model at the same time.

[0129] Exemplarily, the electronic device simultaneously inputs the text corresponding to the first non-standard text such as "Turn up the volume~~" and the audio corresponding to the first standard voice sample such as "Turn up the volume" into the non-standard voice generation model, and the audio corresponding to the first non-standard voice sample such as "Turn up the volume~~" can be obtained. Exemplarily, the electronic device simultaneously inputs the text corresponding to the first non-standard text such as "Turn up the volume uh", and the audio corresponding to the first standard voice sample such as "Turn up the volume" into the non-standard voice generation model, and the audio corresponding to the first non-standard voice sample such as "Turn up the volume uh" can be obtained.

[0130] S303, the electronic device inputs the first non-standard speech sample into the first speech recognition model, and obtains the acoustic features of the first non-standard speech sample after being processed by the feature extraction module and the acoustic module in the first speech recognition model.

[0131] The first speech recognition model is a lightweight speech recognition model preset in the electronic device for recognizing non-standard speech data. Exemplarily, the non-standard speech data may be a first non-standard speech sample.

[0132] It is understandable that, unlike the ASR model, the first speech recognition model does not include a language module and a decoding module. In the ASR model, the output end of the acoustic module is connected to the input end of the language module, and the output end of the language module is connected to the input end of the decoding module. Through the language module and the decoding module, the first non-standard text corresponding to the first non-standard speech sample can be obtained.

[0133] like Figure 5 As shown, the first speech recognition model may include: a feature extraction module and an acoustic module, and the output end of the feature extraction module is connected to the input end of the acoustic module.

[0134] The feature extraction module is used to extract the audio features of the first non-standard voice sample, and the audio features of the first non-standard voice sample may include: time domain information and frequency domain information of the first non-standard voice sample;

[0135] Exemplarily, the time domain information may be, for example, the amplitude envelope, kurtosis, or skewness of the first non-standard speech sample, and the frequency domain information may be, for example, the spectrum or spectrum energy of the first non-standard speech sample.

[0136] The acoustic model is used to map the audio features of the first non-standard speech sample from a high-dimensional space to a low-dimensional space to obtain the acoustic features of the first non-standard speech sample, and the acoustic features are used to indicate the semantic information of the first non-standard speech sample.

[0137] It can be understood that, generally, the dimension of the audio feature of the first non-standard speech sample is higher than the dimension of the acoustic feature.

[0138] For example, refer to Figure 6 The input of the feature extraction module is the first non-standard speech sample, such as the audio corresponding to “turn up the volume”, and the output of the acoustic module is the acoustic feature of the audio corresponding to “turn up the volume”.

[0139] In some embodiments, taking the frequency domain information as spectrum energy as an example, the feature extraction module may include a feature extraction algorithm, and the feature extraction algorithm may be, for example, mel-frequency cepstral coefficients (MFCC), or a filter bank (Fbank), etc. The electronic device may use the feature extraction algorithm to obtain the spectrum energy of the first non-standard speech sample.

[0140] Among them, MFCC means dividing the spectrum of the first non-standard speech sample into multiple bandpass filters on the Mel scale, and then calculating the output energy of each filter, so as to obtain the spectrum energy distribution of the first non-standard speech sample on the Mel scale, and the energy distribution can be the spectrum energy of the first non-standard speech sample.

[0141] Fbank refers to passing the first non-standard speech sample through a set of bandpass filters, each filter corresponding to a frequency range, and then calculating the output energy of each filter to obtain the energy distribution of the first non-standard speech sample in the frequency domain, which energy distribution can be the spectral energy of the first non-standard speech sample.

[0142] It should be understood that the process by which the electronic device obtains the spectral energy of the first non-standard speech sample in the above example does not constitute a limitation of the present application. In some embodiments, the electronic device may obtain the spectral energy of the first non-standard speech sample using methods other than MFCC and Fbank, which is not elaborated in the embodiments of the present application.

[0143] In some embodiments, the acoustic module may include a pre-trained acoustic model, and the electronic device may use the output of the last layer in the acoustic model as the acoustic feature of the first non-standard speech sample, or the electronic device may use the output of the second-to-last layer in the acoustic model as the acoustic feature of the first non-standard speech sample. This embodiment of the present application is not limited to this.

[0144] For example, the pre-trained acoustic model is a zipformer model. Figure 7 It is a structural diagram of the zipformer model, such as Figure 7As shown, the zipformer model may include a feed-forward layer, a multi-head attention layer, a non-linear attention layer, a self-attention layer, a convolution layer, a bypass, and a normalization (biasnorm). Each module may refer to the relevant description in the existing zipformer model.

[0145] Among them, the feedforward layer is composed of a fully connected layer, which can make nonlinear changes to the input data; the multi-head attention layer can calculate the semantic information of different subspaces in parallel through multiple attention heads, obtain multiple attention weights, and distribute the multiple attention weights to the nonlinear attention layer and the self-attention layer respectively; the nonlinear attention layer is used to introduce nonlinear transformations; the self-attention layer is used to calculate the dependencies within the input data; the convolutional layer is used to capture details; the bypass is used to avoid gradient disappearance; and normalization is used to accelerate the convergence of the zipformer model.

[0146] It should be understood that bypass and biasnorm are not layers in the zipformer model.

[0147] Reference Figure 7 The input of the zipformer model is the audio features of the first non-standard speech sample. The electronic device can use the output of the last feed-forward layer "Feed-forword" as the acoustic features of the first non-standard speech sample, or use the output of the penultimate convolution layer "Convolution" as the acoustic features of the first non-standard speech sample.

[0148] In some embodiments, the acoustic feature of the first non-standard speech sample may be referred to as a matching embedding, or an acoustic embedding, or an embedding vector.

[0149] It should be understood that the embodiment of the present application does not elaborate on the process of the electronic device generating the acoustic features of other non-standard voice samples, and can refer to the description of the electronic device generating the acoustic features of the first non-standard voice sample.

[0150] S304, the electronic device stores the acoustic features of the first non-standard voice sample and the first standard text as a group of data in a feature database.

[0151] Since different non-standard speech samples may correspond to different acoustic features, the feature database may store acoustic features of multiple different non-standard speech samples and the standard text corresponding to the acoustic feature of each non-standard speech sample.

[0152] In the embodiments of the present application, an electronic device may obtain a first standard voice sample, a first standard text corresponding to the first standard voice sample, and a first non-standard text corresponding to the first standard voice sample. The electronic device may use a non-standard voice generation model to obtain a first non-standard voice sample, and input the first non-standard voice sample into a first speech recognition model to obtain the acoustic features of the first non-standard voice sample. The electronic device may store the acoustic features of the first non-standard voice sample and the first standard text as a set of data in a feature database. In this method, since the feature database of the electronic device pre-stores the acoustic features of multiple different non-standard voice samples and the standard text corresponding to the acoustic features of each non-standard voice sample, the feature database can be used for subsequent matching, so that the electronic device can recognize non-standard voice commands (voice data) from the user and improve the recognition efficiency.

[0153] In the above embodiments, the process of the electronic device constructing the feature database is introduced. Next, the process of recognizing non-standard voice commands (voice data) from the user based on the constructed feature database is introduced. Figure 8 It is a schematic flowchart of another speech recognition method provided by the embodiments of the present application. Refer to Figure 8 , the process of the speech recognition method provided by the embodiments of the present application may include:

[0154] S801, the electronic device receives first voice data, and the first voice data includes non-standard voice data.

[0155] In some embodiments, the electronic device may receive first voice data (first voice command) from the user through a microphone. Among them, the first voice data may include, but is not limited to, at least one of the following: repeated syllables, prolonged syllables, or pausing syllables.

[0156] Exemplarily, the first voice data may be "turn up the volume", and the syllable "big" in the first voice data is a repeated syllable; exemplarily, the first voice data may be "turn up ~~ the volume", and the syllable "big" in the first voice data is a prolonged syllable; exemplarily, the first voice data may be "turn up uh the volume", and the "uh" in the first voice data is a pausing syllable.

[0157] It can be understood that a syllable may be a certain character or phoneme in the first voice data.

[0158] S802, the electronic device inputs the first voice data into the first speech recognition model to obtain the standard text corresponding to the first voice data.

[0159] Such as Fig. 9As shown, the first speech recognition model may also include: a feature matching module, the input end of the feature matching module is connected to the output end of the acoustic module, and the feature matching module is used to obtain the standard text corresponding to the first speech data. The first speech recognition model is a lightweight speech recognition model preset in the electronic device for recognizing non-standard speech data.

[0160] In some embodiments, the electronic device can input the first voice data into a feature extraction module in a first voice recognition model. After processing by the feature extraction module, a first audio feature can be obtained. The first audio feature can include time domain information and frequency domain information of the first voice data.

[0161] The process of the electronic device acquiring the first audio feature of the first voice data may refer to the description of the electronic device acquiring the audio feature of the first non-standard voice sample in S803.

[0162] In some embodiments, the electronic device may input the first audio feature into an acoustic module in a first speech recognition model, and obtain a first acoustic feature after processing by the acoustic module, where the first acoustic feature is used to indicate semantic information of the first speech data.

[0163] The process of the electronic device acquiring the first audio feature of the first voice data may refer to the description of the electronic device acquiring the acoustic feature of the first non-standard voice sample in S803.

[0164] It should be understood that, generally, the dimension of the first audio feature is higher than the dimension of the first acoustic feature.

[0165] In some embodiments, the feature matching module may include a preset feature database. The electronic device may input the first acoustic feature into the feature matching module in the first speech recognition model, and obtain a target acoustic feature whose similarity with the first acoustic feature is greater than a similarity threshold from the feature database by traversing the feature database. The electronic device may use the standard text corresponding to the target acoustic feature as the standard text corresponding to the first speech data.

[0166] In some embodiments, a similarity calculation method can be preset in the feature matching module. The electronic device can use the similarity calculation method to calculate the similarity between the acoustic features of multiple non-standard speech samples in the feature database and the first acoustic feature to obtain multiple similarity values. The electronic device can compare the maximum similarity value among the multiple similarity values ​​with the similarity threshold. If the maximum similarity value among the multiple similarity values ​​is greater than the similarity threshold, the electronic device can use the acoustic feature of the non-standard speech sample corresponding to the maximum similarity value as the target acoustic feature.

[0167] The similarity calculation method may include but is not limited to any one of the following: cosine similarity, or Euclidean distance, etc. It should be understood that the above similarity calculation method is not described in detail in the present embodiment, and reference may be made to the description in the current prior art.

[0168] Exemplarily, the feature database may include: acoustic feature 1, acoustic feature 2, and acoustic feature 3, etc. The electronic device may calculate the similarity between the multiple acoustic features and the first acoustic feature to obtain the corresponding similarity value 1, similarity value 2, and similarity value 3. Among them, similarity value 1 is the similarity between acoustic feature 1 and the first acoustic feature, similarity value 2 is the similarity between acoustic feature 2 and the first acoustic feature, and similarity value 3 is the similarity between acoustic feature 3 and the first acoustic feature.

[0169] In this example, if the similarity value 1 is the largest, and if the similarity value 1 is greater than the similarity threshold, the electronic device may use the acoustic feature 1 as the target acoustic feature.

[0170] After obtaining the target acoustic feature, the electronic device may use the standard text corresponding to the target acoustic feature in the feature database as the standard text corresponding to the first voice data.

[0171] S803, the electronic device executes the instruction corresponding to the standard text.

[0172] For example, the standard text is: "Turn up the volume", and the corresponding instruction may be: increase the volume by 10DB.

[0173] In an embodiment of the present application, when the electronic device receives the first voice data, the electronic device can input the first voice data into the first voice recognition model to obtain the standard text corresponding to the first voice data, and the electronic device can execute the instruction corresponding to the standard text. On the one hand, since the electronic device can directly obtain the standard text corresponding to the first voice data without interacting with the cloud server when receiving the first voice data, the time for the electronic device to process the first voice data can be saved, thereby improving the recognition efficiency. On the other hand, the resources consumed by the electronic device to run the first voice recognition model are far less than the resources consumed by running the ASR model and the NLU model at the same time, which can save the power consumption of the electronic device.

[0174] In some embodiments, taking the electronic device as a mobile phone as an example, a non-standard voice data recognition function may be integrated into the setting application of the mobile phone, and the non-standard voice data recognition function may include a first voice recognition model. When the user turns on the non-standard voice data recognition function, the electronic device may recognize the non-standard voice data (non-standard voice instructions) from the user based on a preset feature database; when the user does not turn on the non-standard voice recognition function, the electronic device may recognize the standard voice data (standard voice instructions) from the user based on a preset voice instruction library.

[0175] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0177] The speech recognition method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above list sorting method.

[0178] The speech recognition method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal devices can refer to the above related descriptions, which will not be repeated here.

[0179] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0180] The embodiment of the present application provides a chip or a chip system. The chip or chip system includes at least one processor and a communication interface, the communication interface is interconnected with at least one processor through a line, and at least one processor is used to run a computer program or instruction to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above-mentioned related embodiments, and will not be repeated here. Among them, the communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0181] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0182] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0183] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0184] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0185] The above specific implementation methods further explain in detail the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above are only specific implementation methods of the embodiments of the present application and are not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A speech recognition method, characterized in that: include: The electronic device receives first voice data, wherein the first voice data includes non-standard voice data; The electronic device inputs the first voice data into a first voice recognition model to obtain a standard text corresponding to the first voice data; the first voice recognition model is a lightweight voice recognition model preset in the electronic device for recognizing non-standard voice data; Execute the instructions corresponding to the standard text.

2. The method according to claim 1, characterized in that The first speech recognition model includes a feature extraction module, an acoustic module and a feature matching module connected in sequence; The feature extraction module is used to extract audio features of speech data; The acoustic module is used to extract acoustic features of speech data; The feature matching module includes a preset feature database, which includes acoustic features of multiple non-standard speech samples and a standard text corresponding to the acoustic feature of each non-standard speech sample.

3. The method according to claim 1 or 2, characterized in that: The electronic device inputs the first voice data into a first voice recognition model to obtain a standard text corresponding to the first voice data, including: The electronic device inputs the first voice data into a feature extraction module in the first voice recognition model, and obtains a first audio feature after processing by the feature extraction module; the first audio feature is used to indicate time domain information and frequency domain information of the first voice data; The electronic device inputs the first audio feature into the acoustic module in the first speech recognition model, and obtains a first acoustic feature after being processed by the acoustic module; the first acoustic feature is used to indicate semantic information of the first speech data; The electronic device inputs the first acoustic feature into a feature matching module in the first speech recognition model, and obtains a standard text corresponding to the first speech data after being processed by the feature matching module.

4. The method according to claim 3, characterized in that The electronic device inputs the first acoustic feature into a feature matching module in the first speech recognition model, and after being processed by the feature matching module, obtains a standard text corresponding to the first speech data, including: The electronic device traverses a feature database through the feature matching module, and obtains a target acoustic feature whose similarity with the first acoustic feature is greater than a similarity threshold from the feature database; The electronic device uses the standard text corresponding to the target acoustic feature as the standard text corresponding to the first voice data.

5. The method according to claim 1, characterized in that The first voice data includes at least one of a repeated syllable, a dragged syllable or a paused syllable.

6. The method according to claim 1, wherein the electronic device further comprises a non-standard speech generation model, and the method further comprises: The electronic device acquires a first standard voice sample, a first standard text corresponding to the first standard voice sample, and a first non-standard text corresponding to the first standard voice sample; The electronic device simultaneously inputs the first standard voice sample and the first non-standard text into the non-standard voice generation model to obtain a first non-standard voice sample; The electronic device inputs the first non-standard voice sample into the first voice recognition model, and obtains acoustic features of the first non-standard voice sample after being processed by the feature extraction module and the acoustic module in the first voice recognition model; The electronic device stores the acoustic features of the first non-standard speech sample and the first standard text as a set of data in a feature database; Based on the above steps, other groups of data in the feature database are constructed.

7. The method according to claim 6, characterized in that The non-standard speech generation model includes an audio encoding module, a text encoding module, a feature fusion module and an audio decoding module; The output ends of the audio encoding module and the text encoding module are both connected to the input end of the feature fusion module, and the output end of the feature fusion module is connected to the input end of the audio decoding module.

8. The method according to claim 6 or 7, characterized in that: The electronic device simultaneously inputs the first standard voice sample and the first non-standard text into the non-standard voice generation model to obtain the first non-standard voice sample, including: The electronic device inputs the first standard voice sample into the audio encoding module of the non-standard voice generation model to obtain audio features of the first standard voice sample; and inputs the first non-standard text into the text encoding module of the non-standard voice generation model to obtain text features of the first non-standard text; The electronic device simultaneously inputs the audio features of the first standard speech sample and the text features of the first non-standard sample into the feature fusion module of the non-standard speech generation model to obtain the audio features of the first non-standard speech sample; The electronic device inputs the audio features of the first non-standard speech sample into the audio decoding module of the non-standard speech generation model to obtain the first non-standard speech sample.

9. The method according to claim 7, characterized in that: The non-standard speech generation model further includes a random variable generation module; the random variable generation module is connected to the input end of the feature fusion module; The random variable generation module is used to generate audio features of random noise.

10. The method according to claim 9, characterized in that The electronic device simultaneously inputs the audio features of the first standard speech sample and the text features of the first non-standard sample into the feature fusion module of the non-standard speech generation model to obtain the audio features of the first non-standard speech sample, including: The electronic device inputs the audio features of the first standard voice sample, the text features of the first non-standard sample, and the audio features of the random noise into the feature fusion module to obtain the audio features of the first non-standard voice sample.

11. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 10.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. A chip system, characterized in that: The method comprises at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is used to run a computer program or instruction to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 10.

Images