Speech recognition method, electronic device, storage medium and chip system

Abstract

The embodiments of the present application provide a speech recognition method, electronic device, storage medium and chip system, which relate to the field of terminal technology. In this method, a first speech recognition model can be pre-installed in the electronic device, and the first speech recognition model can be a lightweight speech recognition model for recognizing non-standard voice data. When the electronic device receives the first voice data, the electronic device can input the first voice data into the first speech recognition model to obtain the standard text corresponding to the first voice data, and the electronic device can execute the instructions corresponding to the standard text. Since the electronic device does not need to interact with the cloud server, it can directly obtain the standard text corresponding to the first voice data, which can reduce latency and improve user experience. In addition, since the first speech recognition model is a lightweight speech recognition model, it is easy to deploy in electronic devices.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a speech recognition method, electronic device, storage medium, and chip system. Background Art

[0002] With the rapid development of science and technology, voice recognition technology has become an integral part of our lives. For example, users can now control electronic devices through voice commands, eliminating the need for manual operation. This simplifies the interaction process between users and electronic devices and improves interaction efficiency.

[0003] However, due to the user's pronunciation habits, such as repeated, prolonged or paused pronunciation, the non-standard voice commands received by the electronic device from the user do not match the standard voice commands in the voice command library preset by the electronic device. Therefore, the electronic device cannot recognize the non-standard voice commands issued by the user, affecting the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a voice recognition method, electronic device, storage medium, and chip system, which are applied to the field of terminal technology and can enable electronic devices to recognize non-standard voice commands issued by users and improve user experience.

[0005] In the first aspect, an embodiment of the present application proposes a speech recognition method, and the subject executing the method may be an electronic device or a chip in an electronic device. The following description will be made using an electronic device as an example. The method may include: the electronic device receives first voice data, the first voice data may include non-standard voice data, and the electronic device may input the first voice data into a first voice recognition model to obtain a standard text corresponding to the first voice data. The first voice recognition model is a lightweight voice recognition model preset in the electronic device for recognizing non-standard voice data. After obtaining the standard text corresponding to the first voice data, the electronic device may execute the instruction corresponding to the standard text.

[0006] It should be understood that the first voice data may be a voice instruction from the user, such as the audio corresponding to “turn up the volume”.

[0007] In the embodiment of the present application, since the first speech recognition model is a lightweight speech recognition model, it is easy to deploy in an electronic device. Moreover, since the electronic device can obtain the standard text corresponding to the first speech data based on the first speech recognition model without interacting with the cloud server, the latency can be reduced, thereby improving the recognition efficiency of the electronic device and enhancing the user experience.

[0008] In one possible implementation, the first speech recognition model includes a feature extraction module, an acoustic module, and a feature matching module that are connected in sequence; wherein the feature extraction module can be used to extract audio features of speech data, the acoustic module can be used to extract acoustic features of speech data, and the feature matching module can include a preset feature database, which includes acoustic features of multiple non-standard speech samples and standard text corresponding to the acoustic features of each non-standard speech sample.

[0009] The feature extraction module, acoustic module, and feature matching module connected in sequence can be understood as follows: the output end of the feature extraction module is connected to the input end of the acoustic module, and the output end of the acoustic module is connected to the input end of the feature matching module.

[0010] In other words, the output of the feature extraction module is the input of the acoustic module, and the output of the acoustic module is the input of the feature matching module.

[0011] In this implementation, since a standard text in the feature data can correspond to the acoustic features of multiple non-standard voice samples, the electronic device can obtain the standard text corresponding to the acoustic features of the non-standard voice samples based on the acoustic features of the non-standard voice samples, and execute the instructions corresponding to the standard text.

[0012] In one possible implementation, the electronic device may input the first voice data into the feature extraction module in the first voice recognition model, and obtain the first audio feature after processing by the feature extraction module. The first audio feature may include time domain information and frequency domain information of the first voice data. After obtaining the first audio feature, the electronic device may input the first audio feature into the acoustic module in the first voice recognition model, and obtain the first acoustic feature after processing by the acoustic module. The first acoustic feature is used to indicate the semantic information of the first voice data. After obtaining the first acoustic feature, the electronic device may input the first acoustic feature into the feature matching module in the first voice recognition model, and obtain the standard text corresponding to the first voice data after processing by the feature matching module.

[0013] Exemplarily, the first audio feature may be spectrum energy.

[0014] In this implementation, the electronic device processes the first voice data in stages (feature extraction module-acoustic module-feature matching module), and finally maps the first voice data to standard text. Since the electronic device can recognize the standard text, the electronic device can execute the instructions corresponding to the standard text.

[0015] In one possible implementation, the electronic device may traverse a feature database using a feature matching module to obtain a target acoustic feature from the feature database whose similarity to the first acoustic feature is greater than a similarity threshold. After obtaining the target acoustic feature, the electronic device may use the standard text corresponding to the target acoustic feature as the standard text corresponding to the first speech data.

[0016] Exemplarily, the feature database contains acoustic features of multiple non-standard speech samples. The electronic device can calculate the similarity between the acoustic feature of each non-standard speech sample and the first acoustic feature, thereby determining multiple similarities. The electronic device can select the largest similarity among the multiple similarities and determine whether the largest similarity is greater than the similarity threshold. If it is greater than the similarity threshold, the electronic device can use the acoustic feature of the non-standard speech sample corresponding to the largest similarity as the target acoustic feature.

[0017] In this implementation, since the target acoustic feature determined by the electronic device is an acoustic feature of a non-standard speech sample whose similarity to the first acoustic feature is greater than a similarity threshold, the recognition accuracy of the electronic device can be improved.

[0018] In a possible implementation, the first voice data may include at least one of a repeated syllable, a dragged syllable, or a paused syllable.

[0019] In one possible implementation, the electronic device may further include a non-standard speech generation model, and the electronic device may obtain a first standard speech sample, a first standard text corresponding to the first standard speech sample, and a first non-standard text corresponding to the first standard speech sample. The electronic device may input the first standard speech sample and the first non-standard text into the non-standard speech generation model at the same time to obtain the first non-standard speech sample. The electronic device may input the first non-standard speech sample into the first speech recognition model, and after processing by the feature extraction module and the acoustic module in the first speech recognition model, obtain the acoustic features of the first non-standard speech sample. The electronic device may store the acoustic features of the first non-standard speech sample and the first standard text as a set of data in a feature database. Based on the above steps, the electronic device may construct other sets of data in the feature database.

[0020] In this implementation, the electronic device can use a non-standard speech generation model to obtain a first non-standard speech sample, and input the first non-standard speech sample into a first speech recognition model to obtain the acoustic features of the first non-standard speech sample. The acoustic features of the first non-standard speech sample can be used for subsequent matching, thereby saving processing resources of the electronic device and improving recognition efficiency.

[0021] In one possible implementation, the non-standard speech generation model may include an audio encoding module, a text encoding module, a feature fusion module, and an audio decoding module. The outputs of the audio encoding module and the text encoding module are both connected to the input of the feature fusion module, and the output of the feature fusion module is connected to the input of the audio decoding module.

[0022] In this implementation, a first non-standard speech sample can be generated through the audio encoding module, text encoding module, feature fusion module, and audio decoding module in the non-standard speech generation model.

[0023] In one possible implementation, the electronic device may input a first standard voice sample into an audio encoding module of a non-standard voice generation model to obtain audio features of the first standard voice sample, and input a first non-standard text into a text encoding module of the non-standard voice generation model to obtain text features of the first non-standard text. The electronic device may simultaneously input the audio features of the first standard voice sample and the text features of the first non-standard sample into a feature fusion module of the non-standard voice generation model to obtain audio features of the first non-standard voice sample. The electronic device may input the audio features of the first non-standard voice sample into an audio decoding module of the non-standard voice generation model to obtain the first non-standard voice sample.

[0024] In this implementation, a first non-standard voice sample can be obtained after processing by the audio encoding module, the text encoding module, the feature fusion module, and the audio decoding module, so that the electronic device can generate the acoustic features of the first non-standard voice sample based on the first non-standard voice sample.

[0025] In a possible implementation, the non-standard speech generation model may further include a random variable generation module, the output end of the random variable generation module is connected to the input end of the feature fusion module, and the random variable generation module is used to generate audio features of random noise.

[0026] Exemplarily, the input of the random variable generation module may be a standard normal distribution. The random variable generation module may sample from the standard normal distribution to obtain a latent variable in the standard normal distribution. The latent variable may be used as an audio feature of the random noise.

[0027] In this implementation, the audio features of the random noise generated by the random variable generation module can be used to guide the fusion process of the feature fusion module, so that the audio features of the first non-standard speech sample generated by the feature fusion module are diverse and random, thereby avoiding the audio features of the first non-standard speech sample being single.

[0028] In a possible implementation, the electronic device inputs the audio features of the first standard speech sample, the text features of the first non-standard sample, and the audio features of random noise into a feature fusion module to obtain the audio features of the first non-standard speech sample.

[0029] In this implementation, the authenticity of the audio features of the obtained first non-standard speech sample can be improved by adding the audio features of the audio random noise to the feature fusion module.

[0030] In a second aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory is used to store computer-executable instructions, and the processor is used to run the computer-executable instructions stored in the memory to execute the method described in the first aspect or any possible implementation of the first aspect.

[0031] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0032] In a fourth aspect, an embodiment of the present application provides a computer program product comprising a computer program, which, when executed, enables a computer to execute the method described in the first aspect or any possible implementation of the first aspect.

[0033] In a fifth aspect, the present application provides a chip or chip system, comprising at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is configured to execute a computer program or instruction to perform the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip may be an input / output interface, a pin, or a circuit.

[0034] In one possible implementation, the chip or chip system described above in this application further includes at least one memory, wherein instructions are stored in the at least one memory. The memory may be a storage unit within the chip, such as a register or cache, or a storage unit of the chip (such as a read-only memory or random access memory).

[0035] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1A A schematic diagram of a mobile phone interface change;

[0037] Figure 1B It is a schematic diagram of a scenario where a mobile phone controls an air conditioner;

[0038] Figure 2 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0039] Figure 3 It is a schematic flow diagram of a speech recognition method provided by an embodiment of the present application;

[0040] Figure 4 It is a schematic flow diagram of another speech recognition method provided by an embodiment of the present application;

[0041] Figure 5 It is a schematic diagram of the structure of the first speech recognition model provided by an embodiment of the present application;

[0042] Figure 6 It is a schematic flow diagram of another speech recognition method provided by an embodiment of the present application;

[0043] Figure 7 It is a schematic diagram of the structure of an acoustic module provided by an embodiment of the present application; <00利用本申请实施例提供的语音识别方法,识别用户的指令,根据识别结果控制电子设备的运行。

[0044] Figure 8 It is a schematic flow diagram of another speech recognition method provided by an embodiment of the present application;

[0045] Figure 9 It is a schematic diagram of the structure of another first speech recognition model provided by an embodiment of the present application. [[ID=!]] Detailed implementation manners [[ID=!]] [[ID=!]]

[0046] To facilitate a clear description of the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application: [[ID=!]]<!]] [[ID=!]]

[0047] 1. Speech disorder: It can be manifested as non - fluency in the speech expression process. A speech disorder can include, but is not limited to, at least one of the following: pronunciation repetition, prolongation of sounds, and pauses, etc. [[ID=!]] [[ID=!]]

[0048] Among them, pronunciation repetition can be the repetition of a certain character or phoneme. Exemplarily, "kong kong" is the repetition of the character "kong", and "K - K - KONG" is the repetition of the phoneme "K". [[ID=!]] [[ID=!]]

[0049] Prolongation of sounds refers to prolonging the duration of a certain character or phoneme during pronunciation. Exemplarily, "Open ~~ the air conditioner" is the prolongation of the sound of the character "kai". Here, "~~" represents the prolongation of the sound. [[ID=!]] [[ID=!]]

[0050] A pause refers to suddenly getting stuck during the speech process and being unable to smoothly pronounce the next character or phoneme, which can be accompanied by filler words (such as "um" or "uh", etc.). Exemplarily, "Open uh the air conditioner". [[ID=!]]

[0051] In some embodiments, the speech disorder may be referred to as stuttering.

[0052] In some embodiments, taking a voice interaction scenario as an example, and when the voice interaction function is turned on in an electronic device, the content spoken by the user, or the content expressed verbally, or the content pronounced can all be referred to as voice instructions.

[0053] Taking the electronic device as a mobile phone as an example, the voice command may include but is not limited to at least one of the following: "turn up the volume", "turn down the volume", "pause", or "put down a song".

[0054] Taking the electronic device as an in-vehicle system as an example, the voice command may include but is not limited to at least one of the following: "open the map", or "turn on the air conditioner", etc.

[0055] In some embodiments, voice commands may be divided into the following types: standard voice commands and non-standard voice commands.

[0056] Among them, standard voice commands refer to voice commands issued when the user has no speech impairment, and non-standard voice commands refer to voice commands issued when the user has speech impairment.

[0057] For example, a standard voice command may be: “Turn on the air conditioner”, or “Turn up the volume”, etc., and a non-standard voice command may be: “Turn on the air conditioner”, or “Turn up the volume”, etc.

[0058] 2. Other terms

[0059] In the embodiments of the present application, words such as "first" and "second" are used to distinguish identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or execution order, and words such as "first" and "second" do not necessarily mean different.

[0060] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0061] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or plural.

[0062] 3. Electronic devices

[0063] The electronic devices of the embodiments of the present application may include handheld devices, vehicle-mounted devices, etc. with voice recognition functions. For example, some electronic devices include: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, terminal devices in 5G networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc. The embodiments of the present application are not limited to this.

[0064] As an example and not a limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also achieve powerful functions through software support, data interaction, and cloud server interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as mobile phones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0065] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0066] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0067] In the embodiments of the present application, electronic devices or network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also known as main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as the Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0068] Speech recognition technology is widely used in various scenarios, including but not limited to any of the following: smart home scenarios, car scenarios, mobile phone voice assistant scenarios, or medical and health scenarios.

[0069] For example, in the smart home scenario, users can control smart devices in their home, such as speakers, TVs, or air conditioners, through voice commands, so that they can play music, check the weather, or adjust the temperature, which can improve the portability of users' lives.

[0070] For example, taking the in-vehicle scenario as an example, the vehicle is equipped with a voice control function. When the user is driving the vehicle and the user turns on the voice control function in the vehicle, the user can control the in-vehicle system through voice commands, such as controlling the in-vehicle system to make calls, send messages, or query navigation routes, etc., without the need for manual operation by the user, which can reduce user distraction and thus improve driving safety.

[0071] For example, taking the mobile phone voice assistant scenario as an example, the mobile phone is equipped with a voice control function. When the user turns on the voice control function, the user can control the mobile phone through voice commands, such as setting alarm reminders, playing music, or adjusting the volume, etc., without the user manually touching the screen or buttons, saving time.

[0072] For example, a mobile phone is equipped with a voice command library and an automatic speech recognition (ASR) model, and it also has voice control functionality. The voice command library may include standard text corresponding to multiple standard voice commands. For example, the standard text corresponding to a standard voice command may be "play music" or "turn up the volume."

[0073] For example, in response to a voice command (standard voice command or non-standard voice command) issued by a user, such as "turn up the volume" or "turn up the volume very high", the ASR model configured in the mobile phone can convert the voice command received from the user into corresponding text.

[0074] The ASR model may include, but is not limited to, at least one of the following: a Gaussian mixture model-hidden Markov model (GMM-HMM), a deep neural network-hidden Markov model, or a convolutional neural network.

[0075] It should be understood that the ASR model configured in the mobile phone is a pre-trained ASR model. In addition, this application does not elaborate on the structure of the above-mentioned ASR model, and you can refer to the description in the current existing technology.

[0076] In one possible scenario, a user uses a mobile phone to play a TV series, and the user controls the mobile phone to adjust the volume through voice commands. Figure 1A This is a schematic diagram of the changes in the mobile phone interface, refer to Figure 1A In (a), the interface 10 is in response to a voice command input by the user, such as "turn up the volume", and the mobile phone can display a page related to the voice control function, such as the page for waking up the voice control function. Figure 1A (a) may include an icon 101 for waking up the voice control function. The voice command input by the user may be understood as: the user says to the microphone of the mobile phone: "Turn up the volume".

[0077] Accordingly, the microphone of the mobile phone can capture the voice command "turn up the volume" from the user, and the mobile phone can use the preset ASR model to convert the voice command "turn up the volume" into corresponding text, referring to Figure 1A In (b) of the figure, the mobile phone may display interface 11, which may include text corresponding to the voice command, such as "Turn up the volume". In addition, the mobile phone may search for the text corresponding to the voice command in a preset voice command library. Since the voice command "Turn up the volume" is a standard voice command, there is a standard text in the voice command library that is the same as the text. Therefore, the mobile phone may execute the standard voice command corresponding to the standard text, such as controlling the speaker to increase the volume by 10 decibels (DB), and referring to Figure 1A In (c), the mobile phone may display an interface 12 , which may include text corresponding to: “The volume has been increased by 10 DB”.

[0078] In this scenario, when a user experiences a speech impediment, for example, the user enters the voice command "Turn up the volume." After the phone's microphone captures the voice command "Turn up the volume," the phone converts the command into text using a pre-set ASR model. The phone then searches for the text corresponding to the command in a pre-set voice command library. However, since the command "Turn up the volume" is non-standard, there is no standard text matching it in the library. Therefore, the phone cannot recognize the command and cannot execute it. In other words, the phone will not control the speaker to increase the volume by 10 dB.

[0079] In some embodiments, to solve the problem that when a user has a speech disorder with repeated pronunciation, the electronic device cannot recognize a non-standard voice command from the user and thus cannot execute the non-standard voice command, after the electronic device converts the non-standard voice command into the corresponding text through the ASR model, the electronic device will delete a part of the text. For example, the electronic device can retain one of the multiple repeated words or phonemes in the text corresponding to the non-standard voice command and delete the remaining repeated words or phonemes to obtain the text after deletion.

[0080] Since the text after deletion is the same as a standard text in the preset voice command library of the electronic device, the electronic device can recognize the non-standard voice command, and the electronic device can execute the voice command corresponding to the text after deletion.

[0081] Exemplarily, when the user has a speech disorder with repeated pronunciation, the non-standard voice command is "turn up the volume greatly". After the electronic device converts the non-standard voice command "turn up the volume greatly" into the corresponding text through the ASR model, the electronic device can retain one "big" character in the text "turn up the volume greatly" corresponding to the non-standard voice command and delete the remaining "big" characters to obtain the text after deletion "turn up the volume". Since the standard text "turn up the volume" in the voice command library is the same as the text after deletion "turn up the volume", the electronic device can recognize the non-standard voice command, and the electronic device can control the speaker to increase the volume by 10DB.

[0082] However, when the user wakes up the voice control function of the electronic device through a voice command, for example, the voice command for the user to wake up the voice control function of the electronic device is "Hello, Youyou". In response to the voice command "Hello, Youyou" from the user, after the electronic device converts the voice command into the corresponding text, it deletes a part of the text to obtain the text after deletion "Hello, You". Since the text corresponding to the voice command for waking up the electronic device in the preset voice command library is "Hello, Youyou", and this text "Hello, Youyou" is different from "Hello, You", the electronic device cannot recognize the voice command "Hello, Youyou" from the user, and thus cannot wake up the voice control function of the electronic device.

[0083] It should be understood that the above voice command "Hello, Youyou" for the user to wake up the voice control function of the electronic device is a standard voice command. However, since there is a repeated word "You" in the text after the command is converted, the electronic device will process this text as the text corresponding to the non-standard voice command, resulting in the electronic device being unable to recognize the voice command "Hello, Youyou".

[0084] In addition, in this method, the electronic device cannot recognize the voice commands (non-standard voice commands) when the user has speech disorders such as slurring or pausing.

[0085] In order to solve the problem that electronic devices cannot recognize non-standard voice commands when the user has speech disorders, in some embodiments, a pre-trained ASR model and a pre-trained natural language understanding (NLU) model can be pre-deployed in the cloud server.

[0086] A cloud server is a server built on cloud computing technology that provides remote computing and storage services. When an electronic device is connected to the internet, it can access the cloud server for remote computing and storage. For example, an electronic device can upload a voice command to the cloud server, which can then process the command, generate a result, and return the result to the electronic device.

[0087] It should be understood that the embodiments of the present application do not elaborate on the process of training the ASR model and the NLU model, and reference can be made to the description in the current prior art.

[0088] When an electronic device is connected to the internet (network), in response to a user inputting a non-standard voice command, the electronic device's microphone can capture the non-standard voice command from the user and upload the non-standard voice command to a cloud server. In response to the non-standard voice command from the electronic device, the cloud server can convert the non-standard voice command into corresponding text (non-standard text) using a pre-trained ASR model. After obtaining the text corresponding to the non-standard voice command, the cloud server can analyze the text corresponding to the non-standard voice command using a pre-trained NLU model to obtain the user's true intention or need. This true intention or need is the standard text corresponding to the non-standard voice command.

[0089] After obtaining the user's true intention or demand, the cloud server can send the true intention or demand to the electronic device. Accordingly, in response to the user's true intention or demand from the cloud server, the electronic device can search in the preset voice command library. Since the standard text is stored in the preset voice command library of the electronic device, the electronic device can recognize non-standard voice commands, and the electronic device can execute the standard voice commands corresponding to the standard text, thereby solving the problem that the electronic device cannot recognize the voice commands when the user has a speech disorder.

[0090] For example, Figure 1B This is a schematic diagram of a scenario where a mobile phone controls the air conditioner. Figure 1B The scenario may include: a mobile phone, an air conditioner, and a cloud server. The air conditioner may be a Bluetooth-enabled air conditioner. When a Bluetooth connection is established between the mobile phone and the air conditioner, the mobile phone can control the air conditioner to turn on or off.

[0091] In this scenario, in response to a non-standard voice command from the user, such as "Turn on the air conditioner," the phone's microphone can capture the non-standard voice command from the user, and the phone can send the non-standard voice command "Turn on the air conditioner" to the cloud server. The cloud server uses the ASR model to convert the non-standard voice command into corresponding text (non-standard text) and uses this text as input to the NLU model. The NLU model determines that the user's true intention or request is "Turn on the air conditioner." The cloud server can then send the true intention or request "Turn on the air conditioner" to the phone.

[0092] Accordingly, the mobile phone can receive the actual intention or request "turn on the air conditioner" from the cloud server. Since "turn on the air conditioner" is a standard text, the mobile phone can find the standard text "turn on the air conditioner" in the preset voice command library and send the "turn on the air conditioner" command to the air conditioner. Accordingly, when the air conditioner receives the "turn on the air conditioner" command from the mobile phone, it can control the air conditioner to turn on.

[0093] In some embodiments, reference Figure 1B , after the air conditioner is controlled to turn on, the air conditioner can send a "turn on successfully" message to the mobile phone.

[0094] However, in this example, a pre-trained ASR model and a pre-trained NLU model need to be deployed in the cloud server in advance. Furthermore, when the electronic device receives a non-standard voice command from the user, it needs to interact with the cloud server to obtain the user's true intention or request, resulting in low response efficiency. For example, if the network signal is unstable, it takes a long time for the electronic device to obtain the user's true intention or request from the cloud server, affecting the user experience.

[0095] In addition, every time an electronic device receives a non-standard voice command from a user, it needs to call the ASR model and NLU model in the cloud server. The operation of the ASR model and NLU model requires a large amount of computing resources, which not only increases the power consumption of the cloud server, but also accelerates the power consumption of the electronic device.

[0096] If the purpose is to reduce the latency of interaction between electronic devices and cloud servers, pre-trained ASR models and pre-trained NLU models are directly deployed in electronic devices. However, since the operation of ASR models and NLU models requires a large amount of computing resources and the amount of data of ASR models and NLU models is large (such as tens of megabytes), while the storage space and computing resources of electronic devices are limited, it is difficult to deploy ASR models and NLU models in electronic devices.

[0097] Accordingly, an embodiment of the present application provides a speech recognition method, in which a first speech recognition model can be pre-set in an electronic device (the first speech recognition model can refer to the description in the following embodiment), and the electronic device can use the feature extraction module and acoustic module in the first speech recognition model to pre-generate acoustic features of multiple non-standard voice instructions, and store the acoustic features of the multiple non-standard voice instructions and the standard text corresponding to the acoustic features of each non-standard voice sample in a feature database. When a user inputs a non-standard voice instruction, the electronic device can use the first speech recognition model to obtain the acoustic features of the non-standard voice instruction, and match them with the acoustic features of multiple non-standard voice samples in the feature database to obtain the target acoustic features and the standard text corresponding to the target acoustic features, so that the electronic device can execute the voice instruction corresponding to the standard text.

[0098] In this method, since the electronic device does not need to interact with the cloud server, the latency can be reduced, thereby improving the recognition efficiency of the electronic device. Moreover, since the electronic device can match based on the acoustic features of multiple pre-generated non-standard voice samples, the computing resources of the electronic device can be saved, thereby reducing the power consumption of the electronic device and improving the user experience.

[0099] Before introducing the speech recognition method provided by the embodiment of the present application, the structure of the electronic device provided by the embodiment of the present application is first introduced:

[0100] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, referring to Figure 2 The electronic device 200 may include a processor 210, an audio module 211, a speaker 211A, a receiver 211B, a microphone 211C, an earphone jack 211D, a sensor 212, a display 213, an internal memory 214, a wireless communication module 215, and an antenna 1. It should be understood that the structure shown in this embodiment does not constitute a specific limitation on the electronic device 200.

[0101] In the embodiment of the present application, the processor 210 can be used to execute the speech recognition method provided in the embodiment of the present application.

[0102] Microphone 211C can be used to capture non-standard voice commands from the user.

[0103] The display screen 213 can be used to display an interface. For example, when a user wakes up the voice control function of a mobile phone, the display screen 213 can display a wake-up interface.

[0104] The internal memory 214 can be used to store relevant instructions so as to implement the speech recognition method provided in the embodiment of the present application when the processor 210 calls the relevant instructions.

[0105] In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0106] The speech recognition method provided by the embodiments of the present application is described below in conjunction with specific embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0107] The following describes the speech recognition method provided by the embodiment of the present application. Figure 3 A flow chart of a speech recognition method provided in an embodiment of the present application. Figure 3 The speech recognition method provided in the embodiment of the present application may include:

[0108] S301: The electronic device obtains a first standard voice sample, a first standard text corresponding to the first standard voice sample, and a first non-standard text corresponding to the first standard voice sample.

[0109] The first standard voice sample may be a standard voice instruction, the first standard text may be a text corresponding to the standard voice instruction, and the first non-standard text may be a text corresponding to the non-standard voice instruction.

[0110] Exemplarily, the first standard voice sample may be the audio corresponding to "turn up the volume", the first standard text corresponding to the first standard voice sample may be the text corresponding to "turn up the volume", and the first non-standard text corresponding to the first standard voice sample may be the text corresponding to "turn up the volume", or the text corresponding to "turn up the volume", or the text corresponding to "turn up the volume", etc.

[0111] In some embodiments, when the electronic device is first started, or when a user first registers the voice control function in the electronic device, the electronic device may obtain a first standard text corresponding to a first standard voice sample from a preset voice command library. For example, the first standard text corresponding to the first standard voice sample may be: "Turn up the volume."

[0112] In this example, when a user registers the voice control function in an electronic device for the first time, the electronic device may record the user's voiceprint and information about the user's surrounding environment.

[0113] Among them, voiceprint is the specific voice feature of each user. The user-specific voice features may include pronunciation habits, timbre, intonation, or speaking speed. Voiceprint can be used to identify users like fingerprints. By recording the user's voiceprint, electronic devices can distinguish the voice of the user who registered the voice control function from the voices of other users.

[0114] In some embodiments, a text-to-speech (TTS) model is pre-configured in the electronic device. The TTS model is used to convert the first standard text into a first standard voice sample that conforms to the user's voiceprint and surrounding environment information.

[0115] Exemplarily, the electronic device may input the first standard text, the recorded user's voiceprint, and the recorded surrounding environment information into the TTS model, and the output of the TTS model is a first standard voice sample that conforms to the user's voiceprint and the surrounding environment information.

[0116] In some embodiments, the electronic device may add text to the first standard text based on a preset rule to obtain a first non-standard text. Each rule may correspond to a different type of speech disorder. For example, the first standard text may be the text corresponding to "Turn up the volume," and the first non-standard text may be the text corresponding to "Turn up the volume," or the text corresponding to "Turn up the volume," or the text corresponding to "Turn up the volume."

[0117] S302: The electronic device simultaneously inputs a first standard voice sample and a first non-standard text into a non-standard voice generation model to obtain a first non-standard voice sample.

[0118] Among them, the non-standard speech generation model is used to generate non-standard speech samples.

[0119] In some embodiments, reference Figure 4 The non-standard speech generation model may include an audio encoding module, a text encoding module, a feature fusion module, and an audio decoding module. The audio encoding module is used to generate audio features of a first standard speech sample, the text encoding module is used to generate text features of a first non-standard text, the feature fusion module is used to generate audio features of the first non-standard speech sample, and the audio decoding module is used to generate the first non-standard speech sample.

[0120] In this example, the output ends of the audio encoding module and the text encoding module are both connected to the input end of the feature fusion module, and the output end of the feature fusion module is connected to the input end of the audio decoding module.

[0121] For example, Figure 4 As shown, the input of the audio encoding module is a first standard voice sample, such as the audio corresponding to "turn up the volume", the input of the text encoding module is a first non-standard text, such as the text corresponding to "turn up the volume", and the output of the audio decoding module is a first non-standard voice sample, such as the audio corresponding to "turn up the volume".

[0122] In some embodiments, the audio encoding module may include: a variational auto-encoder (VAE), which can capture the audio features of a first standard speech sample by learning the distribution of audio data. The audio features of the first standard speech sample can be represented by a vector in a latent space.

[0123] Reference Figure 4 , the audio features of the first standard speech sample can be represented by vector 1, the text features of the first non-standard text can be represented by vector 2, and the audio features of the first non-standard speech sample can be represented by vector 3.

[0124] In some embodiments, the feature fusion module may include a pre-trained generative model, which is a machine learning model that aims to learn the underlying distribution patterns of the original data and generate new data samples similar to the original data.

[0125] The pre-trained generative model may include, but is not limited to, any of the following: a generative adversarial network (GAN) model or a diffusion model. The original data may include text features of the first non-standard text and audio features of the first standard speech sample. The new data sample may include audio features of the first non-standard speech sample.

[0126] Taking the generative model in the feature fusion module as a diffusion model as an example, in some embodiments, referring to Figure 4 The non-standard speech generation model may further include: a random variable generation module, which may be used to generate audio features of random noise.

[0127] like Figure 4 As shown, the output of the random variable generation module is connected to the input of the feature fusion module. Exemplarily, the random variable generation module can sample from a standard normal distribution to obtain the audio features of random noise.

[0128] It should be understood that since there are multiple first non-standard texts, the electronic device can obtain a first non-standard voice sample by inputting each first non-standard text and the first standard voice sample into the non-standard voice generation model at the same time.

[0129] Exemplarily, the electronic device simultaneously inputs the text corresponding to the first non-standard text such as "Turn up the volume~" and the audio corresponding to the first standard voice sample such as "Turn up the volume" into the non-standard voice generation model, and can obtain the audio corresponding to the first non-standard voice sample such as "Turn up the volume~". Exemplarily, the electronic device simultaneously inputs the text corresponding to the first non-standard text such as "Turn up the volume uh" and the audio corresponding to the first standard voice sample such as "Turn up the volume" into the non-standard voice generation model, and can obtain the audio corresponding to the first non-standard voice sample such as "Turn up the volume uh".

[0130] S303: The electronic device inputs the first non-standard speech sample into the first speech recognition model, and obtains acoustic features of the first non-standard speech sample after processing by the feature extraction module and the acoustic module in the first speech recognition model.

[0131] The first speech recognition model is a lightweight speech recognition model pre-installed in the electronic device for recognizing non-standard speech data. Exemplarily, the non-standard speech data may be a first non-standard speech sample.

[0132] It is understandable that, unlike the ASR model, the first speech recognition model does not include a language module and a decoding module. In the ASR model, the output of the acoustic module is connected to the input of the language module, which in turn is connected to the input of the decoding module. Through the language module and the decoding module, the first non-standard text corresponding to the first non-standard speech sample can be obtained.

[0133] like Figure 5 As shown, the first speech recognition model may include: a feature extraction module and an acoustic module, and the output end of the feature extraction module is connected to the input end of the acoustic module.

[0134] The feature extraction module is used to extract audio features of the first non-standard voice sample, and the audio features of the first non-standard voice sample may include: time domain information and frequency domain information of the first non-standard voice sample;

[0135] For example, the time domain information may be the amplitude envelope, kurtosis, or skewness of the first non-standard speech sample, and the frequency domain information may be the spectrum or spectrum energy of the first non-standard speech sample.

[0136] The acoustic model is used to map the audio features of the first non-standard speech sample from a high-dimensional space to a low-dimensional space to obtain acoustic features of the first non-standard speech sample, where the acoustic features are used to indicate semantic information of the first non-standard speech sample.

[0137] It can be understood that, generally, the dimension of the audio feature of the first non-standard speech sample is higher than the dimension of the acoustic feature.

[0138] For example, refer to Figure 6 The input of the feature extraction module is the first non-standard speech sample, such as the audio corresponding to "turn up the volume uh", and the output of the acoustic module is the acoustic feature of the audio corresponding to "turn up the volume uh".

[0139] In some embodiments, taking the frequency domain information as spectrum energy as an example, the feature extraction module may include a feature extraction algorithm. The feature extraction algorithm may be, for example, mel-frequency cepstral coefficients (MFCC) or a filter bank (Fbank), etc. The electronic device may use the feature extraction algorithm to obtain the spectrum energy of the first non-standard speech sample.

[0140] MFCC refers to dividing the spectrum of the first non-standard speech sample into multiple bandpass filters on the Mel scale, and then calculating the output energy of each filter, thereby obtaining the spectral energy distribution of the first non-standard speech sample on the Mel scale. The energy distribution can be the spectral energy of the first non-standard speech sample.

[0141] Fbank refers to passing the first non-standard speech sample through a set of bandpass filters, each filter corresponding to a frequency range, and then calculating the output energy of each filter to obtain the energy distribution of the first non-standard speech sample in the frequency domain. The energy distribution can be the spectral energy of the first non-standard speech sample.

[0142] It should be understood that the process by which the electronic device obtains the spectral energy of the first non-standard speech sample in the above example does not constitute a limitation of the present application. In some embodiments, the electronic device can obtain the spectral energy of the first non-standard speech sample using methods other than MFCC and Fbank, which is not elaborated in the embodiments of the present application.

[0143] In some embodiments, the acoustic module may include a pre-trained acoustic model, and the electronic device may use the output of the last layer in the acoustic model as the acoustic feature of the first non-standard speech sample, or the electronic device may use the output of the second-to-last layer in the acoustic model as the acoustic feature of the first non-standard speech sample. This embodiment of the present application is not limited to this.

[0144] For example, the pre-trained acoustic model is a zipformer model. Figure 7 It is a structural diagram of the zipformer model, such as Figure 7As shown in the figure, the zipformer model can include a feed-forward layer, a multi-head attention layer, a non-linear attention layer, a self-attention layer, a convolution layer, a bypass, and a biasnorm. For the description of each module, please refer to the relevant description in the existing zipformer model.

[0145] Among them, the feedforward layer is composed of a fully connected layer, which can perform nonlinear changes on the input data; the multi-head attention layer can calculate the semantic information of different subspaces in parallel through multiple attention heads, obtain multiple attention weights, and distribute the multiple attention weights to the nonlinear attention layer and the self-attention layer respectively; the nonlinear attention layer is used to introduce nonlinear transformations; the self-attention layer is used to calculate the internal dependencies of the input data; the convolution layer is used to capture details; the bypass is used to avoid gradient disappearance; and normalization is used to accelerate the convergence of the zipformer model.

[0146] It should be understood that bypass and biasnorm are not layers in the zipformer model.

[0147] Reference Figure 7 The input of the zipformer model is the audio feature of the first non-standard speech sample. The electronic device can use the output of the last feed-forward layer "Feed-forword" as the acoustic feature of the first non-standard speech sample, or use the output of the penultimate convolution layer "Convolution" as the acoustic feature of the first non-standard speech sample.

[0148] In some embodiments, the acoustic features of the first non-standard speech sample may be referred to as matching embeddings, or acoustic embeddings, or embedding vectors.

[0149] It should be understood that the embodiment of the present application does not elaborate on the process of the electronic device generating acoustic features of other non-standard voice samples, and can refer to the description of the electronic device generating acoustic features of the first non-standard voice sample.

[0150] S304: The electronic device stores the acoustic features of the first non-standard voice sample and the first standard text as a set of data in a feature database.

[0151] Since different non-standard speech samples may correspond to different acoustic features, the feature database may store acoustic features of multiple different non-standard speech samples and the standard text corresponding to the acoustic features of each non-standard speech sample.

[0152] In the embodiments of the present application, an electronic device can obtain a first standard speech sample, a first standard text corresponding to the first standard speech sample, and a first non-standard text corresponding to the first standard speech sample. The electronic device can use a non-standard speech generation model to obtain a first non-standard speech sample, and input the first non-standard speech sample into a first speech recognition model to obtain the acoustic features of the first non-standard speech sample. The electronic device can store the acoustic features of the first non-standard speech sample and the first standard text as a set of data in a feature database. In this method, since the feature database of the electronic device pre-stores the acoustic features of multiple different non-standard speech samples and the standard text corresponding to the acoustic features of each non-standard speech sample, the feature database can be used for subsequent matching, so that the electronic device can recognize non-standard speech commands (speech data) from users and improve the recognition efficiency.

[0153] In the above embodiments, the process of the electronic device constructing the feature database is introduced. Next, the process of recognizing non-standard speech commands (speech data) from users based on the constructed feature database is introduced. Figure 8 It is a schematic flowchart of another speech recognition method provided by the embodiments of the present application. Referring to Figure 8 , the process of the speech recognition method provided by the embodiments of the present application can include:

[0154] S801, the electronic device receives first speech data, and the first speech data includes non-standard speech data.

[0155] In some embodiments, the electronic device can receive the first speech data (the first speech command) from the user through a microphone. The first speech data can include, but is not limited to, at least one of the following: repeated syllables, elongated syllables, or paused syllables.

[0156] Exemplarily, the first speech data can be "turn up the volume", and the syllable "big" in the first speech data is a repeated syllable; exemplarily, the first speech data can be "turn up ~~ the volume", and the syllable "big" in the first speech data is an elongated syllable; exemplarily, the first speech data can be "turn up uh the volume", and the "uh" in the first speech data is a paused syllable.

[0157] It can be understood that a syllable can be a certain character or phoneme in the first speech data.

[0158] S802, the electronic device inputs the first speech data into the first speech recognition model to obtain the standard text corresponding to the first speech data.

[0159] Such as Figure 9As shown, the first speech recognition model may further include a feature matching module, wherein the input of the feature matching module is connected to the output of the acoustic module, and the feature matching module is configured to obtain the standard text corresponding to the first speech data. The first speech recognition model is a lightweight speech recognition model pre-installed in the electronic device for recognizing non-standard speech data.

[0160] In some embodiments, the electronic device can input the first voice data into the feature extraction module in the first voice recognition model. After processing by the feature extraction module, the first audio feature can be obtained. The first audio feature can include time domain information and frequency domain information of the first voice data.

[0161] The process of the electronic device acquiring the first audio feature of the first voice data may refer to the description of the electronic device acquiring the audio feature of the first non-standard voice sample in S803.

[0162] In some embodiments, the electronic device may input the first audio feature into the acoustic module in the first speech recognition model, and obtain the first acoustic feature after processing by the acoustic module. The first acoustic feature is used to indicate the semantic information of the first speech data.

[0163] The process of the electronic device acquiring the first audio feature of the first voice data may refer to the description of the electronic device acquiring the acoustic feature of the first non-standard voice sample in S803.

[0164] It should be understood that, generally, the dimension of the first audio feature is higher than the dimension of the first acoustic feature.

[0165] In some embodiments, the feature matching module may include a preset feature database. The electronic device may input the first acoustic feature into the feature matching module in the first speech recognition model, and obtain the target acoustic feature whose similarity with the first acoustic feature is greater than the similarity threshold from the feature database by traversing the feature database. The electronic device may use the standard text corresponding to the target acoustic feature as the standard text corresponding to the first speech data.

[0166] In some embodiments, a similarity calculation method can be preset in the feature matching module. The electronic device can use the similarity calculation method to calculate the similarity between the acoustic features of multiple non-standard speech samples in the feature database and the first acoustic feature to obtain multiple similarity values. The electronic device can compare the maximum similarity value among the multiple similarity values ​​with the similarity threshold. If the maximum similarity value among the multiple similarity values ​​is greater than the similarity threshold, the electronic device can use the acoustic feature of the non-standard speech sample corresponding to the maximum similarity value as the target acoustic feature.

[0167] The similarity calculation method may include but is not limited to any one of the following: cosine similarity, or Euclidean distance, etc. It should be understood that the embodiment of the present application does not elaborate on the aforementioned similarity calculation method, and reference may be made to the description in the current prior art.

[0168] Exemplarily, the feature database may include: acoustic feature 1, acoustic feature 2, and acoustic feature 3, etc. The electronic device may calculate the similarity between the multiple acoustic features and the first acoustic feature to obtain corresponding similarity value 1, similarity value 2, and similarity value 3. Among them, similarity value 1 is the similarity between acoustic feature 1 and the first acoustic feature, similarity value 2 is the similarity between acoustic feature 2 and the first acoustic feature, and similarity value 3 is the similarity between acoustic feature 3 and the first acoustic feature.

[0169] In this example, if similarity value 1 is the largest and if similarity value 1 is greater than the similarity threshold, the electronic device may use acoustic feature 1 as the target acoustic feature.

[0170] After obtaining the target acoustic feature, the electronic device may use the standard text corresponding to the target acoustic feature in the feature database as the standard text corresponding to the first voice data.

[0171] S803, the electronic device executes the instruction corresponding to the standard text.

[0172] For example, the standard text is: "Turn up the volume", and the corresponding instruction may be: increase the volume by 10DB.

[0173] In an embodiment of the present application, when the electronic device receives the first voice data, the electronic device can input the first voice data into the first voice recognition model to obtain the standard text corresponding to the first voice data, and the electronic device can execute the instructions corresponding to the standard text. On the one hand, since the electronic device does not need to interact with the cloud server when receiving the first voice data, it can directly obtain the standard text corresponding to the first voice data, which can save the time of the electronic device in processing the first voice data, thereby improving the recognition efficiency. On the other hand, the resources consumed by the electronic device when running the first voice recognition model are far less than the resources consumed by running the ASR model and the NLU model at the same time, which can save the power consumption of the electronic device.

[0174] In some embodiments, using a mobile phone as an example, the phone's settings application may integrate a non-standard voice data recognition function, which may include a first voice recognition model. If a user enables the non-standard voice data recognition function, the electronic device may recognize non-standard voice data (non-standard voice commands) from the user based on a preset feature database. If the user does not enable the non-standard voice recognition function, the electronic device may recognize standard voice data (standard voice commands) from the user based on a preset voice command library.

[0175] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0177] The speech recognition method of the embodiment of the present application has been described above. The device for performing the above method provided by the embodiment of the present application is described below. Those skilled in the art will understand that the method and device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can perform the steps in the above-mentioned list sorting method.

[0178] The speech recognition method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices. The specific device form of the terminal device can refer to the above related descriptions and will not be repeated here.

[0179] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the above method.

[0180] The embodiments of the present application provide a chip or chip system. The chip or chip system includes at least one processor and a communication interface. The communication interface is interconnected with the at least one processor via a line. The at least one processor is used to run a computer program or instruction to implement the technical solution in the above embodiment. Its implementation principle and technical effects are similar to those of the above-mentioned related embodiments and will not be repeated here. The communication interface in the chip can be an input / output interface, a pin, or a circuit, etc.

[0181] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned method is implemented. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0182] In one possible implementation, computer-readable media may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium designed to carry or store the desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include optical disc, laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above are also intended to be included within the scope of computer-readable media.

[0183] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0184] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram.

[0185] The above specific implementation methods further explain in detail the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above are only specific implementation methods of the embodiments of the present application and are not intended to limit the scope of protection of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the scope of protection of the embodiments of the present application.

Claims

1. A speech recognition method, characterized in that: include: The electronic device receives first voice data, where the first voice data includes non-standard voice data; the non-standard voice data is voice data emitted by a user when a speech disorder occurs; The electronic device inputs the first voice data into a first voice recognition model to obtain a standard text corresponding to the first voice data; the standard text is the text corresponding to the voice data uttered by the user without speech impairment; the first voice recognition model is a lightweight voice recognition model pre-installed in the electronic device for recognizing non-standard voice data; the first voice recognition model includes a feature extraction module, an acoustic module having an acoustic model, and a feature matching module connected in sequence; The feature extraction module is used to extract audio features of speech data; the audio features include: time domain information and frequency domain information; the acoustic module is used to determine acoustic features based on the audio features of the speech data; the acoustic features are the output of the last layer or the second to last layer of the acoustic model, and the dimension of the acoustic features is lower than the dimension of the audio features; the feature matching module includes a preset feature database, which includes the acoustic features of multiple different non-standard speech samples and the standard text corresponding to the acoustic features of each non-standard speech sample; the non-standard speech sample is generated based on the standard speech sample and the non-standard text corresponding to the standard speech sample, and the standard speech sample is obtained based on the standard text, the user's voiceprint and surrounding environment information; Execute the instructions corresponding to the standard text.

2. The method according to claim 1, characterized in that The electronic device inputs the first voice data into a first voice recognition model to obtain a standard text corresponding to the first voice data, including: The electronic device inputs the first speech data into a feature extraction module in the first speech recognition model, and after processing by the feature extraction module, obtains a first audio feature; the first audio feature is used to indicate time domain information and frequency domain information of the first speech data; The electronic device inputs the first audio feature into the acoustic module in the first speech recognition model, and after processing by the acoustic module, obtains a first acoustic feature; the first acoustic feature is used to indicate semantic information of the first speech data; The electronic device inputs the first acoustic feature into a feature matching module in the first speech recognition model, and after processing by the feature matching module, obtains a standard text corresponding to the first speech data.

3. The method according to claim 2, characterized in that The electronic device inputs the first acoustic feature into a feature matching module in the first speech recognition model, and obtains a standard text corresponding to the first speech data after processing by the feature matching module, including: The electronic device traverses the feature database through the feature matching module and obtains a target acoustic feature from the feature database whose similarity with the first acoustic feature is greater than a similarity threshold; the electronic device uses the standard text corresponding to the target acoustic feature as the standard text corresponding to the first voice data.

4. The method according to claim 1, wherein The first voice data includes at least one of a repeated syllable, a dragged syllable, or a paused syllable.

5. The method according to claim 1, wherein the electronic device further comprises a non-standard speech generation model, and the method further comprises: The electronic device obtains a first standard voice sample, a first standard text corresponding to the first standard voice sample, and a first non-standard text corresponding to the first standard voice sample; The electronic device inputs the first standard voice sample and the first non-standard text into the non-standard voice generation model at the same time to obtain a first non-standard voice sample; The electronic device inputs the first non-standard speech sample into the first speech recognition model, and obtains acoustic features of the first non-standard speech sample after processing by the feature extraction module and the acoustic module in the first speech recognition model; The electronic device stores the acoustic features of the first non-standard speech sample and the first standard text as a set of data in a feature database; Based on the above steps, other groups of data in the feature database are constructed.

6. The method according to claim 5, characterized in that The non-standard speech generation model includes an audio encoding module, a text encoding module, a feature fusion module and an audio decoding module; The output ends of the audio encoding module and the text encoding module are both connected to the input end of the feature fusion module, and the output end of the feature fusion module is connected to the input end of the audio decoding module.

7. The method according to claim 5 or 6, characterized in that The electronic device simultaneously inputs the first standard voice sample and the first non-standard text into the non-standard voice generation model to obtain the first non-standard voice sample, including: The electronic device inputs the first standard speech sample into the audio encoding module of the non-standard speech generation model to obtain audio features of the first standard speech sample; and inputs the first non-standard text into the text encoding module of the non-standard speech generation model to obtain text features of the first non-standard text; The electronic device simultaneously inputs the audio features of the first standard speech sample and the text features of the first non-standard sample into the feature fusion module of the non-standard speech generation model to obtain the audio features of the first non-standard speech sample; The electronic device inputs the audio features of the first non-standard speech sample into the audio decoding module of the non-standard speech generation model to obtain the first non-standard speech sample.

8. The method according to claim 6, characterized in that The non-standard speech generation model further includes a random variable generation module; the random variable generation module is connected to the input end of the feature fusion module; The random variable generation module is used to generate audio features of random noise.

9. The method according to claim 8, characterized in that The electronic device simultaneously inputs the audio features of the first standard speech sample and the text features of the first non-standard sample into a feature fusion module of the non-standard speech generation model to obtain the audio features of the first non-standard speech sample, including: The electronic device inputs the audio features of the first standard voice sample, the text features of the first non-standard sample, and the audio features of the random noise into the feature fusion module to obtain the audio features of the first non-standard voice sample.

10. An electronic device, characterized in that: include: processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A chip system, characterized in that: The system comprises at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is configured to run a computer program or instruction to execute the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The method comprises a computer program which, when being executed, causes a computer to execute the method according to any one of claims 1 to 9.

Images