Speech recognition method, local speech recognition device and intelligent electronic device

Abstract

The present invention discloses a speech recognition method, a local speech recognition device and an intelligent electronic device. The speech recognition method comprises the following steps: picking up speech data sent by a user; matching the control instruction corresponding to the speech data with the control instruction stored locally; after the control instruction corresponding to the speech data successfully matches the control instruction stored locally, obtaining the confidence of the control instruction stored locally; when the confidence of the control instruction stored locally meets the local recognition condition, executing the control instruction corresponding to the speech data. The present invention can respond to the control instruction corresponding to the speech data sent by the user in a local recognition manner, which is conducive to improving the response efficiency of speech recognition.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition method, a local speech recognition device and an intelligent electronic device. Background Art

[0002] As a new way of information dissemination, voice interaction is gradually being applied to smart homes, touchless control, voice assistants, etc. Voice recognition technology has high requirements on the response speed of voice recognition. Summary of the invention

[0003] The main purpose of the present invention is to propose a speech recognition method, a local speech recognition device and an intelligent electronic device, aiming to realize local recognition of voice commands and improve the response efficiency of speech recognition.

[0004] To achieve the above object, the present invention provides a speech recognition method, which comprises the following steps:

[0005] Pick up the voice data sent by the user;

[0006] After the control instruction corresponding to the voice data successfully matches the locally stored control instruction, obtaining the confidence of the locally stored control instruction;

[0007] When the confidence of the locally stored control instruction meets the local recognition condition, the control instruction corresponding to the voice data is executed.

[0008] Optionally, after the step of picking up the voice data sent by the user, the voice recognition method further includes:

[0009] Sending the voice data to the cloud;

[0010] After the control instruction successfully matches the locally stored control instruction, after the step of obtaining the confidence of the locally stored control instruction, the speech recognition method further includes:

[0011] When the confidence of the locally stored control instruction does not meet the local recognition condition, the control instruction corresponding to the voice data returned by the cloud is obtained, and the control instruction is executed.

[0012] Optionally, after the step of picking up the voice data sent by the user, the voice recognition method further includes:

[0013] Sending the voice data to the cloud;

[0014] Recognizing the voice data to match the control instruction corresponding to the voice data with the control instruction stored locally;

[0015] When the control instruction does not match the locally stored control instruction, a control instruction corresponding to the voice data returned by the cloud is obtained, and the control instruction is executed.

[0016] Optionally, when the confidence level of the locally stored control instruction meets the local recognition condition, the step of executing the control instruction specifically includes:

[0017] Obtaining the confidence of the multiple learning of the locally stored control instructions;

[0018] When the confidence of the locally stored control instruction learned multiple times reaches a preset threshold, it is determined that the confidence of the locally stored control instruction meets the local recognition condition, and the control instruction is executed.

[0019] Optionally, when the confidence level of the locally stored control instruction meets the local recognition condition, the step of executing the control instruction further includes:

[0020] Obtaining a control instruction returned by the cloud corresponding to the voice data;

[0021] When the control instruction corresponding to the voice data returned by the cloud is consistent with the control instruction stored locally, the control instruction is executed.

[0022] Optionally, after the step of sending the voice data to the cloud, the voice recognition method includes:

[0023] Obtaining interactive content returned by the cloud corresponding to the voice data;

[0024] When the interactive content is a media stream, deleting the voice data;

[0025] When the interactive content is a control instruction, the voice data, the control instruction corresponding to the voice data, and the triggering time of the control instruction are stored.

[0026] Optionally, the speech recognition method further includes:

[0027] Classifying all stored control instructions, and when the number of control instructions in any one of the multiple types of control instructions reaches a preset number, learning and verifying the control instructions in that type multiple times;

[0028] When the confidence level of each verification of the control instruction of this type reaches a preset threshold, the control instruction of this type is stored as a locally stored control instruction.

[0029] Optionally, when the confidence of the locally stored control instruction meets the local recognition condition, after the step of executing the control instruction corresponding to the voice data, the voice recognition method further includes:

[0030] The voice data, the control instruction corresponding to the voice data, and the triggering time of the control instruction are stored.

[0031] The present invention further provides a local speech recognition device, which includes: a memory and a processor, wherein a speech recognition program is stored in the memory, and when the speech recognition program is executed by the processor, the steps of the speech recognition method described above are implemented.

[0032] The present invention further proposes an intelligent electronic device, characterized in that the intelligent electronic device comprises: a memory and a processor, wherein a speech recognition program is stored in the memory, and when the speech recognition program is executed by the processor, the steps of the speech recognition method described above are implemented.

[0033] The present invention uses a voice recognition method to pick up voice data sent by a user, and matches the control instruction corresponding to the voice data with the control instruction stored locally, so as to obtain the confidence of the control instruction stored locally after the control instruction corresponding to the voice data successfully matches the control instruction stored locally, so as to execute the control instruction corresponding to the voice data when the confidence of the control instruction stored locally meets the local recognition condition. The present invention can respond to the control instruction corresponding to the voice data sent by the user in a local recognition manner, which is conducive to improving the response efficiency of voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of a flow chart of a first embodiment of a speech recognition method according to the present invention;

[0036] Figure 2 A schematic diagram of a flow chart of a second embodiment of a speech recognition method according to the present invention;

[0037] Figure 3 A schematic diagram of a flow chart of a third embodiment of a speech recognition method according to the present invention;

[0038] Figure 4 A flowchart of a fourth embodiment of a speech recognition method according to the present invention;

[0039] Figure 5 for Figure 1A detailed flow chart of step S300 in an embodiment;

[0040] Figure 6 for Figure 1 A detailed flow chart of another embodiment of step S300;

[0041] Figure 7 A flowchart of a fifth embodiment of a speech recognition method according to the present invention;

[0042] Figure 8 A flowchart of a sixth embodiment of a speech recognition method according to the present invention;

[0043] Fig. 9 A flowchart of a seventh embodiment of a speech recognition method according to the present invention;

[0044] Fig.10 The present invention is a schematic diagram of the terminal structure of the hardware operating environment of the intelligent electronic device involved in the embodiment of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0048] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0049] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0050] The present invention proposes a speech recognition method, which is suitable for distributed intelligent electronic devices with a voice pickup function, such as mobile terminals, remote controls, smart home devices, smart speakers, etc., and can also be applied to voice assistants of intelligent electronic devices such as mobile phones and tablet computers.

[0051] In recent years, smart speaker products have developed rapidly. With the increase of smart home products and the rapid development of 5G communications, smart speakers, as an important human-computer interaction portal of the Internet of Things, have huge development potential and market demand in the future, and their use scenarios are becoming more and more extensive, such as music listening, teleconferencing, Internet of Things control, voice assistants and other functions. At present, there are two ways to recognize voice commands of smart electronic devices. One is to send it to the cloud for processing, and the other is to simply process and analyze the command locally. Cloud processing has high requirements for the real-time performance of the network, and low and unstable network speeds have a very bad experience for the speaker. Fixed local command processing can only recognize limited commands and is not flexible enough. Both methods will make smart electronic devices have low response efficiency.

[0052] Reference Figures 1 to 9 In one embodiment of the present invention, the speech recognition method comprises the following steps:

[0053] Step S100, picking up voice data sent by the user;

[0054] In this embodiment, the sound information sent by the user can be picked up by a microphone set on an intelligent electronic device, or it can be other voice signal picking devices connected to external devices of the intelligent electronic device, such as smart bracelets, etc. The smart bracelet can only be used as a voice signal picking device to send the collected voice signal to the intelligent electronic device, and the intelligent electronic device performs subsequent processing on the received voice data, or, in order to improve the clarity of the picked-up voice data, the user can use a device specifically used to pick up voice data to pick up voice data and store it on the intelligent electronic device. When performing voice control, it is only necessary to obtain the stored voice data under a specific path. When the microphone on the intelligent electronic device picks up voice data, the sound (audio data) is received via the microphone, and the output end of the microphone can also be connected to an audio processor, which can process such voice signals into audio data, specifically, analog audio data can be converted into digital voice data, that is, the audio processor can convert the sound information into digital information first. The processed audio (voice) data can be sent to the cloud in a format sent by wireless broadband Internet such as WIFI, 4G or 5G networks. The processed audio (voice) data can also be converted into a format output that can be sent to a mobile communication base station via a radio frequency unit in the case of a telephone call mode.

[0055] Reference Figure 2 Step S200: identifying a locally stored control instruction corresponding to the voice data according to the voice data, and obtaining a confidence level of the locally stored control instruction. This step specifically includes:

[0056] Step S210: Match the control instruction corresponding to the voice data with the control instruction stored locally;

[0057] In this embodiment, the interaction of the smart speaker is divided into two categories, one is content services, such as broadcasting news and music. The other is control instructions, such as turning on Bluetooth, adjusting the volume, or related to smart furniture, such as turning off the bedroom light and turning on the TV. For the former content service, after the user's voice data is sent to the cloud, the cloud server sends back a media stream. For the latter control instruction, the cloud returns a control string. In this embodiment, the voice data sent by the user can be sent to the cloud. And every time a voice data is sent to the cloud, the local saves the voice data at the same time and waits for the cloud to return the content of the voice data. If the cloud returns a media stream, the local deletes the voice data. If the cloud returns a control instruction, the local saves the voice data and the control instruction, as well as the trigger time, in the form of a table, so as to associate the voice data with the control instruction, form a mapping relationship, and generate a control instruction-voice data table, which can be specifically represented by the following table. Among them, the voice data can be represented by binary numbers, and the control instruction is represented by a string that the program can run.

[0058] Control instructions Voice data Trigger time Open bedroom light 0101010010101001… 2021-06-14 Close bedroom light 1101010010101001… 2021-06-14 Open TV 1111010010101001… 2021-06-15 Close microwave 0001010010101001… 2021-06-16 …… …… ……

[0059] For example, when the user sends the voice data of "turn on the bedroom light", the voice data stored locally is 0101010010101001…. After it is sent to the cloud, what is returned from the cloud is Open bedroom light. Similarly, when the user sends the voice data of "turn off the bedroom light", the voice data stored locally is 1101010010101001…. After it is sent to the cloud, what is returned from the cloud is Close bedroom light.

[0060] It is understandable that the voice features of the voice data picked up are different depending on the user, and there will be at least differences in user timbre features (such as children, the elderly, the middle-aged, the young, men, women, etc.), as well as differences in language (such as Mandarin, dialects such as Cantonese, Hakka, Tibetan, foreign languages ​​such as English, French, Japanese, etc.). When different users send their own voice data to control the same target device, for example, when a user sends voice data of "turn on the bedroom light" with different voice features such as Mandarin, dialects, and foreign languages, the voice data picked up will be different, and these voice data are sent to the cloud, and the cloud recognizes that these different voice data correspond to the same control instruction. When the local receives the same control instruction returned according to different voice data, the different voice data corresponding to the same control instruction can be associated, so that a many-to-one mapping relationship can be formed between the voice data sent by the user and the control instruction.

[0061] In addition, the intelligent electronic device will store all voice data and the control instructions corresponding to the voice data returned from the cloud in the form of a table, thereby forming a control instruction-voice database. When the voice data sent by the user again is picked up, the voice data may correspond to the first control instruction sent by the user, such as the first voice control of each functional module in the intelligent electronic device, or the user's newly added electrical equipment, such as the first use of voice control to control a newly added microwave oven. It may also be a control instruction corresponding to voice data that has been sent many times, such as common controls for daily electrical equipment, such as turning on / off Bluetooth, adjusting the speaker volume, turning on the TV and turning on the air conditioner. Alternatively, the voice data may also be a content service that will not be stored by the user, such as broadcasting real-time news, or listening to the radio.

[0062] Step S220: Identify the control instruction corresponding to the currently picked-up voice data sent by the user from the locally stored control instructions:

[0063] Specifically, the voice data currently sent by the user can be compared and matched by looking up a table in the control instruction-voice database to determine whether the voice data and control instructions currently sent by the user have been stored. When it is identified that the voice data currently sent by the user does not match any stored voice data, it can be determined that the user has sent an initial control instruction or a content service. When it is identified that the voice data currently sent by the user matches one of the stored voice data, it can be determined that the characteristics of the voice data currently sent by the user are the same as the voice characteristics of the stored voice data, and the control instruction corresponding to the voice data can be obtained by reading the table to identify the control instruction corresponding to the voice data from the stored voice data and the control instruction associated with the voice data.

[0064] Step S230: after the control instruction successfully matches the locally stored control instruction, obtain the confidence of the locally stored control instruction;

[0065] In this embodiment, after all control instructions and voice data are stored locally, the control instructions in the control instruction-voice data table that has been sorted and collected can be classified, and a training model (or learning model) is used to learn each type of control instruction to obtain the confidence of each type of control instruction. Among them, a type of control instruction represents a specific control instruction, such as "Open bedroom light". When classifying, all "Openbedroom light" in the table are classified into one category and their number is counted. Similarly, other control instructions such as the "Open TV" control instruction corresponding to "turn on the TV" can be classified into one category respectively. All control instructions are learned through the learning model, and the learning model is saved to complete the learning process to obtain the confidence of each type of control instruction. In actual application, the intelligent electronic device will continuously accumulate and store the voice data issued by the user each time and the control instruction corresponding to the voice data, and once the control instruction reaches the learning trigger condition, the control instruction will be learned. In this way, after a period of use, as the number of control instruction learning samples increases, the confidence of the control instruction will also increase until the confidence reaches the conditions required for local recognition, thereby completing the learning process. In this way, users can issue their own voices as voice control instructions according to their own language habits, compared to setting standard voices as fixed local recognition-triggered control instructions at the factory, or entering fixed voice data by users as local recognition-triggered control instructions, which results in limited local recognition control instructions and low flexibility. The present invention can dynamically learn based on the voice data issued by the user, and can add voice data and control instructions corresponding to the voice data in real time without limiting the user group and user language, greatly improving the flexibility of voice recognition.

[0066] Step S300: When the confidence level of the locally stored control instruction meets the local recognition condition, execute the control instruction.

[0067] In this embodiment, the locally stored control instructions can be learned by using a learning model for a class of control instructions that meet the learning trigger conditions in the control instruction-voice database. When the confidence of the learned control instructions meets the conditions for local recognition, the confidence of the learned control instructions is determined to reflect the degree of credibility of voice recognition. Therefore, the confidence of the learned control instructions can reflect that the voice data currently input by the user has a high reliability of semantic understanding results that meet the local recognition conditions, and the control instructions corresponding to the currently input voice data can be executed.

[0068] The control instruction can be applied to the self-control of smart speakers, mobile phones, etc., such as turning on and off smart speakers, playing local music, adjusting the volume, etc. After completing local recognition, the corresponding functional modules in the smart speakers can be controlled to respond to the control instruction, such as opening the corresponding application or controlling the corresponding hardware circuit.

[0069] The control instruction can also be used to control the target device, for example, in a control terminal such as a remote controller or a control terminal, and a connection can be established between the control terminal and the target device, for example, communication can be achieved through wireless communication methods such as Bluetooth, WiFi, Zigbee, etc., and the air conditioner, TV, smart light, etc. can be controlled through the control terminal. When used to control the target device, after picking up the control voice data sent by the user, it is compared with the stored voice data to obtain the corresponding control instruction, and when it is determined that the control instruction meets the local recognition condition, the control instruction is sent to the corresponding target device so that the target device executes the control action corresponding to the control instruction. Intelligent electronic devices can also be used in an Internet of Things system, which includes an Internet of Things server. The Internet of Things server can be connected to electrical appliances in each smart home. Users can send control instructions to intelligent electronic devices through voice, and after being locally recognized by the intelligent electronic device, the control instructions are directly sent to the Internet of Things server through a network such as 4G or 5G, so that the Internet of Things server forwards the control instructions to the corresponding target device, and finally realizes the control of the target device. Or smart electronic devices, such as smart speakers, can also be connected to terminals such as mobile phones. After the smart speaker generates a control instruction, it is sent to the terminal via Bluetooth, etc. The terminal then sends the control instruction to the Internet of Things server through a network such as WiFi, 4G, 5G, etc. Finally, the Internet of Things server sends the control instruction to the corresponding electrical equipment to achieve control of the target device.

[0070] The present invention uses a voice recognition method to pick up voice data sent by a user, and matches the control instruction corresponding to the voice data with the control instruction stored locally, so as to obtain the confidence of the control instruction stored locally after the control instruction corresponding to the voice data successfully matches the control instruction stored locally, so as to execute the control instruction corresponding to the voice data when the confidence of the control instruction stored locally meets the local recognition condition. The present invention can respond to the control instruction corresponding to the voice data sent by the user in a local recognition manner, which is conducive to improving the response efficiency of voice recognition.

[0071] Reference Figure 3 In one embodiment, after the step of picking up the voice data sent by the user, the voice recognition method further includes:

[0072] Step S110, sending the voice data to the cloud;

[0073] After the step S200 of identifying the locally stored control instruction corresponding to the voice data according to the voice data and acquiring the confidence of the locally stored control instruction, the voice recognition method further includes:

[0074] Step S400: When the confidence of the locally stored control instruction does not meet the local recognition condition, obtain the control instruction corresponding to the voice data returned by the cloud, and execute the control instruction.

[0075] In this embodiment, when the confidence of the control instruction after learning meets the conditions for realizing local recognition, voice control can be performed by local recognition, and the instruction can be directly executed, while ignoring the cloud parsing process, that is, receiving the control instruction returned by the cloud but not executing it accordingly, which is conducive to improving the response speed of voice control. However, when the confidence of the control instruction after learning does not meet the conditions for realizing local recognition, it is determined that the confidence of the control instruction that has been learned cannot reflect the credibility of voice recognition. Therefore, the confidence of the control instruction that has been learned can reflect that the user's current input voice data has a low reliability of semantic understanding results that meet the local recognition conditions, and the control instruction corresponding to the currently input voice data cannot be executed accordingly. That is, the conditions for responding to the control instruction corresponding to the voice data issued by the user by using the local recognition method have not been met. At this time, the control instruction can be executed by sending the voice data to the cloud and obtaining the control instruction corresponding to the voice data. In this way, the compatibility of local recognition and cloud recognition can be achieved, and when the picked-up voice data fails to meet the local recognition conditions, the voice data output by the user is responded to by cloud recognition. It can be understood that after picking up the voice data sent by the user and obtaining a digital signal after the audio processor processes the voice data, the audio processor can send the digital voice data to the local voice recognition processor and the cloud at the same time, so that the voice recognition processor and the cloud can process the voice data at the same time, realizing local recognition and cloud recognition synchronously. When the confidence of the control instruction stored locally does not meet the local recognition conditions, the control instruction from the cloud can be obtained in time and the control instruction can be responded to quickly, which can ensure the accuracy of voice recognition and response efficiency.

[0076] Reference Figure 4 In one embodiment, after the step of picking up the voice data sent by the user, the voice recognition method further includes:

[0077] Step S110, sending the voice data to the cloud;

[0078] Step S210: Match the control instruction corresponding to the voice data with the control instruction stored locally;

[0079] Step S500: When the control instruction does not match the locally stored control instruction, obtain the control instruction corresponding to the voice data returned by the cloud, and execute the control instruction.

[0080] In this embodiment, when the voice data currently sent by the user matches one of the stored voice data, for example, the user sends a control instruction for the first time, or a voice control sent for different users, or a voice control sent by the user in different languages, and the control instruction corresponding to the voice data can be obtained by reading the table. At this time, if the characteristics of the control voice data currently sent by the user fail to match the voice characteristics of the stored voice data, it can be determined that the control instruction is also not stored, or that no association or mapping relationship has been established between the voice data and the control instruction, resulting in the control instruction corresponding to the voice data not matching the stored control instruction. At this time, the voice data can be sent to the cloud to realize cloud recognition, and the control instruction corresponding to the voice data can be executed. In this process, the voice data and the control instruction corresponding to the voice data will still be stored locally, and the voice data and the control instruction corresponding to the voice data will be used as learning samples. Once any type of control instruction meets the learning conditions, and the confidence of the control instruction meets the conditions required for local recognition, when the user sends the voice data again, voice control can be achieved through local recognition. The present invention can update the voice data-control instruction library in real time, without being limited to specific users and specific terms, which is conducive to improving the flexibility of local voice recognition control. The present invention can also solve the problem of relying entirely on cloud recognition, which leads to low real-time performance and affects the response speed when the network speed is low and unstable.

[0081] Reference Figure 5 In one embodiment, step S300, when the confidence of the locally stored control instruction meets the local recognition condition, the step of executing the control instruction specifically includes:

[0082] Step S310, obtaining the confidence of the multiple learning of the locally stored control instruction;

[0083] Step S320: When the confidence of the locally stored control instruction learned multiple times reaches a preset threshold, determine that the confidence of the locally stored control instruction meets the local recognition condition, and execute the control instruction.

[0084] In this embodiment, the confidence of the control instruction is used to indicate the degree of credibility of local voice recognition. The greater the confidence, the higher the degree of credibility. Conversely, the smaller the confidence, the lower the degree of credibility. During the use of the intelligent electronic device, each time a voice data is picked up and a user voice data is sent to the cloud, when the control instruction is returned in the cloud, the voice data and the control instruction corresponding to the voice are saved in the form of a smart speaker conference table. As the user time increases, the historical voice data stored locally will also increase. After any control instruction reaches the learning trigger condition, the control instruction will be learned multiple times. During the multiple learning processes, when the confidence of each learning of the control instruction reaches the preset threshold, for example, when it reaches 99%, it can be determined that the instruction learning is completed and can be used for local recognition. In this way, local recognition of commonly used user instructions can be achieved, which not only meets the requirements of rapid response, but also can continuously add control instructions that can be used for local recognition, so that the intelligent electronic device can flexibly recognize various control instructions locally, and can achieve voice control of intelligent electronic devices in the absence of a network and cloud recognition.

[0085] Reference Figure 6 In one embodiment, step S300, when the confidence of the locally stored control instruction meets the local recognition condition, the step of executing the control instruction further includes:

[0086] Step S330: Obtain the control instruction returned by the cloud corresponding to the voice data;

[0087] Step S340: When the control instruction corresponding to the voice data returned by the cloud is consistent with the control instruction stored locally, execute the control instruction.

[0088] It can be understood that in order to improve the response speed of the intelligent electronic device, this embodiment will perform voice recognition and cloud recognition simultaneously. Specifically, after picking up the voice data sent by the user, the control instruction corresponding to the voice data will be matched with the stored control instruction successfully, that is, the voice data will be determined to be characterized as a control instruction. In this process, the voice data will be uploaded to the cloud synchronously, and the cloud will recognize the voice data, thereby returning a control instruction corresponding to the voice data to the intelligent electronic device. After local recognition and determining that the confidence of the control instruction stored locally meets the local recognition condition, the control instruction whose confidence meets the local recognition condition can be compared with the control instruction returned by the cloud. When the control instruction recognized locally is the same as the control instruction recognized by the cloud, that is, when it is consistent, it can be further determined that the reliability of local recognition is high, so as to execute the control instruction. When the control instruction recognized locally is inconsistent with the control instruction recognized by the cloud, it can be determined that the reliability of local recognition is low, and the control instruction returned by the cloud is executed. Of course, in other embodiments, when the control instruction recognized locally is inconsistent with the control instruction recognized by the cloud, an error can also be reported to the user and the cloud to detect whether the voice control fails.

[0089] Reference Figure 7 In one embodiment, the speech recognition method further includes:

[0090] Step S610: Obtain the interactive content corresponding to the voice data returned by the cloud;

[0091] Step S620: when the interactive content is a media stream, delete the voice data;

[0092] Step S630: When the interaction content is a control instruction, the voice data, the control instruction corresponding to the voice data, and the triggering time of the control instruction are stored.

[0093] In this embodiment, during the use of the intelligent electronic device, whether in the initial use stage of the intelligent electronic device or after a period of use, the user uses voice to control the intelligent electronic device many times, so that the control instructions corresponding to the stored historical voice data meet the local recognition conditions. The stage, the intelligent electronic device sends each received voice data to the cloud. And when the interactive content corresponding to the voice data is a control instruction, the voice data and the control instruction will be stored. These voice data and the control instructions corresponding to the voice data will be collected to form training samples for local learning. When the interactive content corresponding to the voice data is a media stream, the processor in the intelligent electronic device will parse the media stream to obtain the audio data in the media stream, and convert it into an audio signal and output it as sound, thereby broadcasting news, music, broadcasts, etc.

[0094] Reference Figure 8 In one embodiment, the speech recognition method further includes:

[0095] Step S710, classify all stored control instructions, and when the number of control instructions in any one of the multiple types of control instructions reaches a preset number, perform multiple learning and verification on the control instructions in that type;

[0096] Step S720: When the confidence level of each verification of the control instruction of this type reaches a preset threshold, the control instruction of this type is stored as a locally stored control instruction.

[0097] In order to improve the efficiency of speech recognition and establish a local recognition mechanism, this embodiment can use a multi-classification learning model to classify each control instruction. The multi-classification learning model uses the collected historical speech data to train each control instruction that meets the learning conditions, so as to obtain the confidence of each learning in the multiple learning process of each control instruction that meets the learning conditions. Specifically, the collected speech data-control instruction table can be sorted, and the data in the table can be classified in the form of control instructions. When the number of a category of control instructions in many control instructions reaches a preset number, for example, the number of "Open bedroom light" in the commonly used control instructions reaches more than 100, specifically 100, 120, 150, 200, etc., it can be determined that the control instruction "Open bedroom light" meets the conditions for local learning, and the control instruction is learned, or called training. When the number of other control instructions does not meet the learning conditions, the control instructions continue to be collected and accumulated until the learning conditions are met. Furthermore, in actual application, a preset learning cycle can be set, such as setting a fixed time every day, such as at night, or other times when users use less, or counting all collected voice data samples and learning each time the sample reaches a certain number interval.

[0098] Specifically, all collected control instructions can be shuffled, and then divided into four parts of similar size, and then learning can begin. During learning, three data are randomly taken from the four divided control instructions, and the control instructions of these three control instructions are used as outputs, and the voice data corresponding to these three control instructions are used as inputs, and supervised learning is performed. The remaining one is verified. After each round of learning and verification is completed, all control instructions are re-shuffled and repeatedly learned and verified, and learning is performed four times. If the average confidence of the control instructions learned in each verification set (the number of control instructions reaches the preset number) reaches more than 99%, it is considered that the instruction learning is completed and can be locally recognized. Refer to the following table, which is a confidence table of the control instruction "Open bedroom light" after 4 learnings and each learning in an exemplary embodiment. It can be seen from the table below that when the average confidence of the control instruction "Open bedroom light" reaches more than 99%, it can be determined that the instruction has been learned and can be locally recognized. That is, when the user sends the voice data of "turn on the bedroom light" again, the control instruction can be identified as "Openbedroom light" from the historical voice data. By looking up the table, it is known that the confidence of the control instruction "Open bedroom light" learned multiple times is more than 99%, then it can be determined that the confidence of the control instruction meets the local recognition conditions. At this time, the smart speaker can send the control instruction to the bedroom light through wireless communication, Internet of Things network, etc., thereby turning on the bedroom light.

[0099] Control instructions The first few tests Confidence Open bedroom light 1 99.2 Open bedroom light 2 99.8 Open bedroom light 3 99.5 Open bedroom light 4 99.7

[0100] Similarly, a multi-classification learning model can be used to learn each control instruction that meets the learning conditions and its corresponding voice data one by one, and the multi-classification learning model can be saved in the intelligent electronic device to complete the learning process of the user's commonly used voice control commands.

[0101] It is understandable that as the number of times users use the system increases, the collected historical voice data will continue to accumulate. After long-term use and repeated training of the model, the confidence of each control command can be improved iteratively so that the confidence of the control command reaches the local recognition condition, thereby achieving a recognition rate of 99% for users' commonly used commands, greatly improving the response efficiency of the commands.

[0102] Reference Fig. 9 In one embodiment, the speech recognition method further includes:

[0103] Step S730: store the voice data, the control instruction corresponding to the voice data, and the triggering time of the control instruction.

[0104] In this embodiment, when the control instruction corresponding to the picked-up voice data meets the recognition condition, the control instruction can be completed through local recognition and directly executed. At the same time, the local can also directly store the voice data and the control instruction corresponding to the voice data, as well as the trigger time of the control instruction. And / or, store the voice data and the control instruction corresponding to the voice data returned by the cloud, as well as the trigger time of the control instruction.

[0105] In some embodiments, when the control instructions corresponding to some commonly used and high-frequency voice data are issued by the user multiple times and are locally recognized and executed, the locally stored voice data and the control instructions corresponding to the voice data can be marked. After the local recognition is successful, when the voice data is picked up again (it can also be set to be picked up after a preset number of times), the control instructions associated with the voice data are directly obtained and executed, and the actions of step S300 and step S400 are no longer executed, so as to further improve the response efficiency. For example, "turn on the bedroom light", "turn off the bedroom light", "turn on the TV", "turn off the TV", etc., when the user performs voice control on the commonly used and high-frequency electrical equipment such as the TV and bedroom light, after the intelligent electronic device receives the control voice input by the user, it compares it with the stored voice data, obtains the corresponding control instruction, and after it is a marked control instruction, it can directly execute the control action corresponding to the control instruction.

[0106] The present invention also provides a local speech recognition device, which includes: a memory and a processor. The memory stores a speech recognition program, and when the speech recognition program is executed by the processor, the steps of the speech recognition method described above are implemented.

[0107] In this embodiment, the memory can be used to store software programs and various data. The data storage area can store data created according to the use of the smart speaker (such as voice data-control instruction table, user control habits, etc.). The processor runs or executes the software program and / or module stored in the memory, and calls the data stored in the memory to execute the speech recognition method as described above. The local speech recognition device can be applied to intelligent electronic devices, and can be connected to the central processing unit of the intelligent electronic device so that the central processing unit performs the corresponding function according to the control instruction output by the local speech recognition device. The local speech recognition device can also directly control the operation of the circuit module in the intelligent electronic device to implement the corresponding function of the intelligent electronic device.

[0108] The present invention further provides an intelligent electronic device, which includes: a memory and a processor, wherein a speech recognition program is stored in the memory, and when the speech recognition program is executed by the processor, the steps of the speech recognition method described above are implemented.

[0109] The intelligent electronic device may be a smart speaker, a mobile terminal, a remote controller, etc.

[0110] In one embodiment, the processing of speech recognition can be implemented by a local speech recognition processor, which can be a central processing unit of an intelligent electronic device or a processor specifically used for speech recognition. At the same time, a memory can be set in the intelligent electronic device, and a program for implementing the above method can be stored in the memory, and the above control instruction-speech database can also be stored. When the local speech recognition processor is the control center (CPU) of the intelligent electronic device, various interfaces and lines are used to connect the various parts of the entire intelligent electronic device, and the various functions and processing data of the intelligent electronic device are executed by running or executing the software program and / or module stored in the memory, and calling the data stored in the memory, so as to respond to the control instruction. The processor can also be a single-chip microcomputer, a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, etc.

[0111] The memory 105 can be used to store software programs and various data. The memory 105 may mainly include a program storage area and a data storage area. Optionally, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, etc.), etc.; the data storage area may store data created according to the use of the smart speaker (such as a voice data-control instruction table, user control habits, etc.), etc. Optionally, the memory 105 may include a high-speed random access memory 105, and may also include a non-volatile memory 105, such as at least one disk storage 105 piece, a flash memory device, or other volatile solid-state storage 105 piece. The intelligent electronic device of the present invention can respond to the control instruction corresponding to the voice data issued by the user in a local recognition manner in the intelligent electronic device, which is conducive to improving the response efficiency of voice recognition.

[0112] The present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of speech recognition as described above are implemented. In the embodiments of the intelligent electronic device and the computer-readable storage medium provided by the present invention, all technical features of the above-mentioned speech recognition embodiments are included, and the expansion and explanation content of the specification are basically the same as those of the above-mentioned method embodiments, and will not be repeated here.

[0113] Reference Fig.10 The terminal of the embodiment of the present invention may be a PC, or a smart phone, a tablet computer, an e-book reader, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a portable computer, or other mobile terminal devices with display functions. Figure 1 The terminal may include a processor 101 (e.g., a CPU), a communication bus 102, a user interface 103, a network interface 104, and a memory 105. The communication bus 102 is used to realize the connection and communication between these components; the user interface 103 may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface 104 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface); the memory 105 may be a high-speed RAM memory 105, or a stable memory 105 (non-volatile memory), such as a disk memory 105, and the memory 105 may optionally be a storage device independent of the aforementioned processor 101.

[0114] Those skilled in the art will understand that Fig.10 The terminal structure of the intelligent electronic device hardware operating environment shown in the figure does not constitute a limitation on the intelligent electronic device of the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0115] The above descriptions are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A speech recognition method, characterized in that: The speech recognition method comprises the following steps: Pick up the voice data sent by the user; identifying, according to the voice data, a locally stored control instruction corresponding to the voice data, and obtaining a confidence level of the locally stored control instruction; When the confidence of the locally stored control instruction meets the local recognition condition, executing the control instruction corresponding to the voice data; After the step of picking up the voice data sent by the user, the voice recognition method further includes: Sending the voice data to the cloud; Recognizing the voice data, and when determining that the voice data is voice data of a control instruction, matching the control instruction corresponding to the voice data with the control instruction stored locally; When the control instruction does not match the locally stored control instruction, obtain the control instruction corresponding to the voice data returned by the cloud, store the voice data and the control instruction corresponding to the voice data locally, and execute the control instruction.

2. The speech recognition method according to claim 1, characterized in that: After the step of picking up the voice data sent by the user, the voice recognition method further includes: Sending the voice data to the cloud; After the step of identifying the locally stored control instruction corresponding to the voice data according to the voice data and obtaining the confidence of the locally stored control instruction, the voice recognition method further includes: When the confidence of the locally stored control instruction does not meet the local recognition condition, the control instruction corresponding to the voice data returned by the cloud is obtained, and the control instruction is executed.

3. The speech recognition method according to claim 1, wherein: When the confidence level of the locally stored control instruction meets the local recognition condition, the step of executing the control instruction specifically includes: Obtaining the confidence of the multiple learning of the locally stored control instructions; When the confidence of the locally stored control instruction learned multiple times reaches a preset threshold, it is determined that the confidence of the locally stored control instruction meets the local recognition condition, and the control instruction is executed.

4. The speech recognition method according to claim 3, characterized in that: When the confidence level of the locally stored control instruction meets the local recognition condition, the step of executing the control instruction further includes: Obtaining a control instruction returned by the cloud corresponding to the voice data; When the control instruction corresponding to the voice data returned by the cloud is consistent with the control instruction stored locally, the control instruction is executed.

5. The speech recognition method according to claim 1, wherein: The speech recognition method comprises: The sending of the voice data to the cloud; Obtaining interactive content returned by the cloud corresponding to the voice data; When the interactive content is a media stream, deleting the voice data; When the interactive content is a control instruction, the voice data, the control instruction corresponding to the voice data, and the triggering time of the control instruction are stored.

6. The speech recognition method according to claim 5, characterized in that: The speech recognition method further comprises: Classifying all stored control instructions, and when the number of control instructions in any one of the multiple types of control instructions reaches a preset number, learning and verifying the control instructions in that type multiple times; When the confidence level of each verification of the control instruction of this type reaches a preset threshold, the control instruction of this type is stored as a locally stored control instruction.

7. The speech recognition method according to any one of claims 1 to 6, characterized in that: When the confidence of the locally stored control instruction meets the local recognition condition, after the step of executing the control instruction corresponding to the voice data, the voice recognition method further includes: The voice data, the control instruction corresponding to the voice data, and the triggering time of the control instruction are stored.

8. A local speech recognition device, characterized in that: The local speech recognition device comprises: a memory and a processor, wherein a speech recognition program is stored in the memory, and when the speech recognition program is executed by the processor, the steps of the speech recognition method according to any one of claims 1 to 7 are implemented.

9. An intelligent electronic device, characterized in that: The intelligent electronic device comprises: a memory and a processor, wherein a speech recognition program is stored in the memory, and when the speech recognition program is executed by the processor, the steps of the speech recognition method according to any one of claims 1 to 7 are implemented.

Images