Device control method, system, device and storage medium based on voice recognition
By extracting and matching voice features in voice chip devices and generating device control instructions, the problem of low recognition accuracy caused by the limitation of voice chip resources is solved, and higher voice control accuracy and multilingual applicability are achieved.
Patent Information
- Application Number
- CN202411053706.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-02
AI Technical Summary
In the existing voice chips, due to the limitation of local resources, the recognition model supports fewer entries, resulting in lower accuracy of voice control devices.
The device control method based on voice features is adopted to match voice features in the local storage space of the target device, generate an instruction set for device control, and extract voice features in offline mode to improve recognition accuracy, which is suitable for application scenarios of any language.
It improves the accuracy of voice control devices, can recognize more voices, is suitable for multilingual scenarios, and enhances the control accuracy and flexibility of the device.
Smart Images

Figure CN118588061B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of speech processing technology, and in particular to a device control method, system, device and storage medium based on speech recognition. Background Art
[0002] Voice-controlled devices are widely used in scenarios such as smart life, smart home and smart driving because they simplify user operations. Currently, voice chips are used to convert speech to text to realize voice-controlled devices. However, due to the limited local resources of voice chips, the recognition model supports fewer entries and the recognition accuracy is low, resulting in low accuracy of voice-controlled devices. Summary of the invention
[0003] Based on this, it is necessary to address the technical problem that the existing technology uses voice chips to convert speech to text to realize voice-based control devices. Due to the limited local resources of the voice chip, the recognition model supports fewer entries and the recognition accuracy is low, resulting in low accuracy of the voice control device. A device control method, system, device and storage medium based on speech recognition are proposed.
[0004] In a first aspect, a device control method based on voice recognition is provided, the method is applied to a target device, the target device includes a voice chip, and the method includes:
[0005] When the working mode of the target device is an offline recognition mode, obtaining a target voice feature, wherein the target voice feature is a voice feature extracted by the voice chip or the mobile terminal based on first voice data, and the first voice data is voice data input by a user through the target device or the mobile terminal;
[0006] According to the target voice feature, matching the voice feature in a voice feature library in the local storage space of the target device to obtain a first matching result;
[0007] A first instruction set is generated according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction.
[0008] Furthermore, the method further comprises:
[0009] Acquire first pending-change data, wherein the first pending-change data includes a function identifier and a voice feature;
[0010] updating the voice feature library according to the first data to be updated;
[0011] The function identifier in the first data to be updated is data input by a user through the mobile terminal, and the voice feature in the first data to be updated is a voice feature extracted by the voice chip or the mobile terminal based on learning voice data.
[0012] Furthermore, the method further comprises:
[0013] Obtaining the networking status of the target device and the cloud platform;
[0014] If the networking state is networking, the working mode is determined to be an offline identification mode;
[0015] If the networking state is not connected to the network, the working mode is determined to be the offline identification mode;
[0016] When the working mode is the offline recognition mode, a target text is obtained, wherein the target text is a text obtained by the cloud platform converting the second voice data based on the TTS technology, and the second voice data is voice data input by the user through the target device or the mobile terminal;
[0017] Obtain a second matching result, wherein the second matching result is data obtained by matching text in a text library in the executing device according to the target text by an executing device, and the executing device is any one of the target device, the mobile terminal, and the cloud platform;
[0018] A second instruction set is generated according to the second matching result, wherein the second instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction.
[0019] Furthermore, the method further comprises:
[0020] Acquire second pending-change data, wherein the second pending-change data includes a function identifier and a description text;
[0021] updating a text library in a local storage space of the target device according to the second data to be updated;
[0022] The second matching result is data obtained by the target device matching text in the text library in the target device according to the target text.
[0023] Furthermore, the method further comprises:
[0024] Acquire third voice data sent by the cloud platform, wherein the third voice data is audio data synthesized by the cloud platform according to first information, the first information is information input by a user through the target device or the mobile terminal, and the first information includes: timbre parameters, sound speed parameters, volume parameters and text to be synthesized;
[0025] The third voice data is stored in the local storage space of the target device, wherein the third voice data in the local storage space of the target device is used to respond to the reply instruction.
[0026] Furthermore, the method further comprises:
[0027] Get download request;
[0028] According to the download request, fourth voice data is obtained from the first platform, wherein the fourth voice data is audio data synthesized by the cloud platform according to second information, the second information is information input by a user through the second platform, and the second information includes: timbre parameters, speed parameters, volume parameters and text to be synthesized;
[0029] The fourth voice data is stored in the local storage space of the target device, wherein the fourth voice data in the local storage space of the target device is used to respond to the reply instruction.
[0030] Furthermore, the first voice data is voice data of a human or an animal.
[0031] In a second aspect, a device control system based on voice recognition is provided, the system comprising: a target device and a mobile terminal, the target device and the mobile terminal are communicatively connected, the target device comprises: a voice chip, and the target device is configured to implement the steps of the device control method based on voice recognition as described in any one of the first aspects.
[0032] In a third aspect, a target device is provided, which includes a voice chip, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the voice chip is communicatively connected to the processor, and when the processor executes the computer program, the steps of the device control method based on voice recognition described in any one of the first aspects are implemented.
[0033] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned device control method based on voice recognition are implemented.
[0034] The device control method, system, device and storage medium based on voice recognition of the present application obtain target voice features when the working mode of the target device is an offline recognition mode, wherein the target voice features are voice features extracted by the voice chip or the mobile terminal based on first voice data, and the first voice data is voice data input by the user through the target device or the mobile terminal; according to the target voice features, the voice features are matched in a voice feature library in the local storage space of the target device to obtain a first matching result; according to the first matching result, a first instruction set is generated, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction. The existing voice-to-text conversion method uses a voice chip to implement voice-based control of the device, while the present application extracts voice features and controls the device based on the voice features. When the local resources of the voice chip are limited, the model that extracts voice features can recognize more voices than the voice-to-text model, and the recognition accuracy is higher, thereby improving the accuracy of the voice control device. In addition, the present application controls the device based on the extracted voice features, so that voice data of any language can be used as the first voice data, thereby making the present application suitable for application scenarios of any language and multiple languages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0036] in:
[0037] Figure 1 A diagram of an application environment of a device control method based on voice recognition in one embodiment;
[0038] Figure 2 is a flow chart of a device control method based on voice recognition in one embodiment;
[0039] Figure 3 A structural block diagram of a device control system based on speech recognition in one embodiment;
[0040] Figure 4 FIG. 4 is a structural block diagram of a target device in an embodiment. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] The device control method based on voice recognition provided by the embodiment of the present invention can be applied in Figure 1 The application environment includes: a target device 1, a mobile terminal 2, a cloud platform 3 and a controllable device 4, the mobile terminal 2 is connected to the target device 1 through wireless or wired communication, the cloud platform 3 is connected to the target device 1 and the mobile terminal 2 through communication, and the target device 1 includes a voice chip 11. The voice chip 11 is used for data processing and algorithm capture.
[0043] The target device 1 is configured to implement the following method: when the working mode of the target device is the offline recognition mode, obtaining the target voice feature, wherein the target voice feature is the voice feature extracted by the voice chip or the mobile terminal based on the first voice data, and the first voice data is the voice data input by the user through the target device or the mobile terminal; according to the target voice feature, matching the voice feature in the voice feature library in the local storage space of the target device to obtain a first matching result; generating a first instruction set according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction and a device control instruction, and the target device controls the controllable device corresponding to the device control instruction through the device control instruction. The existing voice-to-text conversion method uses a voice chip to implement voice-based control of the device, while the present application extracts voice features and controls the device based on the voice features. When the local resources of the voice chip are limited, the model that extracts voice features can recognize more voices than the voice-to-text model, and the recognition accuracy is higher, thereby improving the accuracy of the voice control device. In addition, the present application controls the device based on the extracted voice features, so that voice data of any language can be used as the first voice data, thereby making the present application suitable for application scenarios of any language and multiple languages at the same time.
[0044] The target device 1 can be various personal computers, laptops, smart phones, tablets, portable wearable devices, smart gateways and smart home devices.
[0045] It can be understood that the target device 1 also includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the voice chip is communicatively connected to the processor.
[0046] Optionally, the target device 1 further includes a voice input component, which is communicatively connected to the processor. The voice input component may be a microphone. The user inputs voice data to the target device 1 through the voice input component.
[0047] Optionally, the target device 1 further includes a wireless communication component, through which the target device 1 is wirelessly connected to the mobile terminal 2 and the cloud platform 3, and the wireless communication component is communicatively connected to the processor. The wireless communication component may be a Bluetooth component or a WIFI component. The Bluetooth component is a component that performs wireless communication based on Bluetooth communication technology. The WIFI component is a component that performs wireless communication based on WIFI technology (Wireless Fidelity, a wireless local area network technology based on radio waves).
[0048] Optionally, the target device 1 further includes a wired communication component, and the target device 1 is connected to the mobile terminal 2 and the cloud platform 3 by wired communication via the wired communication component.
[0049] Optionally, the voice chip 11 adopts a WTK6900HC chip.
[0050] The mobile terminal 2 may be a smart phone, a tablet computer, or a portable wearable device. A client is installed in the mobile terminal. The user interacts with the mobile terminal through the client.
[0051] The cloud platform 3 can be implemented by an independent server or a server cluster consisting of multiple servers.
[0052] The controllable device 4 can be various personal computers, laptops, smart phones, tablet computers, portable wearable devices, smart gateways and smart home devices. For example, smart home devices include but are not limited to smart TVs, smart desk lamps, smart curtains, smart humidifiers, smart air conditioners, and smart air purifiers.
[0053] It is understandable that the target device 1 is not the controllable device 4 but may also be the controllable device 4 .
[0054] The present invention is described in detail below through specific embodiments.
[0055] See also Figure 2 As shown, Figure 2 A flow chart of a device control method based on voice recognition provided by an embodiment of the present invention is provided. The method is applied to a target device, the target device includes a voice chip, and the method includes:
[0056] S1: when the working mode of the target device is an offline recognition mode, obtaining a target voice feature, wherein the target voice feature is a voice feature extracted by the voice chip or the mobile terminal based on first voice data, and the first voice data is voice data input by a user through the target device or the mobile terminal;
[0057] When the target device is not connected to the cloud platform, the working mode of the target device is an offline identification mode.
[0058] It can be understood that when the working mode of the target device is the offline identification mode, the target device may be in communication connection with the mobile terminal or may not be in communication connection with the mobile terminal.
[0059] Specifically, when the working mode of the target device is the offline recognition mode, the target voice feature sent by the voice chip of the target device may be obtained, or the target voice feature sent by the mobile terminal may be obtained.
[0060] Optionally, the user inputs the first voice data into the target device, and the voice chip in the target device extracts voice features based on the first voice data.
[0061] Optionally, the user inputs the first voice data into a mobile terminal that is communicatively connected to the target device, and the mobile terminal extracts voice features based on the first voice data; alternatively, the mobile terminal sends the first voice data to the target device, and the voice chip extracts voice features based on the first voice data sent by the mobile terminal.
[0062] Voice features are descriptive data that describe the characteristics of voice data. Voice features include, but are not limited to: voiceprint features, pitch features, timbre features, sound intensity features, and sound length features.
[0063] Among them, the voice features are extracted from the first voice data through the voice feature extraction model in the voice chip 11.
[0064] The speech feature extraction model is a model obtained by training a neural network. The model structure and model training method of the model obtained by training a neural network can be selected from the prior art and will not be described in detail here.
[0065] S2: matching the voice feature in a voice feature library in a local storage space of the target device according to the target voice feature to obtain a first matching result;
[0066] Optionally, based on the accurate comparison method, a voice feature identical to the target voice feature is matched in the voice feature library in the local storage space of the target device. If the match is successful, the matching conclusion of the first matching result is determined to be successful; if the match fails, the matching conclusion of the first matching result is determined to be a failure.
[0067] Optionally, based on the method of approximate comparison, a voice feature similar to the target voice feature is matched in the voice feature library in the local storage space of the target device. If the match is successful (that is, the voice feature similar to the target voice feature is matched), the matching conclusion of the first matching result is determined to be successful. If the match fails (that is, the voice feature similar to the target voice feature is not matched), the matching conclusion of the first matching result is determined to be failed. It can be understood that the approximate voice feature refers to a voice feature with a similarity greater than a preset threshold. In other words, a voice feature with a similarity greater than a preset threshold to the target voice feature is matched in the voice feature library in the local storage space of the target device. If the match is successful (that is, the voice feature with a similarity greater than the preset threshold to the target voice feature is matched), the matching conclusion of the first matching result is determined to be successful. If the match fails (that is, the voice feature with a similarity greater than the preset threshold to the target voice feature is not matched), the matching conclusion of the first matching result is determined to be failed.
[0068] Optionally, the similarity can be calculated using cosine similarity.
[0069] It is understandable that when the matching conclusion of the first matching result is successful, the first matching result also includes a function identifier, and the function identifier in the first matching result is the function identifier corresponding to the matched voice feature in the voice feature library in the local storage space of the target device. In other words, the voice feature library in the local storage space of the target device describes the correspondence between the voice feature and the function identifier.
[0070] The function identifier may be data that uniquely identifies a function, such as a function name or a function ID.
[0071] S3: Generate a first instruction set according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction.
[0072] Specifically, when the matching conclusion of the first matching result is successful, instructions are matched from the instruction library in the local storage space of the target device according to the function identifier in the first matching result, and all matched instructions are used as the first instruction set; when the matching conclusion of the first matching result is failed, the preset reply instructions are used as the first instruction set.
[0073] The wake-up instruction is an instruction for waking up the target device. For example, the user inputs the voice data corresponding to the wake-up word "Xiao Wei", and the first instruction set obtained by the target device through steps S1 to S3 includes the wake-up instruction, which is used to wake up the target device.
[0074] The reply instruction is an instruction to play the voice corresponding to the reply word.
[0075] Device control instructions are instructions for controlling a controllable device to perform related operations. For example, a user may input voice data corresponding to the command word "turn on the air conditioner", and the first instruction set obtained by the target device through steps S1 to S3 at least includes the control instructions of the air conditioner (that is, the device control instructions).
[0076] Optionally, the device control instructions include, but are not limited to, air conditioner control instructions, light control instructions, television control instructions, curtain control instructions, and device shutdown instructions.
[0077] It can be understood that the present application realizes offline identification of the target device by extracting voice features and controlling the device based on the voice features.
[0078] The existing voice-to-text conversion method uses a voice chip to implement voice-based control of the device, while the present application extracts voice features and controls the device based on the voice features. When the local resources of the voice chip are limited, the model that extracts voice features can recognize more voices than the voice-to-text model, and the recognition accuracy is higher, thereby improving the accuracy of the voice control device. In addition, the present application controls the device based on the extracted voice features, so that voice data of any language can be used as the first voice data, thereby making the present application suitable for application scenarios of any language and multiple languages at the same time.
[0079] In one embodiment, the method further comprises:
[0080] S41: Acquire first pending-change data, wherein the first pending-change data includes a function identifier and a voice feature;
[0081] Optionally, the user first inputs a function identifier in the mobile terminal, and then inputs voice data (that is, learning voice data) in the mobile terminal. Finally, the mobile terminal extracts voice features based on the learning voice data and sends the function identifier input by the user and the extracted voice features as the first data to be updated to the target device. Alternatively, the mobile terminal sends the function identifier input by the user and the learning voice data as the first associated data to the target device. The target device extracts voice features based on the learning voice data in the first associated data and uses the function identifier sent by the mobile terminal and the voice features extracted by the target device as the first data to be updated.
[0082] Optionally, the user first inputs a function identifier in the mobile terminal, and the mobile terminal sends the function identifier to the target device. The user inputs voice data (that is, learning voice data) in the target device. The target device first extracts voice features based on the learning voice data input by the user, and then uses the function identifier sent by the mobile terminal and the extracted voice features as the first data to be updated.
[0083] S42: updating the voice feature library according to the first data to be updated;
[0084] The function identifier in the first data to be updated is data input by a user through the mobile terminal, and the voice feature in the first data to be updated is a voice feature extracted by the voice chip or the mobile terminal based on learning voice data.
[0085] Specifically, the voice feature library is replaced and updated according to the first data to be updated.
[0086] This embodiment implements offline learning of a voice feature library, and provides a basis for extracting voice features based on the voice feature library and controlling devices based on the voice features.
[0087] In one embodiment, the method further comprises:
[0088] S51: Obtaining the networking status of the target device and the cloud platform;
[0089] If the target device is connected to the cloud platform for normal communication, the networking status is connected; if the target device is not connected to the cloud platform for normal communication, the networking status is disconnected.
[0090] S52: If the networking state is networking, determining the working mode to be an offline identification mode;
[0091] Specifically, if the networking status is online, it means that the device cannot be controlled based on online voice recognition. Therefore, the working mode is determined to be an offline recognition mode.
[0092] S53: if the networking state is not connected to the network, determining the working mode to be the offline identification mode;
[0093] Specifically, if the networking status is not connected to the network, it means that the device can be controlled based on online voice recognition. Therefore, the working mode is determined to be the offline recognition mode.
[0094] S54: when the working mode is the offline recognition mode, obtaining a target text, wherein the target text is a text obtained by the cloud platform converting the second voice data based on the TTS technology, and the second voice data is voice data input by the user through the target device or the mobile terminal;
[0095] Optionally, when the working mode is the offline recognition mode, the user inputs voice data (that is, the second voice data) through the target device; the target device sends the second voice data to the cloud platform connected to the target device; the cloud platform converts the second voice data into text based on TTS technology (speech to text technology), uses the text as the target text, and sends the target text to the target device.
[0096] Optionally, when the working mode is the offline or online recognition mode, the user inputs voice data (that is, the second voice data) through the mobile terminal; the mobile terminal sends the second voice data to the cloud platform connected to the mobile terminal; the cloud platform converts the second voice data into text based on TTS technology (speech to text technology), uses the text as the target text, and sends the target text to the target device connected to the mobile terminal.
[0097] Optionally, when the working mode is the offline recognition mode, the step of acquiring the target text can be replaced by the following steps: when the working mode is the offline recognition mode, acquiring the target text or the feature to be processed, wherein the designated device acquires the voice data input by the user as the second voice data, the designated device is the target device or the mobile terminal, and the designated device inputs the second voice data into a pre-trained sound classification model for classification prediction to obtain a sound classification result; if the sound classification result is an animal sound, the voice chip or the mobile terminal uses the voice features extracted based on the second voice data as the feature to be processed; if the sound classification result is a human voice, the designated device sends the second voice data to the cloud platform; the cloud platform converts the second voice data into text based on the TTS technology to obtain the target text; the target device acquires the target text.
[0098] When the working mode is the offline recognition mode, if the feature to be processed is obtained, the voice feature is matched in the voice feature library in the local storage space of the target device according to the feature to be processed to obtain a target matching result; a target instruction set is generated according to the target matching result, wherein the target instruction set includes one or more of a wake-up instruction, a reply instruction and a device control instruction.
[0099] The sound classification model is a binary classification model. The model structure and model training method of the sound classification model can be selected from the existing technology and will not be described in detail here.
[0100] S55: Obtain a second matching result, wherein the second matching result is data obtained by matching text in a text library in the executing device according to the target text by an executing device, and the executing device is any one of the target device, the mobile terminal and the cloud platform;
[0101] Specifically, if the execution device is the target device, the target device matches text in a text library in the target device according to the target text to obtain the second matching result; if the execution device is the mobile terminal, the mobile terminal matches text in a text library in the mobile terminal according to the target text to obtain the second matching result, and sends the second matching result to the target device; if the execution device is the cloud platform, the cloud platform matches text in a text library in the cloud platform according to the target text to obtain the second matching result, and sends the second matching result to the target device.
[0102] The second matching result includes a matching conclusion and description information. The matching conclusion of the second matching result is a match or a mismatch; if the matching conclusion of the second matching result is a match, the value of the description information of the second matching result includes the function identifier corresponding to the matched text; if the matching conclusion of the second matching result is a mismatch, the value of the description information of the second matching result is empty.
[0103] S56: Generate a second instruction set according to the second matching result, wherein the second instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction.
[0104] Specifically, when the matching conclusion of the second matching result is successful, instructions are matched from the instruction library in the local storage space of the target device according to the function identifier in the second matching result, and all matched instructions are used as the second instruction set; when the matching conclusion of the second matching result is failed, the preset reply instructions are used as the second instruction set.
[0105] In this embodiment, when the working mode is the offline recognition mode, device control based on speech to text is implemented through a cloud platform; compared with offline recognition, the range of voice data that can be recognized by device control based on speech to text implemented through a cloud platform is wider, thereby increasing the response range of device control and improving user experience.
[0106] In one embodiment, the method further comprises:
[0107] S61: Acquire second pending-update data, wherein the second pending-update data includes a function identifier and a description text;
[0108] Specifically, the second to-be-updated data sent by the cloud platform or the mobile terminal is obtained.
[0109] S62: updating the text library in the local storage space of the target device according to the second data to be updated;
[0110] The second matching result is data obtained by the target device matching text in the text library in the target device according to the target text.
[0111] Specifically, the text library in the local storage space of the target device is replaced and updated according to the second data to be updated.
[0112] This embodiment updates the text library in the local storage space of the target device according to the second data to be updated, and then obtains the second matching result by matching the text in the text library in the target device according to the target text, thereby providing a basis for text matching in the target device during online speech recognition.
[0113] In one embodiment, the method further comprises:
[0114] S71: Acquire third voice data sent by the cloud platform, wherein the third voice data is audio data synthesized by the cloud platform according to first information, the first information is information input by a user through the target device or the mobile terminal, and the first information includes: timbre parameters, speed parameters, volume parameters and text to be synthesized;
[0115] Optionally, the user inputs the first information through the target device, and the target device uploads the first information to the cloud platform; the cloud platform obtains third voice data based on the audio data synthesized from the first information, and the third voice data is sent to the target device.
[0116] Optionally, the user inputs the first information through the mobile terminal, and the mobile terminal uploads the first information to the cloud platform; the cloud platform obtains third voice data based on the audio data synthesized from the first information, and the third voice data sends the target device to the target device.
[0117] That is to say, the cloud platform is provided with a speech synthesis engine, and the speech synthesis engine synthesizes audio data according to the first information.
[0118] S72: Storing the third voice data in the local storage space of the target device, wherein the third voice data in the local storage space of the target device is used to respond to the reply instruction.
[0119] Specifically, the voice data in the local storage space of the target device is replaced and updated according to the third voice data.
[0120] When the target device executes the reply instruction, the target device plays the audio data corresponding to the reply instruction (that is, the third voice data in the local storage space of the target device).
[0121] In this embodiment, the third voice data sent by the cloud platform is stored in the local storage space of the target device, so that the target device with a low-end voice chip can play rich reply voice (that is, the third voice data). When the target device has a low-end voice chip, personalized user needs are met and the user experience is further improved.
[0122] In one embodiment, the method further comprises:
[0123] S81: Obtain download request;
[0124] Specifically, the user inputs a download request through the target device, or the user sends a download request through the mobile terminal and the mobile terminal sends the download request to the target device.
[0125] The download request is a request to download voice data.
[0126] S82: acquiring fourth voice data from the first platform according to the download request, wherein the fourth voice data is audio data synthesized by the cloud platform according to second information, the second information is information input by a user through the second platform, and the second information includes: timbre parameters, speed parameters, volume parameters and text to be synthesized;
[0127] Specifically, according to the voice identifier in the download request, voice data is acquired from the first platform, and the voice data is used as the fourth voice data.
[0128] The voice identifier may be data that uniquely identifies a piece of voice data, such as a voice name or a voice ID.
[0129] It is understandable that the second platform and the first platform may be the same application system or may not be the same application system.
[0130] The speech synthesis engine in the cloud platform synthesizes audio data according to the second information, and uses the audio data as the fourth speech data.
[0131] S83: Storing the fourth voice data in the local storage space of the target device, wherein the fourth voice data in the local storage space of the target device is used to respond to the reply instruction.
[0132] Specifically, the voice data in the local storage space of the target device is replaced and updated according to the fourth voice data.
[0133] This embodiment synthesizes audio data according to the second information sent by the second platform on the cloud platform, and sends the synthesized audio data to the first platform, thereby providing rich audio data download resources without the need for compilation and packaging on the cloud platform. When the target device is equipped with a low-end voice chip, the user's personalized needs are met, and the user experience is further improved.
[0134] In one embodiment, the first voice data is voice data of a human or an animal.
[0135] In this embodiment, the first voice data is voice data of a human or an animal, so that both humans and animals can be recognized offline through this application, making this application suitable for application scenarios that require animal-based voice control devices, further improving the user experience.
[0136] See also Figure 3 As shown, in one embodiment, a device control system based on voice recognition is provided, the system comprising: a target device and a mobile terminal, the target device and the mobile terminal are in communication connection, the target device comprises: a voice chip, the target device is configured to implement the steps of the device control method based on voice recognition, the device control method based on voice recognition comprises:
[0137] When the working mode of the target device is an offline recognition mode, obtaining a target voice feature, wherein the target voice feature is a voice feature extracted by the voice chip or the mobile terminal based on first voice data, and the first voice data is voice data input by a user through the target device or the mobile terminal;
[0138] According to the target voice feature, matching the voice feature in a voice feature library in the local storage space of the target device to obtain a first matching result;
[0139] A first instruction set is generated according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction.
[0140] The existing voice-to-text conversion method uses a voice chip to implement voice-based control of the device, while the present application extracts voice features and controls the device based on the voice features. When the local resources of the voice chip are limited, the model that extracts voice features can recognize more voices than the voice-to-text model, and the recognition accuracy is higher, thereby improving the accuracy of the voice control device. In addition, the present application controls the device based on the extracted voice features, so that voice data of any language can be used as the first voice data, thereby making the present application suitable for application scenarios of any language and multiple languages at the same time.
[0141] In one embodiment, the method further comprises:
[0142] Acquire first pending-change data, wherein the first pending-change data includes a function identifier and a voice feature;
[0143] updating the voice feature library according to the first data to be updated;
[0144] The function identifier in the first data to be updated is data input by a user through the mobile terminal, and the voice feature in the first data to be updated is a voice feature extracted by the voice chip or the mobile terminal based on learning voice data.
[0145] In one embodiment, the method further comprises:
[0146] Obtaining the networking status of the target device and the cloud platform;
[0147] If the networking state is networking, the working mode is determined to be an offline identification mode;
[0148] If the networking state is not connected to the network, the working mode is determined to be the offline identification mode;
[0149] When the working mode is the offline recognition mode, a target text is obtained, wherein the target text is a text obtained by the cloud platform converting the second voice data based on the TTS technology, and the second voice data is voice data input by the user through the target device or the mobile terminal;
[0150] Obtain a second matching result, wherein the second matching result is data obtained by matching text in a text library in the executing device according to the target text by an executing device, and the executing device is any one of the target device, the mobile terminal, and the cloud platform;
[0151] A second instruction set is generated according to the second matching result, wherein the second instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction.
[0152] In one embodiment, the method further comprises:
[0153] Acquire second pending-change data, wherein the second pending-change data includes a function identifier and a description text;
[0154] updating a text library in a local storage space of the target device according to the second data to be updated;
[0155] The second matching result is data obtained by the target device matching text in the text library in the target device according to the target text.
[0156] In one embodiment, the method further comprises:
[0157] Acquire third voice data sent by the cloud platform, wherein the third voice data is audio data synthesized by the cloud platform according to first information, the first information is information input by a user through the target device or the mobile terminal, and the first information includes: timbre parameters, sound speed parameters, volume parameters and text to be synthesized;
[0158] The third voice data is stored in the local storage space of the target device, wherein the third voice data in the local storage space of the target device is used to respond to the reply instruction.
[0159] In one embodiment, the method further comprises:
[0160] Get download request;
[0161] According to the download request, fourth voice data is obtained from the first platform, wherein the fourth voice data is audio data synthesized by the cloud platform according to second information, the second information is information input by a user through the second platform, and the second information includes: timbre parameters, speed parameters, volume parameters and text to be synthesized;
[0162] The fourth voice data is stored in the local storage space of the target device, wherein the fourth voice data in the local storage space of the target device is used to respond to the reply instruction.
[0163] In one embodiment, the first voice data is voice data of a human or an animal.
[0164] In one embodiment, a target device is provided, the target device comprising a voice chip, a memory, a processor, and a computer program stored in the memory and executable on the processor, the voice chip is communicatively connected to the processor, and the processor implements the following steps when executing the computer program:
[0165] When the working mode of the target device is an offline recognition mode, obtaining a target voice feature, wherein the target voice feature is a voice feature extracted by the voice chip or the mobile terminal based on first voice data, and the first voice data is voice data input by a user through the target device or the mobile terminal;
[0166] According to the target voice feature, matching the voice feature in a voice feature library in the local storage space of the target device to obtain a first matching result;
[0167] A first instruction set is generated according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction.
[0168] The existing voice-to-text conversion method uses a voice chip to implement voice-based control of the device, while the present application extracts voice features and controls the device based on the voice features. When the local resources of the voice chip are limited, the model that extracts voice features can recognize more voices than the voice-to-text model, and the recognition accuracy is higher, thereby improving the accuracy of the voice control device. In addition, the present application controls the device based on the extracted voice features, so that voice data of any language can be used as the first voice data, thereby making the present application suitable for application scenarios of any language and multiple languages at the same time.
[0169] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is applied to a target device, wherein the target device includes a voice chip, and when the computer program is executed by a processor, the following steps are implemented:
[0170] When the working mode of the target device is an offline recognition mode, obtaining a target voice feature, wherein the target voice feature is a voice feature extracted by the voice chip or the mobile terminal based on first voice data, and the first voice data is voice data input by a user through the target device or the mobile terminal;
[0171] According to the target voice feature, matching the voice feature in a voice feature library in the local storage space of the target device to obtain a first matching result;
[0172] A first instruction set is generated according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction.
[0173] The existing voice-to-text conversion method uses a voice chip to implement voice-based control of the device, while the present application extracts voice features and controls the device based on the voice features. When the local resources of the voice chip are limited, the model that extracts voice features can recognize more voices than the voice-to-text model, and the recognition accuracy is higher, thereby improving the accuracy of the voice control device. In addition, the present application controls the device based on the extracted voice features, so that voice data of any language can be used as the first voice data, thereby making the present application suitable for application scenarios of any language and multiple languages at the same time.
[0174] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or the target device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.
[0175] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0176] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0177] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A device control method based on voice recognition, characterized in that: The method is applied to a target device, the target device includes a voice chip, and the target device is one of a smart phone, a tablet computer, a portable wearable device, a smart gateway, and a smart home device. The method includes: When the working mode of the target device is an offline recognition mode, a target voice feature is obtained, wherein the target voice feature is a voice feature extracted by the voice chip or the mobile terminal based on first voice data, the first voice data is voice data input by a user through the target device or the mobile terminal, and the first voice data is voice data of a person or an animal; According to the target voice feature, matching the voice feature in a voice feature library in the local storage space of the target device to obtain a first matching result; Generate a first instruction set according to the first matching result, wherein the first instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction, and the target device controls a controllable device corresponding to the device control instruction through the device control instruction; The method further comprises: Obtaining the networking status of the target device and the cloud platform; If the networking state is networking, the working mode is determined to be an offline identification mode; If the networking state is not connected to the network, the working mode is determined to be the offline identification mode; When the working mode is the offline recognition mode, a target text is obtained, wherein the target text is a text obtained by the cloud platform converting the second voice data based on the TTS technology, and the second voice data is voice data input by the user through the target device or the mobile terminal; Obtain a second matching result, wherein the second matching result is data obtained by matching text in a text library in the executing device according to the target text by an executing device, and the executing device is any one of the target device, the mobile terminal, and the cloud platform; generating a second instruction set according to the second matching result, wherein the second instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction; When the working mode is the offline recognition mode, the step of acquiring the target text includes: When the working mode is the offline recognition mode, a target text or a feature to be processed is obtained, wherein the designated device obtains voice data input by a user as second voice data, the designated device is the target device or the mobile terminal, and the designated device inputs the second voice data into a pre-trained sound classification model for classification prediction to obtain a sound classification result; If the sound classification result is an animal sound, the voice chip or mobile terminal extracts the voice features obtained based on the second voice data as the features to be processed, and matches the voice features in the voice feature library in the local storage space of the target device according to the features to be processed to obtain a target matching result; and generates a target instruction set according to the target matching result, wherein the target instruction set includes one or more of a wake-up instruction, a reply instruction, and a device control instruction; If the sound classification result is a human voice, the designated device sends the second voice data to the cloud platform to obtain the target text, which is the text obtained by the cloud platform converting the second voice data based on the TTS technology.
2. The device control method based on voice recognition according to claim 1, characterized in that: The method further comprises: Acquire first pending-change data, wherein the first pending-change data includes a function identifier and a voice feature; updating the voice feature library according to the first data to be updated; The function identifier in the first data to be updated is data input by a user through the mobile terminal, and the voice feature in the first data to be updated is a voice feature extracted by the voice chip or the mobile terminal based on learning voice data.
3. The device control method based on voice recognition according to claim 1, characterized in that: The method further comprises: Acquire second pending-change data, wherein the second pending-change data includes a function identifier and a description text; updating a text library in a local storage space of the target device according to the second data to be updated; The second matching result is data obtained by the target device matching text in the text library in the target device according to the target text.
4. The device control method based on voice recognition according to claim 1, characterized in that: The method further comprises: Acquire third voice data sent by the cloud platform, wherein the third voice data is audio data synthesized by the cloud platform according to first information, the first information is information input by a user through the target device or the mobile terminal, and the first information includes: timbre parameters, sound speed parameters, volume parameters and text to be synthesized; The third voice data is stored in the local storage space of the target device, wherein the third voice data in the local storage space of the target device is used to respond to the reply instruction.
5. The device control method based on voice recognition according to claim 1, characterized in that: The method further comprises: Get download request; According to the download request, fourth voice data is obtained from the first platform, wherein the fourth voice data is audio data synthesized by the cloud platform according to second information, the second information is information input by a user through the second platform, and the second information includes: timbre parameters, speed parameters, volume parameters and text to be synthesized; The fourth voice data is stored in the local storage space of the target device, wherein the fourth voice data in the local storage space of the target device is used to respond to the reply instruction.
6. A device control system based on voice recognition, characterized in that: The system includes: a target device and a mobile terminal, the target device and the mobile terminal are communicatively connected, the target device includes: a voice chip, the target device is one of a smart phone, a tablet computer, a portable wearable device, a smart gateway and a smart home device, and the target device is configured to implement the steps of the device control method based on voice recognition as described in any one of claims 1 to 5.
7. A target device, characterized in that:The target device is one of a smart phone, a tablet computer, a portable wearable device, a smart gateway and a smart home device. The target device includes a voice chip, a memory, a processor and a computer program stored in the memory and executable on the processor. The voice chip is communicatively connected to the processor. When the processor executes the computer program, the steps of the device control method based on voice recognition as described in any one of claims 1 to 5 are implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the device control method based on voice recognition as claimed in any one of claims 1 to 5 are implemented.
