Equipment, voice control method and device thereof and readable storage medium
By using preset large language models to analyze speech input and generate task information, and providing feedback with speech synthesis module, the problem of insufficient complex context and fuzzy semantic recognition capabilities in the existing technology is solved, and efficient and accurate task execution and naturalization are achieved human-computer interaction.
Patent Information
- Application Number
- CN202311252440.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-26
- Publication Date
- 2024-11-12
AI Technical Summary
Existing voice interaction technologies have limited recognition capabilities when dealing with complex contexts and fuzzy semantics, making it difficult to accurately understand user needs, affecting the accuracy and efficiency of task execution.
The preset large language model is used to analyze the voice input, generate task information, and provide natural language feedback through the voice synthesis module to achieve smooth interaction between the device and the user.
It improves the recognition ability of voice input, ensures the accuracy and efficiency of task execution, and provides a natural and convenient human-computer interaction experience.
Smart Images

Figure CN118942460A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of control technology, and in particular to a device and a voice control method, apparatus and readable storage medium thereof. Background Art
[0002] In related technical solutions, more and more devices support voice control. The emergence of voice control technology provides convenience for the interaction between users and devices.
[0003] However, in the related technical solutions, the voice interaction solutions supported by the device are mostly: using a voice recognition model to perform voice recognition on the voice input by the user, and then controlling according to the recognition result.
[0004] The above voice interaction solution has the following disadvantages:
[0005] When the user's voice input is relatively complex, the recognition results output by the speech recognition model are unsatisfactory. Its semantic understanding ability is limited, and there are still challenges in handling complex contexts and fuzzy semantics. It is difficult to accurately understand user needs, which affects the accuracy and efficiency of task execution. Summary of the invention
[0006] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.
[0007] To this end, a first aspect of the present invention is to provide a voice control method for a device.
[0008] A second aspect of the present invention provides a voice control device for a device.
[0009] A third aspect of the present invention is to provide another voice control device for equipment.
[0010] A fourth aspect of the present invention provides a readable storage medium.
[0011] A fifth aspect of the present invention provides a device.
[0012] In view of this, according to the first aspect of the present invention, the present invention provides a voice control method for a device, comprising: obtaining a user's voice input; performing voice recognition on the voice input to obtain text information; using a preset language model to parse the text information to obtain task information; and executing a task corresponding to the task information.
[0013] The technical solution of the present application proposes a voice control method for a device. By running the above control method, the recognition ability of voice input can be improved, and thus it has better processing performance in processing complex contexts and fuzzy semantics, and can accurately understand the needs of users, ensuring the accuracy and efficiency of task execution.
[0014] The technical solution of the present application is implemented based on the following principles. Specifically, when voice input is obtained, voice recognition is performed on the voice input to obtain text information corresponding to the voice input, and then the text information is parsed using a preset language model to obtain task information.
[0015] In this process, text information is input into the preset language model. Compared with the solution of directly using voice input, there is no need to train voice recognition on the preset language model, which reduces the cost of model training.
[0016] Among them, the preset language model is a large language model, that is, a large language model (LLM) is an artificial intelligence model designed to understand and generate human language. They are trained on a large amount of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.
[0017] In the technical solution of the present application, a large language model is used to accurately understand user needs and ensure the accuracy of the output task information, thereby improving the success rate of task execution.
[0018] In some technical solutions, optionally, the large language model is a large pre-trained language model that has been fine-tuned and optimized, which can be a Chat Generative Pre-trained Transformer (ChatGPT), which can be understood as a chatbot program. T is a natural language processing tool driven by artificial intelligence technology. It can conduct conversations by understanding and learning human language, and can also interact according to the context of the chat, truly chat and communicate like humans, and can even complete tasks such as writing emails, video scripts, copywriting, translation, code, and writing papers.
[0019] In some technical solutions, optionally, the task information is a structured task instruction.
[0020] In addition, the voice control method of the device proposed in this application also has the following additional technical features.
[0021] In some technical solutions, optionally, in the process of generating task information, the voice control method also includes: generating natural language feedback information based on a preset language model; before executing the task corresponding to the task information, it also includes: synthesizing voice information based on the feedback information; and playing the voice information.
[0022] In this technical solution, a preset language model is used to generate feedback information in the form of natural language. In this process, the advantages of the preset language model can be used to ensure that the output feedback information matches the user's voice input. By playing the voice information synthesized based on the feedback information, voice interaction between the device and the user can be achieved. In this process, a smooth interaction can be achieved between the device and the user, thereby improving the interactive experience of the device.
[0023] In some technical solutions, optionally, speech synthesis is to convert text information into speech signals understandable to humans.
[0024] In the present invention, speech synthesis is used to convert the feedback information of natural language into speech signals (ie, speech information) for the convenience of users to listen.
[0025] In some technical solutions, the voice control method also includes: outputting feedback information.
[0026] In this technical solution, feedback information can be output in the form of text, so that the device can display feedback information to the user in different forms, thereby meeting the interaction needs of different scenarios.
[0027] In some technical solutions, optionally, voice recognition is performed on the voice input to obtain text information, specifically including: using a deep learning-based voice recognition model to perform voice recognition on the voice input to obtain text information.
[0028] In this technical solution, the speech recognition model based on deep learning has better speech synthesis ability. By adopting the above speech training model, the text information obtained by recognition can accurately express the user's ideas. Compared with the recognition solution that adopts word-by-word text recognition, the technical solution of this application can more accurately express the meaning expressed by the speech input, and then fully understand the meaning of the speech input.
[0029] In some technical solutions, speech recognition is optionally the process of converting human speech signals into text information that can be processed by a computer. In the present invention, speech recognition technology is used to convert the user's speech input into text to facilitate subsequent interaction with a preset language model.
[0030] In some technical solutions, optionally, the speech recognition model can be a long short-term memory network (Long Short-Term Memory, LSTM) or a natural language processing model.
[0031] Among them, Natural Language Processing (NLP) is an interdisciplinary field of computer science, artificial intelligence and linguistics, dedicated to enabling computers to understand, parse and generate human language. In the present invention, natural language processing technology is used to parse the user's voice input, extract key information, and interact with the preset language model to generate structured task instructions (i.e., task information) and natural language feedback (i.e., feedback information).
[0032] In some technical solutions, optionally, voice information is synthesized based on the feedback information, specifically including: performing voice synthesis on the feedback information based on an end-to-end voice synthesis model based on deep learning to obtain voice information.
[0033] In this technical solution, voice information is synthesized so that the user and the device can interact through voice, thus overcoming the dependence of the commands used to control the operation of the device on manual input or a fixed command set in the related technical solutions, thereby improving the interactive experience between the user and the device.
[0034] In some technical solutions, optionally, the end-to-end speech synthesis model can be a Tacotron-based speech synthesis model.
[0035] In this technical solution, the above-mentioned end-to-end speech synthesis model is used to make the sentences of the generated voice information more natural and fluent, and the interaction between the user and the device more natural and convenient, thereby improving the user experience and acceptance.
[0036] In some technical solutions, optionally, the task information includes at least one task time and a corresponding task execution location; executing the task corresponding to the task information specifically includes: determining a task execution path based on at least one task time and a corresponding task execution location; and executing the task corresponding to the task information based on the task execution path.
[0037] In this technical solution, when the voice input contains multiple tasks, the preset language model can parse out the task time corresponding to each task and the task execution location corresponding to the task time during the parsing of text information, and then determine the task execution path based on the parsed task time and task execution location, so that the device can use the task execution path as a guide for task execution during the operation process, thereby more accurately understanding user needs and improving the success rate of task execution.
[0038] In the above technical solution, the task execution path can be understood as the order in which multiple tasks are executed in sequence when there are multiple tasks.
[0039] In some technical solutions, the task execution path uses the task time corresponding to each task as a sorting condition, wherein the task time includes the start time and the end time when the task is executed, so as to improve the reliability of task execution.
[0040] In some technical solutions, optionally, the device is a cleaning robot or a service robot, and the voice control method also includes: obtaining current location information of the device; determining a navigation path based on the current location information and the task execution path; wherein the device walks based on the navigation path.
[0041] In this technical solution, during the operation of the equipment, the current position information is obtained so as to guide the movement of the equipment according to the current position information and the task execution path. In this process, the equipment can achieve autonomous walking and automatically perform operations.
[0042] In this process, the user does not need to participate in the movement of the equipment, which overcomes the disadvantage that the equipment requires manual participation of the user to operate and improves the efficiency of the operation.
[0043] In some technical solutions, optionally, the service robot can be a guest room robot in a hotel, or a mobile robot used to serve food in a restaurant. The service robot can deploy different functions according to its different usage scenarios, which will not be elaborated here.
[0044] In some technical solutions, optionally, the current location information can be obtained through a location sensor set on the device.
[0045] Among them, the position sensor can select the corresponding sensor according to the actual usage scenario, and its specific selection will not be repeated here.
[0046] In some technical solutions, optionally, it also includes: obtaining environmental information of the environment in which the device is located; and updating the navigation path based on the environmental information.
[0047] In this technical solution, environmental information is obtained so that the device can update the navigation path according to the environmental information during walking, reduce collisions with other objects in the environment, thereby achieving dynamic obstacle avoidance, protecting the safety of the device itself to the greatest extent, and reducing the impact of the device operation on other objects in the environment.
[0048] In some technical solutions, optionally, the environmental information includes placement information of other objects in the environment except the device.
[0049] In some technical solutions, optionally, the environmental information includes not only the placement position information of other objects, but also the posture information of other objects.
[0050] In this technical solution, by obtaining the above information, obstacles can be avoided to the greatest extent possible.
[0051] In some technical solutions, optionally, environmental information is acquired and perceived using one or more of a camera, a lidar, or an infrared sensor.
[0052] In some technical solutions, optionally, it also includes: when the task information includes keywords for indicating repeated execution, determining the start time, end time and target task associated with the keywords; and determining the periodic task based on the start time, end time and target task.
[0053] In this technical solution, the keywords may be words that can express repeated execution, such as weekly, every interval, monthly, and daily. They can be selected according to the actual usage scenario and will not be described in detail here.
[0054] In the above technical solution, the start time, end time and target task can be obtained by recognizing and parsing the voice input, which will not be described in detail here.
[0055] In the above technical solution, by determining periodic tasks so that the device can perform tasks periodically, there is no need for the user to repeatedly issue voice input instructions, which helps reduce the frequency of interaction between the user and the device, thereby improving the user experience of the device.
[0056] In some technical solutions, optionally, a preset language model is used to parse the text information to obtain task information, including: sending the text information to a preset language model deployed on a server; and receiving the task information output from the preset language model.
[0057] In this technical solution, text information is sent out so that the preset language model deployed on the server can process the received text information and obtain task information. In this process, the text information processing process can be transferred from the local device to the cloud, that is, the server side, and the server's large and sufficient computing power is used to process data, thereby increasing the speed of data processing to meet different needs in different scenarios.
[0058] According to the second aspect of the present invention, the present invention provides a voice control device for a device, comprising: a human-computer interaction module, used to obtain a user's voice input; a voice recognition module, used to perform voice recognition on the voice input to obtain text information; a generation module, used to parse the text information using a preset language model to obtain task information; and an execution module, used to execute a task corresponding to the task information.
[0059] The technical solution of the present application proposes a voice control device for a device, which can improve the recognition ability of voice input, and thus has better processing performance in processing complex contexts and fuzzy semantics, and can accurately understand the needs of users, ensuring the accuracy and efficiency of task execution.
[0060] The technical solution of the present application is implemented based on the following principles. Specifically, when voice input is obtained, voice recognition is performed on the voice input to obtain text information corresponding to the voice input, and then the text information is parsed using a preset language model to obtain task information.
[0061] In this process, text information is input into the preset language model. Compared with the solution of directly using voice input, there is no need to train voice recognition on the preset language model, which reduces the cost of model training.
[0062] Among them, the preset language model is a large language model, that is, a large language model (LLM) is an artificial intelligence model designed to understand and generate human language. They are trained on a large amount of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.
[0063] In the technical solution of the present application, a large language model is used to accurately understand user needs and ensure the accuracy of the output task information, thereby improving the success rate of task execution.
[0064] In some technical solutions, optionally, the large language model is a large pre-trained language model that has been fine-tuned and optimized, which can be a Chat Generative Pre-trained Transformer (ChatGPT), which can be understood as a chatbot program. T is a natural language processing tool driven by artificial intelligence technology. It can conduct conversations by understanding and learning human language, and can also interact according to the context of the chat, truly chat and communicate like humans, and can even complete tasks such as writing emails, video scripts, copywriting, translation, code, and writing papers.
[0065] In some technical solutions, optionally, the task information is a structured task instruction.
[0066] In addition, the voice control device of the device proposed in this application also has the following additional technical features.
[0067] In some technical solutions, optionally, in the process of generating task information, the generation module is also used to: generate natural language feedback information based on a preset language model; before executing the task corresponding to the task information, the voice control device of the device also includes: a speech synthesis module, used to synthesize voice information based on the feedback information; and play the voice information.
[0068] In this technical solution, a preset language model is used to generate feedback information in the form of natural language. In this process, the advantages of the preset language model can be used to ensure that the output feedback information matches the user's voice input. By playing the voice information synthesized based on the feedback information, voice interaction between the device and the user can be achieved. In this process, a smooth interaction can be achieved between the device and the user, thereby improving the interactive experience of the device.
[0069] In some technical solutions, optionally, speech synthesis is to convert text information into speech signals understandable to humans.
[0070] In the present invention, speech synthesis is used to convert the feedback information of natural language into speech signals (ie, speech information) for the convenience of users to listen.
[0071] In some technical solutions, the voice control method also includes: outputting feedback information.
[0072] In this technical solution, feedback information can be output in the form of text, so that the device can display feedback information to the user in different forms, thereby meeting the interaction needs of different scenarios.
[0073] In some technical solutions, optionally, the speech recognition module is specifically used to: use a deep learning-based speech recognition model to perform speech recognition on speech input to obtain text information.
[0074] In this technical solution, the speech recognition model based on deep learning has better speech synthesis ability. By adopting the above speech training model, the text information obtained by recognition can accurately express the user's ideas. Compared with the recognition solution that adopts word-by-word text recognition, the technical solution of this application can more accurately express the meaning expressed by the speech input, and then fully understand the meaning of the speech input.
[0075] In some technical solutions, speech recognition is optionally the process of converting human speech signals into text information that can be processed by a computer. In the present invention, speech recognition technology is used to convert the user's speech input into text to facilitate subsequent interaction with a preset language model.
[0076] In some technical solutions, optionally, the speech recognition model can be a long short-term memory network (Long Short-Term Memory, LSTM) or a natural language processing model.
[0077] Among them, Natural Language Processing (NLP) is an interdisciplinary field of computer science, artificial intelligence and linguistics, dedicated to enabling computers to understand, parse and generate human language. In the present invention, natural language processing technology is used to parse the user's voice input, extract key information, and interact with the preset language model to generate structured task instructions (i.e., task information) and natural language feedback (i.e., feedback information).
[0078] In some technical solutions, optionally, the speech synthesis module is specifically used to: perform speech synthesis on the feedback information based on an end-to-end speech synthesis model based on deep learning to obtain speech information.
[0079] In this technical solution, voice information is synthesized so that the user and the device can interact through voice, thus overcoming the dependence of the commands used to control the operation of the device on manual input or a fixed command set in the related technical solutions, thereby improving the interactive experience between the user and the device.
[0080] In some technical solutions, optionally, the end-to-end speech synthesis model can be a Tacotron-based speech synthesis model.
[0081] In this technical solution, the above-mentioned end-to-end speech synthesis model is used to make the sentences of the generated voice information more natural and fluent, and the interaction between the user and the device more natural and convenient, thereby improving the user experience and acceptance.
[0082] In some technical solutions, optionally, the task information includes at least one task time and a corresponding task execution location; the execution module is specifically used to: determine the task execution path based on at least one task time and the corresponding task execution location; and execute the task corresponding to the task information based on the task execution path.
[0083] In this technical solution, when the voice input contains multiple tasks, the preset language model can parse out the task time corresponding to each task and the task execution location corresponding to the task time during the parsing of text information, and then determine the task execution path based on the parsed task time and task execution location, so that the device can use the task execution path as a guide for task execution during the operation process, thereby more accurately understanding user needs and improving the success rate of task execution.
[0084] In the above technical solution, the task execution path can be understood as the order in which multiple tasks are executed in sequence when there are multiple tasks.
[0085] In some technical solutions, the task execution path uses the task time corresponding to each task as a sorting condition, wherein the task time includes the start time and the end time when the task is executed, so as to improve the reliability of task execution.
[0086] In some technical solutions, optionally, the device is a cleaning robot or a service robot, and the voice control device also includes: a position information module, used to obtain the current position information of the device; a path planning module, used to determine the navigation path according to the current position information and the task execution path; wherein the device walks based on the navigation path.
[0087] In this technical solution, during the operation of the equipment, the current position information is obtained so as to guide the movement of the equipment according to the current position information and the task execution path. In this process, the equipment can achieve autonomous walking and automatically perform operations.
[0088] In this process, the user does not need to participate in the movement of the equipment, which overcomes the disadvantage that the equipment requires manual participation of the user to operate and improves the efficiency of the operation.
[0089] In some technical solutions, optionally, the service robot can be a guest room robot in a hotel, or a mobile robot used to serve food in a restaurant. The service robot can deploy different functions according to its different usage scenarios, which will not be elaborated here.
[0090] In some technical solutions, optionally, the current location information can be obtained through a location sensor set on the device.
[0091] Among them, the position sensor can select the corresponding sensor according to the actual usage scenario, and its specific selection will not be repeated here.
[0092] In some technical solutions, optionally, it also includes: a sensor module, which is used to obtain environmental information of the environment in which the device is located; and a path planning module is also used to update the navigation path based on the environmental information.
[0093] In this technical solution, environmental information is obtained so that the device can update the navigation path according to the environmental information during walking, reduce collisions with other objects in the environment, thereby achieving dynamic obstacle avoidance, protecting the safety of the device itself to the greatest extent, and reducing the impact of the device operation on other objects in the environment.
[0094] In some technical solutions, optionally, the environmental information includes placement information of other objects in the environment except the device.
[0095] In some technical solutions, optionally, the environmental information includes not only the placement position information of other objects, but also the posture information of other objects.
[0096] In this technical solution, by obtaining the above information, obstacles can be avoided to the greatest extent possible.
[0097] In some technical solutions, optionally, environmental information is acquired and perceived using one or more of a camera, a lidar, or an infrared sensor.
[0098] In some technical solutions, optionally, it also includes: a time management module, which is used to determine the start time, end time and target task associated with the keywords when the task information includes keywords for indicating repeated execution; and determine the periodic task based on the start time, end time and target task.
[0099] In this technical solution, the keywords may be words that can express repeated execution, such as weekly, every interval, monthly, and daily. They can be selected according to the actual usage scenario and will not be described in detail here.
[0100] In the above technical solution, the start time, end time and target task can be obtained by recognizing and parsing the voice input, which will not be described in detail here.
[0101] In the above technical solution, by determining periodic tasks so that the device can perform tasks periodically, there is no need for the user to repeatedly issue voice input instructions, which helps reduce the frequency of interaction between the user and the device, thereby improving the user experience of the device.
[0102] In some technical solutions, optionally, it also includes: a cloud computing module, used to send text information to a preset language model deployed on a server; and receive task information output from the preset language model.
[0103] In this technical solution, text information is sent out so that the preset language model deployed on the server can process the received text information and obtain task information. In this process, the text information processing process can be transferred from the local device to the cloud, that is, the server side, and the server's large and sufficient computing power is used to process data, thereby increasing the speed of data processing to meet different needs in different scenarios.
[0104] According to the third aspect of the present invention, the present invention provides a voice control device for a device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of any method in the first aspect are implemented.
[0105] According to a fourth aspect of the present invention, the present invention provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of any method in the first aspect are implemented.
[0106] According to a fifth aspect of the present invention, the present invention provides a device, comprising: a voice control device as in any of the above aspects; and / or a readable storage medium as in the above aspects.
[0107] In some technical solutions, optionally, the device includes one of the following: a cleaning robot or a service robot.
[0108] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0109] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0110] Figure 1 A schematic diagram showing a flow chart of a method for voice control of a device in an embodiment of the present application is shown;
[0111] Figure 2 A schematic block diagram showing a voice control device of a device in an embodiment of the present invention is shown;
[0112] Figure 3 A schematic block diagram showing a voice control device of another device in an embodiment of the present invention is shown;
[0113] Figure 4 A schematic block diagram showing the interaction between mobile phone software, a server and a device in an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0114] In order to more clearly understand the above aspects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0115] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0116] In one embodiment of the present application, Figure 1 As shown, a voice control method for a device is provided, comprising:
[0117] Step 102, obtaining the user's voice input;
[0118] Step 104, performing speech recognition on the speech input to obtain text information;
[0119] Step 106, using a preset language model to parse the text information to obtain task information;
[0120] Step 108: Execute the task corresponding to the task information.
[0121] The technical solution of the present application proposes a voice control method for a device. By running the above control method, the recognition ability of voice input can be improved, and thus it has better processing performance in processing complex contexts and fuzzy semantics, can accurately understand user needs, and ensure the accuracy and efficiency of task execution.
[0122] The technical solution of the present application is implemented based on the following principles. Specifically, when voice input is obtained, voice recognition is performed on the voice input to obtain text information corresponding to the voice input, and then the text information is parsed using a preset language model to obtain task information.
[0123] In this process, text information is input into the preset language model. Compared with the solution of directly using voice input, there is no need to train voice recognition on the preset language model, which reduces the cost of model training.
[0124] Among them, the preset language model is a large language model, that is, a large language model (LLM) is an artificial intelligence model designed to understand and generate human language. They are trained on a large amount of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.
[0125] In the technical solution of the present application, a large language model is used to accurately understand user needs and ensure the accuracy of the output task information, thereby improving the success rate of task execution.
[0126] In some technical solutions, optionally, the large language model is a large pre-trained language model that has been fine-tuned and optimized, which can be a Chat Generative Pre-trained Transformer (ChatGPT), which can be understood as a chatbot program. T is a natural language processing tool driven by artificial intelligence technology. It can conduct conversations by understanding and learning human language, and can also interact according to the context of the chat, truly chat and communicate like humans, and can even complete tasks such as writing emails, video scripts, copywriting, translation, code, and writing papers.
[0127] In some technical solutions, a large pre-trained language model is optionally used to improve the model's ability to understand complex contexts and fuzzy semantics through fine-tuning and optimization, overcome the problem of insufficient semantic understanding ability, thereby more accurately understanding user needs and improving the success rate of task execution.
[0128] In some technical solutions, optionally, the task information is a structured task instruction.
[0129] In some technical solutions, optionally, in the process of generating task information, the voice control method also includes: generating natural language feedback information based on a preset language model; before executing the task corresponding to the task information, it also includes: synthesizing voice information based on the feedback information; and playing the voice information.
[0130] In this technical solution, a preset language model is used to generate feedback information in the form of natural language. In this process, the advantages of the preset language model can be used to ensure that the output feedback information matches the user's voice input. By playing the voice information synthesized based on the feedback information, voice interaction between the device and the user can be achieved. In this process, a smooth interaction can be achieved between the device and the user, thereby improving the interactive experience of the device.
[0131] In some technical solutions, optionally, speech synthesis is to convert text information into speech signals understandable to humans.
[0132] In the present invention, speech synthesis is used to convert the feedback information of natural language into speech signals (ie, speech information) for the convenience of users to listen.
[0133] In some technical solutions, the voice control method also includes: outputting feedback information.
[0134] In this technical solution, feedback information can be output in the form of text, so that the device can display feedback information to the user in different forms, thereby meeting the interaction needs of different scenarios.
[0135] In some technical solutions, optionally, voice recognition is performed on the voice input to obtain text information, specifically including: using a deep learning-based voice recognition model to perform voice recognition on the voice input to obtain text information.
[0136] In this technical solution, the speech recognition model based on deep learning has better speech synthesis ability. By adopting the above speech training model, the text information obtained by recognition can accurately express the user's ideas. Compared with the recognition solution that adopts word-by-word text recognition, the technical solution of this application can more accurately express the meaning expressed by the speech input, and then fully understand the meaning of the speech input.
[0137] In some technical solutions, speech recognition is optionally the process of converting human speech signals into text information that can be processed by a computer. In the present invention, speech recognition technology is used to convert the user's speech input into text to facilitate subsequent interaction with a preset language model.
[0138] In some technical solutions, optionally, the speech recognition model can be a long short-term memory network (Long Short-Term Memory, LSTM) or a natural language processing model.
[0139] Specifically, advanced speech recognition models based on deep learning (such as models based on LSTM networks or Transformer) are used to improve the accuracy and robustness of speech recognition, so that the system can effectively recognize user voice input in various environments.
[0140] Among them, Natural Language Processing (NLP) is an interdisciplinary field of computer science, artificial intelligence and linguistics, dedicated to enabling computers to understand, parse and generate human language. In the present invention, natural language processing technology is used to parse the user's voice input, extract key information, and interact with the preset language model to generate structured task instructions (i.e., task information) and natural language feedback (i.e., feedback information).
[0141] In some technical solutions, optionally, voice information is synthesized based on the feedback information, specifically including: performing voice synthesis on the feedback information based on an end-to-end voice synthesis model based on deep learning to obtain voice information.
[0142] In this technical solution, voice information is synthesized so that the user and the device can interact through voice, thus overcoming the dependence of the commands used to control the operation of the device on manual input or a fixed command set in the related technical solutions, thereby improving the interactive experience between the user and the device.
[0143] In some technical solutions, optionally, the end-to-end speech synthesis model can be a Tacotron-based speech synthesis model.
[0144] In this technical solution, the above-mentioned end-to-end speech synthesis model is used to make the sentences of the generated voice information more natural and fluent, and the interaction between the user and the device more natural and convenient, thereby improving the user experience and acceptance.
[0145] In the above technical solution, through voice input and output and natural language processing technology, efficient communication between users and devices is achieved, improving interaction efficiency and user experience.
[0146] In some technical solutions, optionally, the task information includes at least one task time and a corresponding task execution location; executing the task corresponding to the task information specifically includes: determining a task execution path based on at least one task time and a corresponding task execution location; and executing the task corresponding to the task information based on the task execution path.
[0147] In this technical solution, when the voice input contains multiple tasks, the preset language model can parse out the task time corresponding to each task and the task execution location corresponding to the task time during the parsing of text information, and then determine the task execution path based on the parsed task time and task execution location, so that the device can use the task execution path as a guide for task execution during the operation process, thereby more accurately understanding user needs and improving the success rate of task execution.
[0148] In the above technical solution, the task execution path can be understood as the order in which multiple tasks are executed in sequence when there are multiple tasks.
[0149] In some technical solutions, the task execution path uses the task time corresponding to each task as a sorting condition, wherein the task time includes the start time and the end time when the task is executed, so as to improve the reliability of task execution.
[0150] In some technical solutions, optionally, the device is a cleaning robot or a service robot, and the voice control method also includes: obtaining current location information of the device; determining a navigation path based on the current location information and the task execution path; wherein the device walks based on the navigation path.
[0151] In this technical solution, during the operation of the equipment, the current position information is obtained so as to guide the movement of the equipment according to the current position information and the task execution path. In this process, the equipment can achieve autonomous walking and automatically perform operations.
[0152] In this process, the user does not need to participate in the movement of the equipment, which overcomes the disadvantage that the equipment requires manual participation of the user to operate and improves the efficiency of the operation.
[0153] In some technical solutions, optionally, the service robot can be a guest room robot in a hotel, or a mobile robot used to serve food in a restaurant. The service robot can deploy different functions according to its different usage scenarios, which will not be elaborated here.
[0154] In some technical solutions, optionally, the current location information can be obtained through a location sensor set on the device.
[0155] Among them, the position sensor can select the corresponding sensor according to the actual usage scenario, and its specific selection will not be repeated here.
[0156] In some technical solutions, optionally, it also includes: obtaining environmental information of the environment in which the device is located; and updating the navigation path based on the environmental information.
[0157] In this technical solution, environmental information is obtained so that the device can update the navigation path according to the environmental information during walking, reduce collisions with other objects in the environment, thereby achieving dynamic obstacle avoidance, protecting the safety of the device itself to the greatest extent, and reducing the impact of the device operation on other objects in the environment.
[0158] In some technical solutions, optionally, the environmental information includes placement information of other objects in the environment except the device.
[0159] In some technical solutions, optionally, the environmental information includes not only the placement position information of other objects, but also the posture information of other objects.
[0160] In this technical solution, by obtaining the above information, obstacles can be avoided to the greatest extent possible.
[0161] In some technical solutions, optionally, environmental information is acquired and perceived using one or more of a camera, a lidar, or an infrared sensor.
[0162] In some technical solutions, optionally, it also includes: when the task information includes keywords for indicating repeated execution, determining the start time, end time and target task associated with the keywords; and determining the periodic task based on the start time, end time and target task.
[0163] In this technical solution, the keywords may be words that can express repeated execution, such as weekly, every interval, monthly, and daily. They can be selected according to the actual usage scenario and will not be described in detail here.
[0164] In the above technical solution, the start time, end time and target task can be obtained by recognizing and parsing the voice input, which will not be described in detail here.
[0165] In the above technical solution, by determining periodic tasks so that the device can perform tasks periodically, there is no need for the user to repeatedly issue voice input instructions, which helps reduce the frequency of interaction between the user and the device, thereby improving the user experience of the device.
[0166] In some technical solutions, optionally, a preset language model is used to parse the text information to obtain task information, including: sending the text information to a preset language model deployed on a server; and receiving the task information output from the preset language model.
[0167] In this technical solution, text information is sent out so that the preset language model deployed on the server can process the received text information and obtain task information. In this process, the text information processing process can be transferred from the local device to the cloud, that is, the server side, and the server's large and sufficient computing power is used to process data, thereby increasing the speed of data processing to meet different needs in different scenarios.
[0168] In some embodiments, Figure 2 As shown, the present invention provides a voice control device 200 for a device, including: a human-computer interaction module 202, used to obtain a user's voice input; a voice recognition module 204, used to perform voice recognition on the voice input to obtain text information; a generation module 206, used to parse the text information using a preset language model to obtain task information; an execution module 208, used to execute a task corresponding to the task information.
[0169] The technical solution of the present application proposes a voice control device 200 for a device, which can improve the recognition ability of voice input, and thus has better processing performance in processing complex contexts and fuzzy semantics, and can accurately understand the needs of users, ensuring the accuracy and efficiency of task execution.
[0170] The technical solution of the present application is implemented based on the following principles. Specifically, when voice input is obtained, voice recognition is performed on the voice input to obtain text information corresponding to the voice input, and then the text information is parsed using a preset language model to obtain task information.
[0171] In this process, text information is input into the preset language model. Compared with the solution of directly using voice input, there is no need to train voice recognition on the preset language model, which reduces the cost of model training.
[0172] Among them, the preset language model is a large language model, that is, a large language model (LLM) is an artificial intelligence model designed to understand and generate human language. They are trained on a large amount of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis.
[0173] In the technical solution of the present application, a large language model is used to accurately understand user needs and ensure the accuracy of the output task information, thereby improving the success rate of task execution.
[0174] In some technical solutions, optionally, the large language model is a large pre-trained language model that has been fine-tuned and optimized, which can be a Chat Generative Pre-trained Transformer (ChatGPT), which can be understood as a chatbot program. T is a natural language processing tool driven by artificial intelligence technology. It can conduct conversations by understanding and learning human language, and can also interact according to the context of the chat, truly chat and communicate like humans, and can even complete tasks such as writing emails, video scripts, copywriting, translation, code, and writing papers.
[0175] In some technical solutions, optionally, the task information is a structured task instruction.
[0176] In some technical solutions, optionally, in the process of generating task information, the generation module is also used to: generate natural language feedback information based on a preset language model; before executing the task corresponding to the task information, the voice control device of the device also includes: a speech synthesis module, used to synthesize voice information based on the feedback information; and play the voice information.
[0177] In this technical solution, a preset language model is used to generate feedback information in the form of natural language. In this process, the advantages of the preset language model can be used to ensure that the output feedback information matches the user's voice input. By playing the voice information synthesized based on the feedback information, voice interaction between the device and the user can be achieved. In this process, a smooth interaction can be achieved between the device and the user, thereby improving the interactive experience of the device.
[0178] In some technical solutions, optionally, speech synthesis is to convert text information into speech signals understandable to humans.
[0179] In the present invention, speech synthesis is used to convert the feedback information of natural language into speech signals (ie, speech information) for the convenience of users to listen.
[0180] In some technical solutions, the voice control method also includes: outputting feedback information.
[0181] In this technical solution, feedback information can be output in the form of text, so that the device can display feedback information to the user in different forms, thereby meeting the interaction needs of different scenarios.
[0182] In some technical solutions, optionally, the speech recognition module is specifically used to: use a deep learning-based speech recognition model to perform speech recognition on speech input to obtain text information.
[0183] In this technical solution, the speech recognition model based on deep learning has better speech synthesis ability. By adopting the above speech training model, the text information obtained by recognition can accurately express the user's ideas. Compared with the recognition solution that adopts word-by-word text recognition, the technical solution of this application can more accurately express the meaning expressed by the speech input, and then fully understand the meaning of the speech input.
[0184] In some technical solutions, speech recognition is optionally the process of converting human speech signals into text information that can be processed by a computer. In the present invention, speech recognition technology is used to convert the user's speech input into text to facilitate subsequent interaction with a preset language model.
[0185] In some technical solutions, optionally, the speech recognition model can be a long short-term memory network (Long Short-Term Memory, LSTM) or a natural language processing model.
[0186] Among them, Natural Language Processing (NLP) is an interdisciplinary field of computer science, artificial intelligence and linguistics, dedicated to enabling computers to understand, parse and generate human language. In the present invention, natural language processing technology is used to parse the user's voice input, extract key information, and interact with the preset language model to generate structured task instructions (i.e., task information) and natural language feedback (i.e., feedback information).
[0187] In some technical solutions, optionally, the speech synthesis module is specifically used to: perform speech synthesis on the feedback information based on an end-to-end speech synthesis model based on deep learning to obtain speech information.
[0188] In this technical solution, voice information is synthesized so that the user and the device can interact through voice, thus overcoming the dependence of the commands used to control the operation of the device on manual input or a fixed command set in the related technical solutions, thereby improving the interactive experience between the user and the device.
[0189] In some technical solutions, optionally, the end-to-end speech synthesis model can be a Tacotron-based speech synthesis model.
[0190] In this technical solution, the above-mentioned end-to-end speech synthesis model is used to make the sentences of the generated voice information more natural and fluent, and the interaction between the user and the device more natural and convenient, thereby improving the user experience and acceptance.
[0191] Specifically, the device takes a robot as an example, for example:
[0192] 1. The user sends navigation requests to the system through voice, for example: "Take me to the nearest coffee shop."
[0193] 2. The speech recognition module receives the speech input and converts it into text: "Take me to the nearest coffee shop".
[0194] 3. The generation module parses the text and generates structured task instructions, such as: {"action":"navigate","target":"nearest_coffee_shop"}, and natural language feedback, such as: "OK, we are planning a route to the nearest coffee shop for you."
[0195] 4. The speech synthesis module converts natural language feedback into speech signals.
[0196] 5. The human-computer interaction module transmits voice signals to the user through the speaker, informing the user that the route is being planned.
[0197] 6. The robot task execution module receives the structured task instructions, parses the instructions and plans the navigation path. Then, the robot starts to execute the navigation task according to the planned path.
[0198] 7. During the execution of the task, the system can provide natural language feedback according to the real-time situation, for example: "Turn left 100 meters ahead", and convert it into a voice signal through the speech synthesis module, and finally convey it to the user through the human-computer interaction module.
[0199] In this embodiment, the deficiencies in speech recognition, semantic understanding and human-computer interaction in the prior art are effectively solved. At the same time, the system provides a more natural and convenient human-computer interaction experience, allowing users to easily complete navigation tasks in various scenarios.
[0200] For example, in an indoor environment, the embodiments of the present application can also be applied to the implementation of complex time and task related functions, such as, for example, the following voice input: "Clean the bedroom at 3 pm, mop the living room and kitchen before 5 pm, and then go back to dry the rag." The steps are as follows:
[0201] 1. The voice recognition module converts the voice command into text: "Clean the bedroom at 3 pm, mop the living room and kitchen before 5 pm, and then go back to dry the rag."
[0202] 2. The generation module parses the text and generates structured task instructions, for example:
[0203] jsonCopy code[{"action":"clean","location":"bedroom","time":"15:00"},{"action":"mop","location":["living_room","kitchen"],"deadline":"17:00"},{"action":"dry","object":"cleaning_cloth","after_previous":true}], while generating natural language feedback: "OK, I will clean the bedroom at 3pm, mop the living room and kitchen before 5pm, and dry the mop when I'm done".
[0204] 3. The speech synthesis module converts natural language feedback into speech signals.
[0205] 4. The human-computer interaction module transmits voice signals to the user through the speaker to inform the user of the task plan.
[0206] 5. The execution module performs the corresponding tasks at the specified time and place according to the generated structured task instructions.
[0207] Specifically, first, perform the task of cleaning the bedroom at 3 pm; then, complete the mopping tasks in the living room and kitchen before 5 pm; finally, dry the rag according to the order of tasks. In the above embodiment, an advanced deep learning-based speech recognition model is used to effectively recognize user voice input in different accents, speaking speeds, and noise environments. Through targeted training, high-accuracy speech recognition can be achieved, and the accuracy and robustness of speech recognition can be improved, so that the system can better understand user needs.
[0208] By fine-tuning and optimizing large pre-trained language models (such as ChatGPT), the model's ability to understand complex contexts and fuzzy semantics is improved, and accurate semantic understanding and task instruction generation are achieved. It can generate accurate structured task instructions and natural language feedback based on user input, providing users with precise task execution.
[0209] Advanced speech synthesis technology is used to convert natural language feedback into voice signals, providing a natural and smooth human-computer interaction experience. Users can communicate with the system through simple voice commands, achieving simple and efficient operation and natural and smooth human-computer interaction.
[0210] The execution module parses and plans the path according to the structured task instructions, adjusts the robot's movement, and controls sensors and other equipment to meet user needs. This optimized task execution method can improve the robot's work efficiency, reduce energy consumption, achieve more efficient task completion, and realize flexible indoor and outdoor navigation task execution.
[0211] Combining multiple modules such as speech recognition, semantic understanding, human-computer interaction and task execution, it provides users with a more natural and convenient navigation experience. Users can easily complete navigation tasks in various scenarios, improving their quality of life and work efficiency.
[0212] In the above technical solutions, the above technical solutions improve the accuracy and naturalness of speech recognition, semantic understanding and human-computer interaction, and can also achieve flexible and efficient task execution in various indoor and outdoor scenarios. At the same time, the optimized task execution module helps to improve the robot's work efficiency and reduce energy consumption, achieve more efficient task completion, and bring a better experience to users.
[0213] The technical solution proposed in this application adopts a modular design, and the various modules have good coordination and compatibility, and are easy to expand and upgrade, so that the system can be easily expanded and upgraded during future technological development to adapt to changing user needs and technological advances.
[0214] In addition, the above technical solution is not only suitable for indoor navigation tasks in a home environment, but also supports multi-scenario applications. For example, it can be widely used in multiple fields such as commerce, industry, and medical care to achieve voice semantic navigation in various scenarios, which further improves the practicality and market value of the present invention.
[0215] At the same time, by optimizing the robot's task execution module, more accurate navigation and obstacle avoidance functions can be achieved, effectively reducing possible safety risks during task execution. In addition, the human-computer interaction module can provide real-time feedback on task execution and abnormal situations, improve the user's real-time monitoring ability of robot operations, ensure safety during use, and enhance safety.
[0216] It should be noted that natural language interaction reduces the difficulty of communication between users and robots. Users do not need to learn complex operating interfaces and instructions, but can complete task settings through simple voice commands, which greatly reduces the threshold and difficulty of use.
[0217] In some technical solutions, optionally, the task information includes at least one task time and a corresponding task execution location; the execution module is specifically used to: determine the task execution path based on at least one task time and the corresponding task execution location; and execute the task corresponding to the task information based on the task execution path.
[0218] In this technical solution, when the voice input contains multiple tasks, the preset language model can parse out the task time corresponding to each task and the task execution location corresponding to the task time during the parsing of text information, and then determine the task execution path based on the parsed task time and task execution location, so that the device can use the task execution path as a guide for task execution during the operation process, thereby more accurately understanding user needs and improving the success rate of task execution.
[0219] In the above technical solution, the task execution path can be understood as the order in which multiple tasks are executed in sequence when there are multiple tasks.
[0220] In some technical solutions, the task execution path uses the task time corresponding to each task as a sorting condition, wherein the task time includes the start time and the end time when the task is executed, so as to improve the reliability of task execution.
[0221] In addition, the modular design ensures good collaboration and compatibility between modules, and can be easily expanded and upgraded to adapt to changing user needs and technological advances.
[0222] In some technical solutions, optionally, the device is a cleaning robot or a service robot, and the voice control device also includes: a position information module, used to obtain the current position information of the device; a path planning module, used to determine the navigation path according to the current position information and the task execution path; wherein the device walks based on the navigation path.
[0223] In this technical solution, during the operation of the equipment, the current position information is obtained so as to guide the movement of the equipment according to the current position information and the task execution path. In this process, the equipment can achieve autonomous walking and automatically perform operations.
[0224] In this process, the user does not need to participate in the movement of the equipment, which overcomes the disadvantage that the equipment requires manual participation of the user to operate and improves the efficiency of the operation.
[0225] In some technical solutions, optionally, the service robot can be a guest room robot in a hotel, or a mobile robot used to serve food in a restaurant. The service robot can deploy different functions according to its different usage scenarios, which will not be elaborated here.
[0226] In some technical solutions, optionally, the current location information can be obtained through a location sensor set on the device.
[0227] Among them, the position sensor can select the corresponding sensor according to the actual usage scenario, and its specific selection will not be repeated here.
[0228] In this embodiment, it is not only applicable to indoor navigation tasks in a home environment, but can also be widely used in multiple fields such as commerce, industry, and medical treatment to achieve voice semantic navigation in various scenarios, which further improves practicality and market value.
[0229] In some technical solutions, optionally, it also includes: a sensor module, which is used to obtain environmental information of the environment in which the device is located; and a path planning module is also used to update the navigation path based on the environmental information.
[0230] In this technical solution, environmental information is obtained so that the device can update the navigation path according to the environmental information during walking, reduce collisions with other objects in the environment, thereby achieving dynamic obstacle avoidance, protecting the safety of the device itself to the greatest extent, and reducing the impact of the device operation on other objects in the environment.
[0231] In some technical solutions, optionally, the environmental information includes placement information of other objects in the environment except the device.
[0232] In some technical solutions, optionally, the environmental information includes not only the placement position information of other objects, but also the posture information of other objects.
[0233] In this technical solution, by obtaining the above information, obstacles can be avoided to the greatest extent possible.
[0234] In some technical solutions, optionally, environmental information is acquired and perceived using one or more of a camera, a lidar, or an infrared sensor.
[0235] In this embodiment, more accurate navigation and obstacle avoidance functions are achieved through optimization of the modules, effectively reducing safety risks that may occur during mission execution.
[0236] In addition, the human-computer interaction module can provide real-time feedback on task execution status and abnormal situations, improve the user's real-time monitoring ability of robot operations, and ensure safety during use.
[0237] In some technical solutions, optionally, it also includes: a time management module, which is used to determine the start time, end time and target task associated with the keywords when the task information includes keywords for indicating repeated execution; and determine the periodic task based on the start time, end time and target task.
[0238] In this technical solution, the keywords may be words that can express repeated execution, such as weekly, every interval, monthly, and daily. They can be selected according to the actual usage scenario and will not be described in detail here.
[0239] In the above technical solution, the start time, end time and target task can be obtained by recognizing and parsing the voice input, which will not be described in detail here.
[0240] In the above technical solution, by determining periodic tasks so that the device can perform tasks periodically, there is no need for the user to repeatedly issue voice input instructions, which helps reduce the frequency of interaction between the user and the device, thereby improving the user experience of the device.
[0241] In some technical solutions, optionally, it also includes: a cloud computing module, used to send text information to a preset language model deployed on a server; and receive task information output from the preset language model.
[0242] In this technical solution, text information is sent out so that the preset language model deployed on the server can process the received text information and obtain task information. In this process, the text information processing process can be transferred from the local device to the cloud, that is, the server side, and the server's large and sufficient computing power is used to process data, thereby increasing the speed of data processing to meet different needs in different scenarios.
[0243] In some embodiments, Figure 4 As shown, the mobile phone software (application, APP) or the software deployed on the device, or the web page obtains the voice input and outputs it to the server. The Flask backend (a framework) in the server outputs it to ChatGPT, which combines Low-Rank Adaptation of Large Language Models, LORA (a model) to feedback feedback information and task information.
[0244] Among them, the Manager in the device outputs to the Planner, among which AI (artificial intelligence) performs framework management.
[0245] In some embodiments, Figure 3 As shown, the present invention provides a voice control device 300 of a device, including a processor 302 and a memory 304, wherein the memory 304 stores programs or instructions that can be run on the processor 302, and when the programs or instructions are executed by the processor 302, the steps of the method in any one of the above embodiments are implemented.
[0246] In some embodiments, the present invention provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method in any of the above embodiments are implemented.
[0247] In some embodiments, the present invention provides a device, comprising: a voice control device of the device in any of the above aspects; and / or a readable storage medium as described above.
[0248] In some technical solutions, optionally, the device includes one of the following: a cleaning robot or a service robot.
[0249] The term "first" or "second" in the specification and claims of the present application may include one or more of the features explicitly or implicitly. In the textual description of the present invention, unless otherwise specified, "plurality" means two or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the objects connected before and after are in an "or" relationship.
[0250] In the claims, specification and drawings of the present invention, the term "multiple" refers to two or more than two. Unless otherwise clearly defined, the orientation or position relationship indicated by the terms "upper" and "lower" is based on the orientation or position relationship shown in the drawings, which is only for the purpose of more conveniently describing the present invention and making the description process simpler, rather than indicating or implying that the device or element referred to must have the specific orientation described, be constructed and operated in a specific orientation, so these descriptions cannot be understood as limiting the present invention; the terms "connect", "install", "fix" and the like should be understood in a broad sense. For example, "connection" can be a fixed connection between multiple objects, or a detachable connection between multiple objects, or an integral connection; it can be a direct connection between multiple objects, or an indirect connection between multiple objects through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood based on the specific circumstances of the above data.
[0251] In the claims, specification and drawings of the present invention, the description of the terms "one embodiment", "some embodiments", "specific embodiments" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the claims, specification and drawings of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0252] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for voice control of a device, It is characterized in that include: Get the user's voice input; Performing speech recognition on the speech input to obtain text information; Using a preset language model to parse the text information to obtain task information; Execute the task corresponding to the task information.
2. The method for voice control of a device according to claim 1, It is characterized in that In the process of generating the task information, the voice control method further includes: Generate natural language feedback information based on the preset language model; Before executing the task corresponding to the task information, the method further includes: synthesizing speech information based on the feedback information; Play the voice message.
3. The method for voice control of a device according to claim 1, It is characterized in that The performing speech recognition on the speech input to obtain text information specifically includes: The speech input is subjected to speech recognition using a speech recognition model based on deep learning to obtain the text information.
4. The method for voice control of a device according to claim 2, It is characterized in that The synthesizing speech information based on the feedback information specifically includes: Based on an end-to-end speech synthesis model of deep learning, speech synthesis is performed on the feedback information to obtain the speech information.
5. A method for voice control of a device according to any one of claims 1 to 4, It is characterized in that The task information includes at least one task time and a corresponding task execution location; The executing the task corresponding to the task information specifically includes: Determine a task execution path based on at least one of the task time and the corresponding task execution location; Based on the task execution path, a task corresponding to the task information is executed.
6. The method for voice control of a device according to claim 5, It is characterized in that The device is a cleaning robot or a service robot, and the voice control method further includes: Obtaining current location information of the device; Determine a navigation path according to the current location information and the task execution path; Wherein, the device walks based on the navigation path.
7. The method for voice control of a device according to claim 6, It is characterized in that Also includes: Obtaining environmental information of the environment in which the device is located; The navigation path is updated based on the environmental information.
8. The method for voice control of a device according to claim 5, It is characterized in that Also includes: In a case where the task information includes a keyword for indicating repeated execution, determining a start time, an end time, and a target task associated with the keyword; A periodic task is determined based on the start time, the end time, and the target task.
9. The method for voice control of a device according to claim 5, It is characterized in that The adopting a preset language model to parse the text information to obtain task information includes: Sending the text information to the preset language model deployed on the server; Receive task information output from the preset language model.
10. A voice control device for a device, It is characterized in that include: Human-computer interaction module, used to obtain user's voice input; A speech recognition module, used to perform speech recognition on the speech input to obtain text information; A generation module, used to parse the text information using a preset language model to obtain task information; An execution module is used to execute the task corresponding to the task information.
11. A voice control device for a device, It is characterized in that The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.
12. A readable storage medium, It is characterized in that The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
13. A device, It is characterized in that include: A voice control device for a device as claimed in claim 10 or 11; and / or The readable storage medium as claimed in claim 12.
14. The device according to claim 13, It is characterized in that The device comprises one of the following: Cleaning robot or service robot.
Citation Information
Cited By
Interaction method and device for upper limb cooperative control of humanoid robot
CN119927906A
An interaction method and device for collaborative control of the upper limbs of a humanoid robot
CN119927906B
Robot navigation method and system, storage medium and electronic equipment
CN120704335A
