Method for realizing voice control of IOT (Internet of Things) equipment and multi-brand voice assistant
By designing a general voice service platform server, the technical development problems of third-party brands in the interactive control of multi-brand voice assistants are solved, efficient unified management and voice control of multi-brand voice assistants on IOT devices are achieved, and development costs and work are reduced. , improves the flexibility of IoT devices.
Patent Information
- Application Number
- CN202310568734.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-22
AI Technical Summary
Third-party brands need to develop matching skills, actions or voice applications for different voice assistants, resulting in high technology development costs and high workloads, and it is difficult to achieve interactive control of multi-brand voice assistants.
Design a general voice service platform server to provide the ability to process voice requests. Through the design of a hierarchical architecture, including access layer, application layer, data access layer, external service access layer and message queue services, it supports multi-brand voice assistants unified management and voice control of IoT devices.
Provide brands with efficient and unified management of voice assistants for efficient and unified management of voice assistants for brands that do not have deep technical development capabilities, and support multiple voice assistants to control voice over the same IOT device, reducing development costs and workload, improving the flexibility of IOT devices, and adapting to Future demands of the Internet of Things industry.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present invention relates to the field of voice control technology, and in particular to a method for realizing voice control of an IOT device and a multi-brand voice assistant. Background Art
[0002] With the popularization of IoT applications and the continuous maturity of voice control technology, more and more consumers (or end users) are controlling IoT devices (e.g., smart home devices) by talking to voice assistants.
[0003] The voice service platform mainly provides services for voice assistants and voice assistant devices, enabling voice assistants or voice assistant devices to hear and respond to the voice commands of end users. The interaction between end users and voice assistants includes a series of requests and responses. Requests are questions asked by end users, and responses are answers to questions. In the context of IoT applications, the way to respond to end users is mainly reflected in executing device actions (for example, turning off lights, opening curtains, etc.) and voice responses.
[0004] Voice assistants refer to Baidu Xiaodu, Tmall Genie, Amazon Alexa, Google Assistant, etc. Voice assistants are generally installed on dedicated smart speakers, mobile phones or other hardware devices, which are collectively referred to as voice assistant devices. For example, Xiaodu smart speakers are equipped with Xiaodu voice assistant and Baidu's conversational artificial intelligence operating system DuerOS; the corresponding devices for Tmall Genie voice assistant are Tmall Genie Cube smart speakers, Tmall Genie CC smart screens, etc.; the corresponding devices for Amazon Alexa voice assistant can be Amazon Echo, Echo Dot and other smart speakers; the corresponding smart voice devices for Google Assistant can be Google Nest Mini, Google Home Mini, Google Home, Sonos One and other smart speakers.
[0005] Voice assistant devices typically have microphones, speakers, processors, memory, communication facilities, and other hardware and software that can detect and process human speech to obtain information requested on behalf of the end user. This information is expressed as a request message (or intent) according to a predefined protocol and is communicated to the server through a communication network.
[0006] On the voice assistant server, the voice application receives and processes the request message and determines the appropriate response. The response will be reflected in a response message expressed according to a predefined protocol. The response message is sent to the voice assistant device through the communication network. The voice assistant interprets the response message and speaks or renders (or takes the action specified by the response). The work of the voice application is supported by the infrastructure of the operating system and other processes running on the server. The service provided by the server to the client voice assistant device, which is able to interact with the end user, is called a voice assistant service (or skill, action or voice application).
[0007] Typically, voice servers, voice assistant devices, voice assistants, voice assistant services, predefined protocols, and basic voice applications are part of the voice assistant framework design. The framework also provides dedicated APIs for third-party brands to create skills that enable voice assistants to interact with their own brand devices.
[0008] Third-party brands hope that their own brand devices can interact and control with voice assistants on multiple platforms at the same time. Generally, third-party brands need to develop according to the skills, actions or voice applications defined by different voice assistants to match the functions of their own brand devices. This requires high technical capabilities and a huge workload. If different voice assistants update skills, actions or voice applications, third-party brands need to respond to development in a timely manner to meet the needs of end users, and maintenance costs will also increase. Summary of the invention
[0009] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for realizing voice control of IOT devices and multi-brand voice assistants. By designing a universal voice service platform server, brand owners who do not have deep technical development capabilities can manage voice assistants efficiently and uniformly. The platform developers provide the basic construction and capability maintenance of related voice services, allowing brand owners to focus more on the upgrade of smart hardware products. During the use of end users, the same IOT device can be voice-controlled by multiple voice assistants at the same time, which improves the flexibility of IOT device control and adapts to the future needs of the development of the Internet of Things industry.
[0010] The technical solution adopted by the present invention to solve the technical problem is: a method for realizing voice control of an IOT device and a multi-brand voice assistant, the method comprising a universal voice service platform server (105) capable of processing voice requests, the service clients of the universal voice service platform server (105) comprising terminal users (101) and brand users; the terminal user (101) sends a request and / or obtains a response to the universal voice service platform server (105) through a voice assistant device (102) and a voice assistant server (104); the brand user operates and uses the voice service platform interface (108), the voice service platform interface (108) builds a SaaS platform based on a user interface (121), and performs information configuration with the universal voice service platform server (105); the universal voice service platform server (105) sends instructions and / or reports attributes to the IOT device (107) through a voice service application (106).
[0011] Furthermore, one of the IOT devices (107) implements voice control with one or more terminal users (101) and / or brand users through the universal voice service platform server (105).
[0012] Furthermore, the universal voice service platform server (105) uses a layered architecture and has no dependency on other layers in the system. The universal voice service platform server (105) includes an access layer, an application layer, a data access layer, an external service access layer, a data storage layer, and a message queue service.
[0013] Furthermore, the access layer is used to provide an Open API to external services or management consoles for access, and to provide the ability for devices or services to maintain long connections; the application layer includes voice service applications that handle specific business logic for various users; the data access layer is used to provide the ability to provide upper-layer applications with data access to this system; the external service access layer is used to integrate the access capabilities of other sub-services of the platform and receive Xlink internal MQ messages.
[0014] Furthermore, the terminal user (101) sends a request (111) through oral voice, and the voice assistant device (102) equipped with a voice assistant receives the request (111) and processes it into request information (113); the request information (113) is transmitted to the voice assistant server (104) through the communication network (103); the voice assistant server (104) parses and generates platform request information (115) according to the design framework of its own platform, and transmits it to the general voice service platform server (105).
[0015] Furthermore, the general voice service platform server (105) sends an instruction (117) to the voice service application (106); the voice service application (106) receives the instruction (117) and then processes it into an execution action (119); after receiving the execution action (119), the IOT device (107) executes the operation and reports the device status (120) to the voice service application (106); the voice service application (106) reports the attribute (118) to the general voice service platform server (105).
[0016] Furthermore, the universal voice service platform server (105) processes the received request information (113) according to the mechanism required by the voice assistant server (104), and returns it to the voice assistant server (104) in the form of platform response information (116); after receiving the platform response information (116), the voice assistant server (104) generates a voice response message (114) and transmits it to the voice assistant device (102) via the communication network (103); the voice assistant device (102) returns a response (112) to the terminal user (101) in an audio format.
[0017] Furthermore, the brand user creates, stores and manages the voice assistant device through the voice service platform interface (108), sets the configuration information for interconnecting the voice assistant server with the general voice service platform server, configures the product information corresponding to the IOT device, and configures the command information and reported attribute information of the IOT device.
[0018] The beneficial effect of the present invention is that compared with the prior art, the method provided by the present invention realizes voice control of IOT devices and multi-brand voice assistants, by providing a universal voice service platform server for third-party brand users and providing the ability to process voice requests, so that brand users who do not have deep technical development capabilities can manage voice assistants efficiently and uniformly, thereby providing third-party brands with voice capabilities that can be quickly deployed. The voice capabilities can be applied to the quick access of voice assistants such as Baidu Xiaodu, Tmall Genie, Amazon Alexa, and Google Assistant.
[0019] The general voice service platform server provides an intuitive and easy-to-use user interface. Brand users do not need to have development technical capabilities. They can maintain voice-related configuration information and functional instructions of IoT products in the voice AI management module according to the operating instructions. In addition, the general voice service platform server will update functions and services from time to time, and will automatically develop them for third-party brand users who use the platform. In a sense, using unified voice capabilities to achieve the connection of voice assistants will help third-party brand users accelerate the launch of voice control functions and focus more on the upgrade of smart hardware products.
[0020] During the use process of end users, the same IOT device can be voice-controlled by multiple voice assistants at the same time, which improves the flexibility of IOT device control and adapts to the future needs of the development of the Internet of Things industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 This is a flow chart of the terminal user or brand user controlling the IOT device provided in Embodiment 1 and Embodiment 2 of the present invention.
[0022] Figure 2 This is a diagram of the operation interface of the brand user management voice assistant in Example 2 of the present invention.
[0023] Figure 3 This is a diagram of the operation interface for brand users to set configuration information for interconnecting the voice assistant server and the general voice service platform server in Example 2 of the present invention.
[0024] Figure 4 This is a diagram of the operating interface for brand users to configure product information corresponding to IOT devices in Example 2 of the present invention.
[0025] Figure 5 This is a diagram of the operation interface for brand users to configure the sending instruction information of the IOT device in Example 2 of the present invention.
[0026] Figure 6 This is a diagram of the operating interface for brand users to configure reporting attribute information of IOT devices in Example 2 of the present invention.
[0027] Figure 7 It is a schematic diagram of the architecture design of the universal voice service platform server in the present invention.
[0028] Among them, 101-terminal user; 102-voice assistant device; 103-communication network; 104-voice assistant server; 105-general voice service platform server; 106-voice service application; 107-IOT device; 108-voice service platform interface; 111-request; 112-response; 113-request information; 114-response information; 115-platform request information; 116-platform response information; 117-instruction; 118-attribute; 119-execution action; 120-device status. Implementation
[0029] The present invention is further described below by specific examples, but these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. Example
[0030] like Figure 1 As shown, the end user controls the entire process of IOT devices through a voice assistant.
[0031] The terminal user 101 sends a request (intention) 111 through oral voice, for example, "Xiaodu Xiaodu, turn on the living room light". After receiving the request, the voice assistant device 102 equipped with the voice assistant processes it into request information 113. The request information 113 is transmitted to the voice assistant server 104 through the communication network 103. The voice assistant server 104, such as the Baidu Xiaodu voice assistant server, parses and generates platform request information 115 according to the design framework of its own platform, and transmits it to the general voice service platform server 105. The general voice service platform server 105 sends an instruction 117 (or data endpoint, object model attribute) to the voice service application 106. After receiving the instruction, the voice service application 106 then processes it into an execution action 119 (or data endpoint, object model attribute). After receiving the execution action 119, the IoT device 107 performs the operation according to the instruction, for example, turning on the living room light. When the state of the IoT device 107 is that the light is turned on, the device state 120 is reported to the voice service application 106. The voice service application 106 reports the attribute 118 to the general voice service platform server 105. The general voice service platform server 105 processes the received information according to the mechanism required by the voice assistant server 104, and returns it to the voice assistant server 104 in the form of platform response information 116. After receiving the information, the voice assistant server 104 generates a voice response message 114 and transmits it to the voice assistant device 102 through the communication network 103. The voice assistant device 102 returns the response 112 to the end user 101 in an audio format. For example, the Xiaodu voice assistant answers: "The lights in the living room are on."
[0032] like Figure 7As shown, the standardized universal voice service platform server uses a layered architecture that is independent of the dependencies of other layers in the system, including the access layer, application layer, data access layer, external service access layer, data storage layer, and message queue service.
[0033] Voice-web-open: Provides an Open API for external services or management console access.
[0034] Voice-gateway: Provides the capability of long-term connection of devices or services.
[0035] Voice-baidu-xiaodu: A voice service application that handles the specific business logic of Baidu Xiaodu voice assistant.
[0036] Voice-tmall-genius: A voice service application that handles the specific business logic of Amazon Alexa.
[0037] Voice-amazon-alexa: A voice service application that handles the specific business logic of the Amazon Alexa voice assistant.
[0038] Voice-google-assistant: The voice service application that handles the specific business logic of the Google Assistant voice assistant.
[0039] Voice-xband: A voice service application that handles the specific business logic of brand-agnostic voice assistants.
[0040] Voice-service-base: Provides upper-layer applications with the ability to access data in this system.
[0041] Voice-service-intergate: integrates the access capabilities of other sub-services on the platform and receives Xlink internal MQ messages. Example
[0042] like Figure 1-7 As shown, brand users control the entire process of IOT devices through voice assistants.
[0043] Brand users, who mainly operate through the voice service platform interface 108. The voice service platform interface 108 builds a SaaS (Software as a Service) platform based on the user interface 121 for brand users to create, store and manage voice assistant devices 102 (such as Figure 3 ), set the configuration information of the interconnection between the voice assistant server 104 and the general voice service platform server 105 (such as Figure 4), configure the product information corresponding to the IOT device 107 (such as Figure 5 ), configure the instruction information issued by the IOT device 107 (such as Figure 6 ) and reporting attribute information (such as Figure 7 ). The brand user platform does not need to participate in the construction of the universal voice service platform server 105, and can create, edit, and manage the voice assistant device 102 through interface operations. The cloud-based universal voice service platform server 105 will update and enhance functions from time to time according to the iteration of the voice assistant server 104, and brand commercial users can also automatically obtain updates and enhancements. The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The patent protection scope of the present invention should be defined by the claims.
Claims
1. A method for realizing voice control of IOT devices and multi-brand voice assistants, characterized in that: The method includes a universal voice service platform server (105) capable of processing voice requests, and the service clients of the universal voice service platform server (105) include terminal users (101) and brand users; the terminal user (101) sends a request and / or obtains a response to the universal voice service platform server (105) through a voice assistant device (102) and a voice assistant server (104); the brand user operates and uses the voice service platform interface (108), and the voice service platform interface (108) builds a SaaS platform based on a user interface (121) and performs information configuration with the universal voice service platform server (105); the universal voice service platform server (105) sends instructions and / or reports attributes to an IOT device (107) through a voice service application (106).
2. A method for realizing voice control of an IOT device with a multi-brand voice assistant as claimed in claim 1, characterized in that: One of the IOT devices (107) implements voice control with one or more terminal users (101) and / or brand users through the universal voice service platform server (105).
3. A method for realizing voice control of an IOT device with a multi-brand voice assistant as claimed in claim 1, characterized in that: The universal voice service platform server (105) uses a layered architecture and has no dependency on other layers in the system. The universal voice service platform server (105) includes an access layer, an application layer, a data access layer, an external service access layer, a data storage layer, and a message queue service.
4. A method for realizing voice control of an IOT device with a multi-brand voice assistant as claimed in claim 3, characterized in that: The access layer is used to provide an Open API to external services or management consoles for access, and to provide the ability for devices or services to maintain long connections; the application layer includes voice service applications that handle specific business logic for various users; the data access layer is used to provide the ability to provide data access to the system to upper-layer applications; the external service access layer is used to integrate the access capabilities of other sub-services of the platform and receive Xlink internal MQ messages.
5. A method for realizing voice control of an IOT device with a multi-brand voice assistant as described in claim 1, 2 or 3, characterized in that: The terminal user (101) sends a request (111) through oral voice, and the voice assistant device (102) equipped with a voice assistant receives the request (111) and processes it into request information (113); the request information (113) is transmitted to the voice assistant server (104) through the communication network (103); the voice assistant server (104) parses and generates platform request information (115) according to the design framework of its own platform, and transmits it to the general voice service platform server (105).
6. A method for realizing voice control of an IOT device with a multi-brand voice assistant as claimed in claim 3, characterized in that: The general voice service platform server (105) sends an instruction (117) to the voice service application (106); the voice service application (106) receives the instruction (117) and then processes it into an execution action (119); after receiving the execution action (119), the IOT device (107) executes the operation and reports the device status (120) to the voice service application (106); the voice service application (106) reports the attribute (118) to the general voice service platform server (105).
7. A method for realizing voice control of an IOT device and a multi-brand voice assistant as claimed in claim 4, characterized in that: The universal voice service platform server (105) processes the received request information (113) according to the mechanism required by the voice assistant server (104), and returns it to the voice assistant server (104) in the form of platform response information (116); after receiving the platform response information (116), the voice assistant server (104) generates a voice response message (114) and transmits it to the voice assistant device (102) via the communication network (103); the voice assistant device (102) returns a response (112) to the terminal user (101) in an audio manner.
8. A method for realizing voice control of an IOT device with a multi-brand voice assistant as described in claim 1, 2 or 3, characterized in that: The brand user creates, stores and manages the voice assistant device through the voice service platform interface (108), sets the configuration information for interconnecting the voice assistant server with the general voice service platform server, configures the product information corresponding to the IOT device, and configures the command information and reported attribute information of the IOT device.
