Server, intelligent device and intelligent device control method

Abstract

The invention provides a server, intelligent equipment and an intelligent equipment control method. A server can obtain area device information, including a plurality of first areas and device information of intelligent devices in the first areas. And the server obtains the main control equipment of the first area according to the equipment information and informs each main control equipment. A master control device sends an instruction response request to a server after detecting a voice instruction of a user, and the server can obtain a target master control device and a target first area corresponding to the target master control device. And the server determines to-be-controlled intelligent equipment based on the target first area and the target main control equipment, and sends a response instruction to the to-be-controlled intelligent equipment. And the to-be-controlled intelligent equipment can execute the response instruction so as to meet user requirements. And the user is prevented from being inquired to start which equipment, so that the use experience of the user is improved.

Description

Server, smart device, and smart device control method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on November 23, 2022, with application number 202211476607.8, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present disclosure relates to the technical field of display devices, and in particular to a server, an intelligent device, and an intelligent device control method. Background Art

[0004] A smart home system is a multi-device network system built on IoT technology. Through network communication, it enables the coordinated operation of multiple devices to meet user needs. For example, a smart home system can be composed of smart terminal devices such as smart TVs, smart refrigerators, smart terminals, and Magic Cube screens. Users can interact with these smart terminal devices to control the playback of specific media on their smart TVs, or control the refrigeration temperature of their smart refrigerators by interacting with the Magic Cube screen, achieving multi-device coordinated operation.

[0005] Smart home systems can support intelligent voice control interactions, which rely on semantic recognition of user voice commands to control device operation. When implementing voice control, user voice commands can be ambiguous, such as "turn on the TV." If a smart home system includes multiple TVs, the system may not know which one the user wants to turn on and may therefore turn on all TVs simultaneously, impacting the user experience.

[0006] Current solutions often introduce all devices of that type to the user and then ask them which device they want to control. However, when the user needs to control a large number of devices of the same type, introducing all the devices to the user takes a long time, making the entire process cumbersome and affecting the user experience.

[0007] Summary of the Invention

[0008] In a first aspect, some embodiments of the present disclosure provide a server, comprising a communication module and a control module. The communication module is configured to establish a communication connection with a smart device; the control module is configured to:

[0009] Acquire regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions;

[0010] Acquire a master control device of the first area based on the device information, where each first area corresponds to one master control device;

[0011] In response to a command response request sent by the master device after detecting a user's voice command, obtaining a target master device and a target first area corresponding to the target master device; the command response request includes the voice command, and the voice command is used to control the smart device;

[0012] A smart device to be controlled is determined based on the target first area and the target main control device, and a response instruction is sent to the smart device to be controlled.

[0013] In a second aspect, some embodiments of the present disclosure provide a smart device comprising a display, a communicator, and a controller. The communicator is configured to establish a communication connection with a server and other smart devices; and the controller is configured to:

[0014] In response to a user's power-on instruction, the smart device is started and a startup message is sent to the server;

[0015] receiving device information sent by the server, calculating a capability score based on the device information and sending the score to the server, so that the server acquires the master device according to the capability score;

[0016] In response to the master device notification information sent by the server, determining that the smart device is the master device of the first area where the smart device is located;

[0017] In response to a voice command input by a user for controlling a smart device, a command response request is generated and sent to a server, so that the server obtains a target master device and a target first area corresponding to the target master device, and obtains a smart device to be controlled and sends a response command to the smart device to be controlled, wherein the command response request includes the voice command.

[0018] In a third aspect, some embodiments of the present disclosure provide a smart device control method, which is applied to a smart home system, wherein the smart home system includes a server and multiple smart devices, and the smart devices establish a communication connection with the server; the smart device control method includes:

[0019] The server obtains regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions;

[0020] The server obtains a master control device of the first area based on the device information, where each first area corresponds to a master control device;

[0021] After detecting the user's voice command, the smart device serving as the master device sends a command response request to the server, wherein the command response request includes the voice command, and the voice command is used to control the smart device;

[0022] The server obtains a target master device and a target first area corresponding to the target master device;

[0023] The server determines a smart device to be controlled based on the target first area and the target main control device, and sends a response instruction to the smart device to be controlled;

[0024] The smart device to be controlled executes the response instruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 illustrates a usage scenario of a smart home system according to some embodiments;

[0026] FIG2 is a block diagram of a hardware configuration of a smart device according to some embodiments;

[0027] FIG3 is a schematic diagram of an Internet of Things system according to some embodiments;

[0028] FIG4 is a schematic diagram illustrating the effect of multiple smart devices responding to voice interaction according to some embodiments;

[0029] FIG5 is a schematic diagram of a user interface according to some embodiments;

[0030] FIG6 is a schematic diagram of an application panel according to some embodiments;

[0031] FIG7 is a schematic diagram of a device control interface according to some embodiments;

[0032] FIG8 is a schematic diagram of a functional interface according to some embodiments;

[0033] FIG9 is a schematic diagram of a control area according to some embodiments;

[0034] FIG10 is a schematic diagram of a relay device list according to some embodiments;

[0035] FIG11 is a schematic diagram illustrating intelligent control mode confirmation information displayed on a display according to some embodiments;

[0036] FIG12 is a flowchart illustrating the interaction between various components of a smart home system according to some embodiments;

[0037] FIG13 is a schematic diagram of a confirmation interface of a master device according to some embodiments;

[0038] FIG14 is a schematic diagram of a master device switching interface according to some embodiments;

[0039] FIG15 is a schematic diagram of a rotation prompt interface according to some embodiments. DETAILED DESCRIPTION

[0040] The smart home system described in the embodiment of the present disclosure is a network system based on a specific area network and established based on a unified control service. The smart home system may include multiple smart devices 200 that establish communication connections with each other. Multiple smart devices 200 can achieve communication connections between devices by accessing the same local area network. Multiple smart devices 200 can also directly form a point-to-point network through a unified communication protocol to achieve communication connection. For example, multiple smart devices 200 can be connected to the same wireless local area network so that multiple smart devices 200 can communicate with each other. For example, a smart device 200 can also establish communication connections with multiple other smart devices 200 through Bluetooth, infrared, cellular network, power carrier communication, etc.

[0041] The smart devices 200 are devices with communication capabilities that can receive, send, and execute control commands to achieve specific functions. These smart devices 200 include, but are not limited to, smart display devices, smart terminals, smart home appliances, smart gateways, smart lighting devices, smart audio devices, and gaming devices. The multiple smart devices 200 that comprise a smart home system can be of the same or different types.

[0042] For example, as shown in Figure 1, the same smart home system may include a smart TV, a smart speaker, a smart refrigerator, multiple smart lamps, etc. These smart devices 200 can be distributed in different locations to meet the usage requirements at the corresponding locations.

[0043] It should be noted that the smart home system described in this disclosure does not limit the scope of application of the solutions to be protected by this disclosure. That is, in actual applications, the server, smart devices, and multi-device voice wake-up method provided in this disclosure are not limited to the smart home field but are also applicable to other systems that support intelligent voice control, such as smart office systems, smart service systems, smart management systems, and industrial production systems.

[0044] In a smart home system, users can operate each smart device 200 through a smart terminal 300 or a control device 100. In some embodiments, the control device 100 can be a remote control. Communication between the remote control and the smart device 200 includes infrared protocol communication, Bluetooth protocol communication, or at least one other short-range communication method, controlling the smart device 200 wirelessly or wired. Users can input user commands to control the smart device 200 through at least one method, such as buttons on the remote control, voice input, and control panel input.

[0045] Smart terminal 300 can perform the same functions as control device 100 and, based on the corresponding control functions of control device 100, supports further expanded functions and multi-device control. For example, smart terminal 300 can be a control device such as a mobile phone, tablet computer, or computer. A "universal remote control" application can be installed on smart terminal 300. By launching this application and selecting the model of smart device 200 or pairing it, the user can simulate the functions of control device 100 paired with smart device 200.

[0046] Smart device 200 has a specific hardware configuration depending on its actual function. As shown in FIG2 , taking a display device as an example, smart device 200 with a display function may include at least one of a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0047] In some embodiments, the controller 250 includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0048] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a controller, and a component for displaying video content, image content, and a menu control interface, as well as a user control UI interface, etc.

[0049] In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0050] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0051] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0052] In some embodiments, the controller 250 controls the operation of the smart device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the smart device 200. For example, in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform operations related to the object selected by the user command.

[0053] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus, etc.

[0054] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific sound or gesture, and the user input interface may recognize the sound or gesture through a sensor to receive the user input command.

[0055] In some embodiments, the smart device 200 also communicates data with the server 400. The smart device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the smart device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of server groups.

[0056] The smart devices 200 and server 400 that comprise the smart home system can communicate with each other to achieve comprehensive control of the smart home system. Specifically, the smart devices 200 can report data to the server 400 through applications, allowing the server 400 to analyze, determine, and process the data reported by the smart devices 200 to generate control instructions for the smart devices 200. The server 400 then issues the generated control instructions to the smart devices 200, thereby controlling the operating status of the smart devices 200.

[0057] Multiple smart devices 200 in the smart home system can also establish communication connections with each other. The smart device 200 can be an IOT (Internet of Things) device, and multiple smart devices 200 together constitute an IoT system. Figure 3 is a schematic diagram of an IoT system according to some embodiments. The IoT system generates and collects data from different dimensions through the IoT to achieve collaborative control of multiple smart devices 200. By connecting multiple smart devices 200 to the same wireless local area network, a communication connection relationship can be established between the multiple smart devices 200. Any device can send shared data to the IoT system, and any device can obtain data shared by other devices through the IoT system to achieve collaborative operation in the smart home system. The smart device 200 can also be an AIoT (Artificial Intelligence & Internet of Things) device, and multiple smart devices 200 can form an AIoT system.

[0058] In some embodiments, in order to improve the user experience, the smart device 200 may have an intelligent voice control function so that the user can control the smart device 200 by voice.

[0059] The smart device 200 may have a built-in intelligent voice system to support the user's intelligent voice control. The intelligent voice control refers to the interactive process in which the user operates the smart device 200 by inputting voice audio data. In order to achieve intelligent voice control, the smart device 200 may include an audio input device and an audio output device. Among them, the audio input device is used to collect the voice audio data input by the user, and may be a built-in or external microphone device of the smart device 200. The audio output device is used to make a sound to play a voice response. For example, as shown in Figure 4, when the user inputs a wake-up word such as "Hi! Xiao×" through the audio input device, the smart device 200 can play a voice response of "I am here" through the audio output device to guide the user to complete the subsequent voice input.

[0060] In some embodiments, the intelligent voice system built into the smart device 200 also supports a one-shot mode, that is, a "one-shot" mode. In this mode, the user can directly implement the control function through fewer voice inputs. For example, in the traditional mode, if the user wants to control the smart device 200 to play movie resources, he needs to first input the voice "Hi, Xiao ×", and after the smart device 200 responds "I'm here", he needs to input "I want to watch a movie", and then the smart device 200 will respond "The following movies were found for you". In the "one-shot" mode, the user can directly input "Hi! Xiao ×, I want to watch a movie", and the smart device 200 will directly respond "The following movies were found for you" after receiving the voice command, reducing the number of voice interactions and improving the efficiency of voice interactions.

[0061] In some embodiments, considering the user's usage experience, the smart device 200 may also support some other functions. For example, the smart device 200 may have the ability to control other smart devices, thereby controlling other smart devices to implement some functions.

[0062] For multiple smart devices 200 in the same smart home system, users can control the linkage of multiple devices through intelligent voice control. For example, a user can input the voice command "Turn on the bedroom light" through a smart speaker. The smart speaker can respond to this voice command by generating a control command for turning on the light. The control command is then sent to the lamp named "bedroom" in the smart home system to control the bedroom light to turn on. At the same time, the smart speaker also responds to the user's voice input by playing feedback voice content such as "I have turned on the bedroom light for you" to reply to the user.

[0063] When multiple smart devices 200 are linked and controlled, control instructions can be directly transmitted to the controlled device by the smart device 200 that receives the user's voice and audio data, or they can be transmitted to a specific relay device such as a router through the smart device 200, and then transmitted to the controlled device by the relay device. In some embodiments, control instructions can also be transmitted to the controlled device through the server 400. For example, when a user controls a smart device 200 in the smart home system through a smart terminal 300 outside the local area network where the smart home is located, the smart terminal 300 can first send the control instruction to the server 400, and the server 400 can then transmit the control instruction to the smart device 200 for control.

[0064] In some embodiments, the user can control other smart devices using the smart device 200. In the embodiments of the present disclosure, the control process is introduced by taking a display device as an example.

[0065] After the user controls the display device to turn on, the display device can display a user interface. Figure 5 is a schematic diagram of a user interface according to some embodiments. As shown in Figure 5, the user interface includes a first navigation bar 500, a second navigation bar 510, a function bar 520 and a content display area 530. The function bar 520 includes multiple function controls such as "viewing history", "my applications" and "my favorites". Among them, the content displayed in the content display area 530 will change with the changes in the selected controls in the first navigation bar 500 and the second navigation bar 510. The user can click on the "My Applications" control in the function bar 520 to input a display instruction for the application panel page to trigger entry into the corresponding application panel. It should be noted that the user can also input a selection operation for the function control in other ways to trigger entry into the application panel. For example, use voice control function or search function to control entry into the application panel page.

[0066] The user can view the applications installed on the display device, i.e., the functions supported by the display device, through the application panel. In the user interface shown in Figure 5, the user can select the "My Applications" control to trigger the display device to display the application panel. The user can then select and open an application to access that application's functions. It should be noted that applications installed on the display device can be system applications or third-party applications. Figure 6 is a schematic diagram of the application panel according to some embodiments. As shown in Figure 6, the application panel includes several controls: "Online Media," "Cable TV," "Device Control," and "Video Chat." The user can control the display device to watch online media by clicking the "Online Media" control. The user can use the display device to watch a variety of media channels, including various programs provided by cable TV providers, by clicking the "Cable TV" control. The user can use the display device to engage in video chat by clicking the "Video Chat" control. The user can use the display device to control other smart devices by clicking the "Device Control" control.

[0067] After the user clicks the device control control, the display device can obtain relevant information of other smart devices that can currently be controlled, thereby generating a device control interface and providing it to the user.

[0068] The display device can obtain the device information of other smart devices in the smart home system, including the device names of other smart devices and supported functions. The display device can send a request to the server to obtain the device information of other smart devices. When installing a smart home system, when a new smart device is connected, the device information of the smart device can be logged into the server. Therefore, the server can query the device information of each smart device and send it to the display device. After receiving the information sent by the server, the display device can collect information about other smart devices and generate a device control interface. Figure 7 is a schematic diagram of a device control interface according to some embodiments. As shown in Figure 7, the device control interface includes controls for other smart devices. Each control represents a smart device, and the device name can be displayed in the control, including the TV in the living room on the second floor, the refrigerator in the kitchen on the first floor, the speakers in the second bedroom, and the TV in the living room on the first floor. When the smart device logs information to the server, the user can customize the device name of the smart device to facilitate subsequent distinction. The user can select the smart device they want to control based on the device name in the device control interface.

[0069] When the user selects a smart device, the display device can display the functional interface of the smart device. Figure 8 is a schematic diagram of a functional interface according to some embodiments. As shown in Figure 8, when the user chooses to control the TV in the living room on the second floor, the display device can display the functional interface of the TV in the living room on the second floor, which includes the functions supported by the TV in the living room on the second floor, including playing cable TV, playing online media, playing music, turning on / off the TV, etc. When the user selects one of the functions, the display device can send a control instruction to the TV in the living room on the second floor to instruct it to implement the function. After receiving the control instruction, the TV in the living room on the second floor can execute the control instruction to implement the corresponding function.

[0070] In some embodiments, smart device 200 also has relay capabilities, which refers to the ability to continue displaying the playback screen of another smart device. For example, if smart device A is playing a certain media source, and smart device B has relay capabilities, smart device A can continue playing the media source, and the playback progress of the media source can be kept consistent during the relay.

[0071] When a user is using the smart device 200, for example, when using the smart device 200 to play a certain media asset, the user can call out the control area, which includes several functional controls to adjust the current media asset playback process. Figure 9 is a schematic diagram of a control area according to some embodiments. As shown in Figure 9, the control area includes but is not limited to at least one of a "relay" control, a "playlist" control, a "clarity" control, and a "recommended function" control. Among them, the user can trigger the smart device 200 to perform a corresponding operation by operating a certain control. It should be noted that the user can realize the relay function by operating the "relay" control, or can control the smart device 200 to realize the relay function by other means, such as using the voice control function.

[0072] When the user clicks the relay control, a relay device list may be displayed on the display, and the relay device list includes all relayable smart devices. FIG10 is a schematic diagram of a relay device list according to some embodiments. As shown in FIG10 , the relay device list includes several smart devices in an embodiment of the present disclosure. The user can select a smart device as a relay device. For example, when the user selects "Second Floor Living Room TV", the second floor living room TV can implement the relay function, thereby continuing to display media resources from the current device.

[0073] In the relay device list, the user can use the "?" control to trigger the relay function introduction page, which at least displays information about the relay function, such as relay operation instructions. The relay device list also includes a "Continue Playing on Original Device" toggle button for canceling the relay operation.

[0074] In some embodiments, when a user voice-controls a smart home system, the instructions they issue may be vague, and it may be difficult to determine which smart device the user wants to control. For example, the user may say, "Turn on the TV." Considering that the user may have multiple TVs in their home, it is impossible to determine which one the user needs to turn on. Turning on all the TVs would affect the user experience. Therefore, the user can be introduced to all devices of that type in the home, for example, "You currently have a TV in the first-floor living room, a TV in the second-floor living room, and a TV in the master bedroom. Which one do you want to turn on?" The user can specify the specific TV to turn on and control it accurately. However, when the user needs to control a large number of devices of the same type, for example, when the user wants to turn on a light, but the smart home system may include dozens or even hundreds of lights, introducing the information of all the lights to the user will take a long time, and the user will find the entire process cumbersome, seriously affecting the user's user experience.

[0075] To address the aforementioned issues, the smart home system provided in the embodiments of the present disclosure has intelligent control functionality. Each smart device can be equipped with intelligent control functionality. When a user issues a voice command, the smart home system can automatically identify the specific smart device to be controlled, rather than repeatedly asking the user, thereby improving the user experience. Smart devices are configured with an intelligent control mode, enabling all smart devices in this mode to collectively implement intelligent control functionality. Users can send intelligent control commands to smart devices to activate these modes.

[0076] In some embodiments, the user can send smart control instructions to the smart device 200 by operating a designated button on the remote control. The user can also send smart control instructions to the smart device 200 by voice input to control the smart device 200 to perform the smart control mode process.

[0077] In some embodiments, the user can also send smart control instructions to the smart device 200 through a preset gesture. The smart device 200 can detect the user's behavior through an image collector, such as a camera. When the user makes a preset gesture or action, it can be considered that the user has sent a smart control instruction to the smart device 200.

[0078] In some embodiments, when a user uses a smart device to control the smart device 200, such as a mobile phone, a smart control instruction can also be sent to the smart device 200. In actual application, a control can be set in the mobile phone, and the user can use the control to select whether to enter the smart control mode, thereby sending the smart control instruction to the controller 250.

[0079] In some embodiments, a smart control mode option may be set in the UI interface of the smart device 200 , and when the user clicks on the option, the smart device 200 may be controlled to enter the smart control mode.

[0080] In some embodiments, to prevent users from accidentally triggering the smart control mode, when the controller 250 receives a smart control command, it can control the display 260 to display a smart control mode confirmation message, thereby allowing the user to reconfirm whether to control the smart device 200 to perform the smart control mode process. Figure 11 is a schematic diagram of the display 260 displaying the smart control mode confirmation message according to some embodiments.

[0081] In some embodiments, when the smart device 200 enters the smart control mode, the server can obtain the smart device to be controlled according to the user's voice command, so that the smart device responds to the user's command.

[0082] FIG12 is a flowchart of the interaction between the various parts of the smart home system according to some embodiments. It includes a user 121, a server 122, a master device 123, and a smart device to be controlled 124, wherein:

[0083] Step 1201: The server 122 obtains the user's regional device information;

[0084] The regional device information includes device information of a plurality of first regions and smart devices in the first regions;

[0085] Step 1202: The server 122 obtains the master device of the first area based on the device information;

[0086] Each of the first areas corresponds to a master control device;

[0087] Step 1203: User 121 sends a voice command to the main control device 123;

[0088] Step 1204: the master device 123 sends an instruction response request to the server 122;

[0089] After detecting the user's voice command, the smart device serving as the master device sends a command response request to the server, wherein the command response request includes the voice command, and the voice command is used to control the smart device;

[0090] Step 1205: The server 122 obtains the target master device and the target first area;

[0091] Step 1206: The server 122 determines the smart device to be controlled based on the target first area and the target master device;

[0092] Step 1207: The server 122 sends a response instruction to the smart device 124 to be controlled;

[0093] Step 1208: The smart device 124 to be controlled executes the response instruction.

[0094] In some embodiments, the server may first count the status of all smart devices in the user's home, and may obtain device information of the smart devices in the smart home system.

[0095] Each room in a user's home can be defined as a zone. In the present embodiment, the zone corresponding to each room is referred to as a first zone. Therefore, a user's home may include multiple first zones, such as a master bedroom, a first-floor living room, a second-floor living room, and a second bedroom, each of which is a first zone. Based on this, the smart devices in all first zones can be counted to determine the smart devices contained in each first zone.

[0096] It should be noted that when a smart device is installed, the room where the smart device is located, that is, the first zone to which the smart device belongs, can be reported to the server. Therefore, the server can determine the first zone to which each smart device belongs.

[0097] In order to facilitate the management of smart devices, the smart home system in the user's home can be divided according to the situation of the user's house area. The smart devices contained in each first area can be distinguished, and a smart device can be selected to manage other smart devices in the area.

[0098] In some embodiments, the server may include a communication module and a control module, wherein the communication module may establish a communication connection with the smart device to exchange information.

[0099] The control module may first obtain the user's regional device information. The regional device information refers to the first area of ​​the user's house, including the first areas into which the user's house is divided, and the device information of the smart devices contained in each first area.

[0100] Based on the regional device information, the number of rooms in the user's home and the smart devices installed in each room can be determined. Whenever a new smart device is installed in the smart home system, the device information and the regional situation of the smart device can be reported to the server, allowing the server to accurately understand the user's regional device information.

[0101] In some embodiments, to facilitate management of smart devices, one smart device may be selected in each first zone, referred to as a master device in the disclosed embodiments. The master device may manage other smart devices in the first zone, for example, by sending various control instructions to control the other smart devices.

[0102] In order to avoid the situation where all smart devices respond to the user's voice commands at the same time and thus affect the user, it can also be set that the main control device in each first area responds to the user's voice commands, and other devices do not respond. The main control device then controls other smart devices to execute user commands, thereby improving the user experience.

[0103] Therefore, if there are a large number of smart devices in the user's home, by selecting a master device in each room and only having the master device respond to the user, it is possible to avoid reducing the accuracy of the nearest and unique response when all voice devices in multiple areas participate in the nearest wake-up decision.

[0104] In some embodiments, the server may select a master device for each first area. The server may select master devices for all first areas based on the user's regional device information, and specifically select a corresponding master device based on the device information of the smart device in each first area.

[0105] Considering that a smart device may not be able to control other smart devices when it is in a powered-off state, the server can filter all the smart devices in a powered-on state to obtain the master device.

[0106] The user can send a power-on command to the smart device, such as a voice command or pressing a power button on the smart device, to start the smart device.

[0107] The smart device may include a display, a communicator, and a controller. The communicator can establish a communication connection with a server and other smart devices to enable information exchange. In response to a power-on command input by a user, the controller can activate the smart device to put it into a power-on state. After the smart device is powered on, the controller can send a startup message to the server indicating that the smart device is in a power-on state.

[0108] The server can obtain the smart devices that have been activated in each first area, that is, the smart devices that have sent activation information. For each activated smart device, the server can determine it as a first candidate device. The server can select a device from the first candidate devices as the master device.

[0109] In some embodiments, after obtaining the first candidate devices, the server may first filter out device information of each first candidate device from the regional device information, and send the device information to the first candidate devices.

[0110] Device information may include the name of the smart device, its capabilities, and the region to which it belongs. Considering that a master device may be required to control other smart devices, the stronger the capabilities of a smart device, the more suitable it is to serve as the master device to control other devices. To this end, the capabilities of each first candidate device in the first region can be evaluated to select the device with the highest capabilities as the master device.

[0111] The server may send capability information of each first candidate device to each first candidate device. The smart device serving as the first candidate device may receive the device information sent by the server, which may specifically be capability information. Based on the capability information, the smart device may obtain its own capability score to represent the strength of its own capability.

[0112] In some embodiments, the capability information includes capability data of several capabilities of preset types, including proximity wake-up capability, other smart device control capability, collaborative control capability, relay capability, device service capability, and display capability.

[0113] Proximity wake-up capability allows a smart device to determine the distance between itself and the user making the voice call, compare it with other smart devices, and select the closest smart device to respond. Whether a smart device has proximity wake-up capability can be reported to the server during installation. If the smart device has proximity wake-up capability, the capability data can be 1; if not, the capability data can be 0.

[0114] The "other smart device control capability" refers to the ability of a smart device to control other smart devices. In the embodiments of the present disclosure, this can be further expressed as the ability of a smart device to control other smart devices in the local area. If the smart device has the other smart device control capability, the capability data can be 1; if the smart device does not have the other smart device control capability, the capability data can be 0.

[0115] Collaborative control capability refers to the ability of a smart device to coordinate with other smart devices to respond to user commands when it's unable to do so. For example, if a smart device, such as a TV, requests to adjust the room temperature but the TV doesn't have the capability to do so, it can notify the air conditioner to adjust the room temperature. If a smart device has collaborative control capability, the capability data can be 1; if it doesn't, the capability data can be 0.

[0116] If the smart device has the relay capability, the capability data may be 1; if the smart device does not have the relay capability, the capability data may be 0.

[0117] Device service capabilities refer to the services that a smart device can perform. Services can include playing media, playing music, adjusting temperature, adjusting brightness, and other common service functions that a smart device can perform. A total number of services can be pre-defined, and the percentage of services that a smart device can perform relative to the total number of services can be measured. This percentage is then used to determine the device's service capability.

[0118] Display capability refers to the screen size that the display of a smart device can display, which can be the size of the display.

[0119] The smart device can obtain a capability score based on the capability data corresponding to these capabilities. A weight coefficient can be set for each capability in advance, and the capability score can be obtained based on the weight coefficient and capability data. It can be calculated according to the following formula (1):

[0120] Among them, λ represents the capability score, α represents the nearby wake-up capability, β represents the ability to control other smart devices, ω represents the collaborative control capability, θ represents the relay capability, μ represents the device business capability, and δ represents the display capability.

[0121] In some embodiments, after obtaining the capability score, the smart device may send the capability score to the server. The server may count the capability scores of all first candidate devices in each first area and select the first candidate device with the largest capability score as the master device.

[0122] In some embodiments, the server may also calculate the capability score of each first candidate device to determine the master device. For the calculation method, please refer to the above content and will not be repeated here.

[0123] The server may send a master device notification message to the master device corresponding to each first zone, where the master device notification message is used to notify the master device that it is the master device of the first zone in which it is located. At the same time, the server may send a silent instruction to other smart devices in the first zone other than the master device, where the silent instruction indicates not to respond to the user's voice command.

[0124] If the smart device receives the master device notification information sent by the server, it can determine that it is the master device of the first area to which it currently belongs, and it can respond to the user's voice command.

[0125] If the smart device receives the silent instruction sent by the server, it can be confirmed that it is not the master device and does not respond to the user's voice instruction, so it can ignore the user's voice instruction.

[0126] In some embodiments, after receiving the master device notification information sent by the server, the smart device may ask the user whether it needs to determine itself as the master device.

[0127] A master device automatic setup switch can be configured on the server, with each room (first zone) having a master device automatic setup switch. The master device automatic setup switch controls whether the master device in the current first zone can be automatically setup without user confirmation. Users can use their mobile terminals to configure whether the master device automatic setup switch is enabled in each room.

[0128] If the master device in the first area automatically sets the switch to the on state, a smart device can directly confirm that it is the master device after receiving the master device notification information sent by the server.

[0129] If the master device in the first area automatically sets the switch to the off state, the smart device can ask the user whether to set itself as the master device after receiving the master device notification information sent by the server. The controller of the smart device can control the display to display the master device confirmation interface. Figure 13 is a schematic diagram of the master device confirmation interface according to some embodiments. As shown in Figure 13, the master device confirmation interface includes master device confirmation information, a confirmation control, and a cancel control. The master device confirmation information is "whether to set this device as the master device", the confirmation control is "yes", and the cancel control is "no".

[0130] In response to the user selecting the confirmation control, the controller may determine that the smart device is the master device in the first area. In response to the user selecting the cancellation control, the controller may determine that the smart device is not the master device.

[0131] If the user does not make a selection and a certain time has passed, for example, a threshold may be preset, and when the time exceeds the threshold, the controller may directly determine the smart device as the master device.

[0132] You can also set it so that when the time exceeds this threshold, the controller detects whether the original master device is offline, for example, due to a power outage or a server notification. If the original master device is offline, the controller directly determines this smart device as the master device. If the original master device is still online, the results obtained from this master device are discarded and the original master device status remains unchanged (i.e., the previous master device remains unchanged).

[0133] If the user cancels the smart device as the master device, the server can continue to search for the device with the highest scoring capability among the remaining smart devices and notify the user, or it can stop searching for the master device in the first area. This embodiment of the disclosure is not limited to this.

[0134] In some embodiments, after the server has selected the master device, when a newly started smart device is detected, which may be a new smart device sending startup information to the server, the server can re-acquire the master device.

[0135] When a newly started smart device is detected in the first area, the server may determine the smart device as a second candidate device and detect whether the second candidate device can serve as a master device.

[0136] The server can filter out the device information of the second candidate device and send the capability information of the second candidate device to the second candidate device. The smart device as the second candidate device can calculate its own capability score based on the capability information and send the capability score to the server.

[0137] The server may receive the capability score sent by the second candidate device and perform comparison.

[0138] If the capability score of the second candidate device is greater than the capability score of the current master device, the server may determine the second candidate device as the new master device. At this time, the server may send a master device notification message to the second candidate device and send a silent instruction to the original master device.

[0139] If the second candidate device's capability score is less than or equal to the master device's capability score, no processing is performed and the master device remains unchanged. The server can also send a silent command to the second candidate device to prevent it from responding to the user's voice command.

[0140] In some embodiments, when the server determines that the second candidate device is the new master device, it can send a master device notification message to the second candidate device. After receiving the message, the smart device serving as the second candidate device can display the master device switching interface on the display. Figure 14 is a schematic diagram of the master device switching interface according to some embodiments. As shown in Figure 14, the master device switching interface includes master device switching information, confirmation controls, and cancel controls. Among them, the master device switching information can include the original master device information and information asking whether to switch, which can be "whether to switch the original master device (refrigerator) to this device", the confirmation control is "yes", and the cancel control is "no".

[0141] In response to the user selecting the confirmation control, it means that the user switches the master device to the current device, and the controller can determine that the smart device is the master device in the first area.

[0142] In response to the user selecting the cancel control, indicating that the user does not want to switch the master device, the controller can determine that the smart device is not the master device. At the same time, the second candidate device can send a cancel switch instruction to the server to notify the server that the user has canceled the master device switch and keeps the original master device unchanged.

[0143] In some embodiments, after the second candidate device displays the master device switching interface, the user may not see the interface, resulting in no selection.

[0144] If the user has not made a selection for a certain period of time (e.g., a preset threshold), the system can detect whether the original master device is offline after the time has passed. If so, the second candidate device can determine itself as the new master device. If the original master device is still online, the acquisition results for the current master device can be discarded, and the original master device's status can be maintained, i.e., the previous master device remains unchanged.

[0145] The second candidate device may also display the master device switching interface again to prompt the user whether to switch the master device. If the user still does not make a selection after displaying the master device switching interface a preset number of times, such as 5 times, the second candidate device may directly determine the smart device as the master device.

[0146] In some embodiments, after acquiring the master devices in each first area, if the user issues a voice command, each master device can respond to the user's command. Simultaneous responses from multiple master devices may cause confusion to the user. Therefore, the server can re-screen all master devices to obtain a single master device to respond to the user's command.

[0147] All master devices can make proximity wake-up decisions. By selecting the master device closest to the user to determine the room the user is in, the smart home system can be woken up nearby.

[0148] Users can use voice to issue various commands to control smart devices, such as turning on lights and playing music. After detecting the user's voice command, all master control devices can collect the user's voice command audio data, generate a command response request based on the voice command audio data, and then send the command response request to the server. Based on the command response requests sent by all master control devices, the server can filter all master control devices and determine the master control device that will respond to the user's command.

[0149] In some embodiments, the master control device may generate different instruction response requests based on its own computing power.

[0150] The master device can be pre-set with a system utilization threshold, such as 75%. When the current system utilization of the master device is higher than the preset system utilization threshold, the computing power is considered insufficient. When the current system utilization of the master device is lower than or equal to the preset system utilization threshold, the computing power is considered sufficient.

[0151] When the master device's computing power is insufficient, the master device can generate a command response request based on the collected voice command audio data, such that the command response request includes the voice command audio data. The master device can also analyze the voice command audio data to obtain the voice command text. The command response request can include both the voice command audio data and the voice command text.

[0152] When the master device has sufficient computing power, it can analyze the collected voice command audio data and calculate the voice energy value. The voice energy value represents the strength of the audio data signal and can indicate the user's distance from the master device. A higher voice energy value indicates closer proximity to the master device, while a lower voice energy value indicates further distance. The master device can generate a command response request based on the voice energy value. The command response request can include the voice energy value and the text of the voice command.

[0153] The master device may send a command response request to the server.

[0154] In some embodiments, before calculating the voice energy value, the master control device may first perform acoustic echo cancellation (AEC) processing on the voice command audio data to eliminate the influence of the sound emitted by the master control device itself on the voice energy value.

[0155] In some embodiments, after receiving a command response request from a master device, the server may analyze all command response requests. For a command response request that includes a speech energy value, the server may obtain and store the corresponding relationship between the speech energy value and the master device. In the disclosed embodiments, the speech energy value calculated by the master device itself is referred to as the first speech energy value.

[0156] For a command response request including voice command audio data, the server can obtain a voice energy value based on the voice command audio data. In the present embodiment, the voice energy value obtained by the server is referred to as a second voice energy value. The server can also obtain a correspondence between the second voice energy value and the master control device.

[0157] After acquiring the speech energy values ​​of all master control devices, the server may select the speech energy value with the largest value among all the first speech energy values ​​and all the second speech energy values, and determine the master control device corresponding to the speech energy value as the target master control device.

[0158] The server can determine the first area corresponding to the target master device, referred to as the target first area in the disclosed embodiment. The area where the user is currently located can be determined to be the target first area. Therefore, the smart device that the user needs to control is also in the target first area, and the server can control the corresponding smart device in that area.

[0159] In some embodiments, the server can obtain the device type based on the voice command. The type of the device to be controlled can be determined based on the voice command text sent by the master device, for example, the device type of "turn on the light" is the light.

[0160] Considering that there may be multiple smart devices of the same device type in the target first area, for example, there may be more than a dozen lights, the server may further narrow down the area where the smart devices to be controlled are located.

[0161] For the first area, it can be pre-divided into multiple small areas, which are called second areas in the embodiment of the present disclosure. For example, for the first area, the living room on the first floor, it can be divided into 9 second areas, which can be evenly divided in the form of a nine-square grid.

[0162] The server may obtain second area information of the target first area, wherein the second area information includes a plurality of second areas included in the target first area and smart devices in each second area.

[0163] The server can obtain the second area where the target master device is located based on the second area information, which is referred to as the target second area in the embodiment of the present disclosure. The server can obtain the second area that contains smart devices of the device type and is closest to the target second area.

[0164] To calculate the distance between two second areas, the center points of the two second areas can be obtained and the distance between the two center points can be calculated. The distances between all second areas can be pre-calculated and uploaded to a server for pre-storage. If there are multiple second areas that are equidistant from the target second area, the closest second area can be determined based on a preset priority. For example, a priority of front > left > right > back can be set, and the closest second area can be selected based on this priority.

[0165] The server may obtain a smart device corresponding to the device type in the nearest second area, and determine the smart device as the smart device to be controlled.

[0166] Figure 15 is a schematic diagram of the second area of ​​the target first area according to some embodiments. As shown in Figure 15, the target main control device is a TV, and the target first area can be a living room on the first floor, which can include 9 second areas. Among them, the second area in the upper left corner includes a refrigerator and light 4, the second area in the middle includes light 1, and the three second areas on the right include light 2, TV, and light 3 respectively. The second area where the TV is located is the target second area. Assume that the user's voice command is "turn on the light", so the device type of the smart device to be controlled is a light. The server can first determine the second area with lights, and there are four second areas in total. The server calculates the distance from the four second areas to the target second area, where the areas where lights 1, lights 2, and lights 3 are located are the same distance from the target second area. Therefore, the preset priority of front > left > right > back can be used for filtering, so the second area in front of the TV (the area of ​​light 1) is the closest second area. Light 1 is the smart device to be controlled.

[0167] In some embodiments, the server may generate a response instruction according to the user's voice instruction, and send the response instruction to the smart device to be controlled.

[0168] After receiving the response instruction, the smart device serving as the smart device to be controlled can execute the response instruction to meet the user's needs.

[0169] The present disclosure also provides a method for controlling a smart device, which is applied to a smart home system. The method includes:

[0170] The server obtains regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions;

[0171] The server obtains a master control device in the first area based on the device information, where each first area corresponds to a master control device;

[0172] After detecting the user's voice command, the smart device serving as the master device sends a command response request to the server. The command response request includes the voice command, which is used to control the smart device.

[0173] The server obtains a target master device and a target first area corresponding to the target master device;

[0174] The server determines the smart device to be controlled based on the target first area and the target main control device, and sends a response instruction to the smart device to be controlled;

[0175] The smart device to be controlled executes the response instruction.

[0176] In some embodiments, the device information includes capability information of the smart device. The server obtains the master device of the first area based on the device information, including:

[0177] Traverse the smart devices that have been started in the first area to determine the first candidate device; send the capability information of the first candidate device to the first candidate device, so that the first candidate device calculates the capability score according to the capability information; receive the capability score fed back by the first candidate device, and mark the first candidate device with the largest capability score as the master device.

[0178] In some embodiments, after the server selects a master device based on the capability score, the server further includes:

[0179] Based on the detection of a newly started smart device in the first area, it is determined as a second candidate device; the capability information of the second candidate device is sent to the second candidate device; the capability score fed back by the second candidate device is received; if the capability score of the second candidate device is greater than the capability score of the master device, the second candidate device is marked as a new master device; if the capability score of the second candidate device is less than or equal to the capability score of the master device, the first candidate device with the largest capability score is maintained as the master device.

[0180] In some embodiments, the command response request includes voice command audio data collected by the master device or a first voice energy value obtained by the master device based on the voice command. The server obtains the target master device, further comprising:

[0181] If there is a command response request including voice command audio data, a second voice energy value is obtained based on the voice command audio data; and the master control device corresponding to the voice energy value with the largest value between the first voice energy value and the second voice energy value is determined as the target master control device.

[0182] In some embodiments, after the server obtains the master device of the first area based on the device information, the server further includes:

[0183] Send master device notification information to the master device, where the master device notification information is used to notify the master device that it is the master device in the first area where the master device is located; send a silent instruction to other smart devices other than the master device in the first area, where the silent instruction is used to instruct other smart devices not to respond to the user's voice command.

[0184] In some embodiments, the server obtains the smart device to be controlled based on the target first area and the target master device, further comprising:

[0185] Obtain the device type based on voice commands; obtain the second area information of the target first area, the second area information includes several second areas contained in the target first area and the smart devices in the second area; obtain the target second area where the target main control device is located based on the second area information; obtain the second area that contains the smart devices of the device type and is closest to the target second area; determine the smart device of the device type in the nearest second area as the smart device to be controlled.

[0186] In some embodiments, the device information includes capability information of the smart device, and the capability information includes capability data of several capabilities of preset types; the several capabilities include nearby wake-up capability, other smart device control capability, collaborative control capability, relay capability, device business capability and display capability.

[0187] The smart device obtains the capability score based on the device information, and also includes: calculating the capability score based on the preset weight coefficient and capability data.

[0188] In some embodiments, before the controller determines that the smart device is the master device of the first zone where the smart device is located, the controller includes:

[0189] The control display displays a master device confirmation interface, which includes master device confirmation information, a confirmation control, and a cancel control; in response to the user selecting the confirmation control, the smart device is determined to be the master device in the first area; in response to the user selecting the cancel control, the smart device is determined not to be the master device in the first area.

Claims

1. A server, comprising: A communication module is configured to establish a communication connection with a smart device; The control module is configured as follows: Acquire regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions; Acquire a master control device of the first area based on the device information, where each first area corresponds to one master control device; In response to a command response request sent by the master device after detecting a user's voice command, obtaining a target master device and a target first area corresponding to the target master device; the command response request includes the voice command, and the voice command is used to control the smart device; A smart device to be controlled is determined based on the target first area and the target main control device, and a response instruction is sent to the smart device to be controlled.

2. The server according to claim 1, wherein the device information includes capability information of the smart device; and the control module is configured to obtain the master device of the first area based on the device information, and is specifically configured to: determining a first candidate device according to the activated smart devices in the first area; sending the capability information of the first candidate device to the first candidate device, so that the first candidate device calculates a capability score according to the capability information; Receive the capability score fed back by the first candidate device, and mark the first candidate device with the largest capability score as the master device.

3. The server according to claim 2, wherein after the control module selects a master control device according to the capability score, the control module is further configured to: Determining a newly started smart device as a second candidate device based on detecting that the smart device exists in the first area; sending the capability information of the second candidate device to the second candidate device; receiving a capability score fed back by the second candidate device; If the capability score of the second candidate device is greater than the capability score of the master device, mark the second candidate device as a new master device; If the capability score of the second candidate device is less than or equal to the capability score of the master device, the first candidate device with the largest capability score is maintained as the master device.

4. The server according to claim 1, wherein the command response request includes voice command audio data collected by the master device or a first voice energy value obtained by the master device based on the voice command; and the control module executes acquiring the target master device, specifically configured to: For a command response request including command audio data, determining a second voice energy value corresponding to the master device sending the command response request according to the voice command audio data in the command response request; For the command response request that does not include the command audio data, obtaining, from the command response request, a first voice energy value corresponding to the master control device that sends the command response request; The master control device corresponding to the largest speech energy value between the first speech energy value and the second speech energy value is determined as the target master control device.

5. The server according to claim 1, after the control module acquires the master device of the first area based on the device information, is further configured to: Sending master control device notification information to the master control device, where the master control device notification information is used to notify the master control device that the master control device is the master control device of the first area where the master control device is located; A silence instruction is sent to other smart devices in the first area except the master device, where the silence instruction is used to instruct other smart devices not to respond to the user's voice instruction.

6. The server according to claim 1, wherein the control module is further configured to: obtaining a device type based on the voice command; Acquire second area information of the target first area, where the second area information includes a plurality of second areas included in the target first area and smart devices in the second areas; Acquire a target second region where the target master control device is located based on the second region information; Acquire a second area that contains smart devices of the device type and is closest to the target second area; A smart device of the device type in the nearest second area is determined as a smart device to be controlled.

7. A smart device comprising: monitor; a communicator configured to establish a communication connection with a server and other smart devices; The controller is configured as: In response to a user's power-on instruction, the smart device is started and a startup message is sent to the server; receiving device information sent by the server, calculating a capability score based on the device information and sending the score to the server, so that the server acquires the master device according to the capability score; In response to the master device notification information sent by the server, determining that the smart device is the master device of the first area where the smart device is located; In response to a voice command input by a user for controlling a smart device, a command response request is generated and sent to a server, so that the server obtains a target master device and a target first area corresponding to the target master device, and obtains a smart device to be controlled and sends a response command to the smart device to be controlled, wherein the command response request includes the voice command.

8. The smart device according to claim 7, wherein the device information includes capability information of the smart device, the capability information including capability data of a plurality of capabilities of a preset type; the plurality of capabilities including proximity wake-up capability, other smart device control capability, collaborative control capability, relay capability, device service capability, and display capability; The controller is further configured to: A capability score is calculated based on a preset weight coefficient and the capability data.

9. The smart device according to claim 7, before the controller determines that the smart device is the master device of the first area where the smart device is located, is further configured to: Controlling the display to display a master device confirmation interface, wherein the master device confirmation interface includes master device confirmation information, a confirmation control, and a cancel control; In response to a user selecting the confirmation control, determining that the smart device is a master device in the first area; In response to the user selecting the cancel control, it is determined that the smart device is not a master device in the first area.

10. A method for controlling a smart device, applied to a smart home system, wherein the smart home system includes a server and a plurality of smart devices, wherein the smart devices establish a communication connection with the server; the method for controlling the smart device comprises: The server obtains regional device information, where the regional device information includes device information of a plurality of first regions and smart devices in the first regions; The server obtains a master control device of the first area based on the device information, where each first area corresponds to a master control device; After detecting the user's voice command, the smart device serving as the master device sends a command response request to the server, wherein the command response request includes the voice command, and the voice command is used to control the smart device; The server obtains a target master device and a target first area corresponding to the target master device; The server determines a smart device to be controlled based on the target first area and the target main control device, and sends a response instruction to the smart device to be controlled; The smart device to be controlled executes the response instruction.

Images