Device control method, and device

Abstract

A device control method and apparatus, a device, a storage medium, and a computer program product. The device control method comprises: acquiring operational data obtained by a first device via an awakened target function (310); and, on the basis of the operational data, controlling the first device to provide a device control service (330). By virtue of the operational data obtained by the first device via different awakened target functions, a user is no longer limited to controlling only a single intelligent device (130), but can also control any intelligent device other than the first device, thereby greatly improving device control flexibility. Moreover, these intelligent devices (130) can be controlled to execute a broader range of actions, without requiring the user to repeatedly perform multi-step operations. This allows for simplicity and convenience, thereby effectively solving the problem of poor device control convenience and flexibility in the related art, and thus meeting to a great extent the needs of users in different application scenarios.

Description

Equipment control methods and equipment Technical Field

[0001] This disclosure relates to the field of Internet of Things (IoT) technology, and more specifically, to a device control method and device. Background Technology

[0002] With the development of IoT technology, the control of smart devices is gradually being widely applied in various scenarios, including but not limited to smart home scenarios, office scenarios, entertainment scenarios, traffic safety scenarios, and so on.

[0003] Currently, the control of smart devices mainly relies on control devices (such as remote controls), voice, actions, and automated control schemes configured in the client. However, the convenience and flexibility of device control based on these control methods cannot yet meet user needs. Summary of the Invention

[0004] This disclosure provides a device control method, apparatus, device, storage medium, and computer program product, the technical solutions of which are as follows:

[0005] According to one aspect of this disclosure, a device control method includes: acquiring working data of a first device under a target function that has been activated; and controlling the first device to provide device control services based on the working data.

[0006] According to one aspect of this disclosure, a device control apparatus includes: a data acquisition module for acquiring working data of a first device under a target function that has been activated; and a device control module for controlling the first device to provide device control services based on the working data.

[0007] According to one aspect of this disclosure, an apparatus includes a processing unit, an action recognition unit, and a voice acquisition unit; the action recognition unit and the voice acquisition unit are electrically connected to the processing unit respectively; the action recognition unit is configured to acquire corresponding working data when the target function of the apparatus being activated is an action recognition function; the voice acquisition unit is configured to acquire corresponding working data when the target function of the apparatus being activated is a voice acquisition function; the processing unit is configured to acquire the working data obtained by the apparatus under the activated target function; and the processing unit is further configured to control the apparatus to provide apparatus control services based on the working data.

[0008] According to one aspect of this disclosure, an apparatus includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, implements the apparatus control method as described above.

[0009] According to one aspect of this disclosure, a storage medium having a computer program stored thereon implements the device control method as described above when executed by one or more processors.

[0010] According to one aspect of this disclosure, a computer program product includes a computer program that, when executed by one or more processors, implements the device control method as described above.

[0011] The beneficial effects of the technical solution provided in this disclosure are:

[0012] In the above technical solution, based on the working data obtained by the first device under the target function of wake-up, the first device can provide corresponding device control services to other smart devices based on the working data. In other words, by using the different target functions woken up by the first device, the user can no longer only control a single smart device, but can control any smart device that is different from the first device, which greatly improves the flexibility of device control. Moreover, it can control these smart devices to perform more multi-dimensional actions without having to repeat multi-step operations. It is simple and convenient, thus effectively solving the problem of poor convenience and flexibility of device control in related technologies, and greatly meeting user needs. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0014] Figure 1 is a schematic diagram of the implementation environment according to this disclosure;

[0015] Figure 2 is a hardware structure diagram of a device according to an exemplary embodiment;

[0016] Figure 3 is a flowchart illustrating a device control method according to an exemplary embodiment;

[0017] Figure 4 is a schematic diagram illustrating a first device in a second state according to an exemplary embodiment;

[0018] Figure 5 is a flowchart illustrating another device control method according to an exemplary embodiment;

[0019] Figure 6 is a flowchart illustrating step 510 according to an exemplary embodiment;

[0020] Figure 7 is a flowchart illustrating another device control method according to an exemplary embodiment;

[0021] Figure 8 is a flowchart illustrating step 511 according to an exemplary embodiment;

[0022] Figure 9 is a flowchart illustrating step 513 according to an exemplary embodiment;

[0023] Figure 10 is a flowchart illustrating another device control method according to an exemplary embodiment;

[0024] Figure 11 is a hardware structure diagram of a first device according to an exemplary embodiment;

[0025] Figure 12 is a schematic diagram illustrating the switching of a first device in different modes according to an exemplary embodiment;

[0026] Figure 13 is a schematic diagram of the specific implementation of a device control method in an application scenario;

[0027] Figure 14 is a hardware structure diagram of a first device in an application scenario;

[0028] Figure 15 is a schematic diagram illustrating the specific implementation of the first device switching between different modes in an application scenario.

[0029] Figure 16 is a schematic diagram of the specific implementation of the first device controlling the smart device based on the gateway in an application scenario;

[0030] Figure 17 is a structural block diagram of a device control apparatus according to an exemplary embodiment;

[0031] Figure 18 is a structural block diagram of a device according to an exemplary embodiment. Detailed Implementation

[0032] The embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this disclosure, and should not be construed as limiting this disclosure.

[0033] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this disclosure means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0034] As mentioned earlier, existing smart devices mainly rely on voice, gestures, client applications, or single-function control devices (such as remote controls) for control, which is too simplistic and lacks flexible, multi-dimensional device control capabilities.

[0035] Specifically, on the one hand, it can only control a single smart device: for example, existing control devices such as knobs, joysticks, and switches are usually designed for a single smart device, or the user controls a single smart device through a single action, or the client is configured to automatically control the living room light, and can only control the living room light. If other lights, curtains or more smart devices need to be controlled, they need to be reconfigured in the client for different smart devices, and it is impossible to control different smart devices in real time according to the user's intention.

[0036] On the other hand, when controlling the same smart device to perform different actions, users need to repeatedly perform multiple steps: for example, adjusting the brightness and color temperature of the living room light often requires users to perform multiple steps, or to operate multiple different control devices separately, such as one knob or button controlling the brightness and another knob or button controlling the brightness. This will inevitably affect the user experience, not only because it is not smooth, but also because it is relatively costly.

[0037] Furthermore, even with control devices that offer voice capture functionality, while users can operate them through conversation, current devices require users to first press a specific button to enter voice capture mode before they can control the device via conversation. In dimly lit environments, users may not be able to locate this specific button, which still reduces the ease of use for the user.

[0038] Therefore, the device control method provided in this disclosure can effectively improve the convenience and flexibility of device control. Accordingly, the device control method is applicable to device control devices, which can be deployed on devices. The device can be a computer device configured with a von Neumann architecture, such as a desktop computer, laptop computer, server, etc.; the device can also be an electronic device with central control function, such as a gateway; the device can also refer to an electronic device with voice acquisition and motion recognition functions, such as a remote control equipped with a microphone, six-axis sensor, etc.

[0039] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0040] Figure 1 is a schematic diagram of the implementation environment involved in a device control method. The implementation environment includes at least a user terminal 110, a smart device 130, a server 170, and network devices. In Figure 1, the network devices include a gateway 150 and a router 190, which is not intended to be a specific limitation.

[0041] The user terminal 110, which can also be considered as a user terminal or terminal, can deploy (or install) the client associated with the smart device 130. This user terminal 110 can be an electronic device such as a smartphone, tablet, laptop, desktop computer, smart control panel, or other device with display and control functions, and is not limited here.

[0042] The client, associated with the smart device 130, is essentially where the user registers an account and configures the smart device 130. For example, the configuration includes adding a device identifier to the smart device 130, so that when the client runs on the user terminal 110, it can provide the user with functions such as device display and device control of the smart device 130. This client can be in the form of an application or a web page. Correspondingly, the interface for displaying the device on the client can be in the form of a program window or a web page, and there is no limitation here.

[0043] Smart device 130 is deployed in gateway 150 and communicates with gateway 150 through its own configured communication module, thereby being controlled by gateway 150. It should be understood that smart device 130 generally refers to one of a plurality of smart devices 130. This embodiment of the disclosure only uses smart device 130 as an example; that is, this embodiment of the disclosure does not limit the number or type of smart devices deployed in gateway 150. In one application scenario, smart device 130 is deployed in gateway 150 by accessing it through a local area network. The process of smart device 130 accessing gateway 150 through a local area network includes: gateway 150 first establishing a local area network, and smart device 130 joining the local area network established by gateway 150 by connecting to it. This local area network includes, but is not limited to, ZIGBEE or Bluetooth. Among them, the smart device 130 can be a smart printer, smart fax machine, smart camera, smart air conditioner, smart door lock, smart light, or a human body sensor, door and window sensor, temperature and humidity sensor, water immersion sensor, natural gas alarm, smoke alarm, wall switch, wall socket, wireless switch, wireless wall sticker switch, cube controller, curtain motor, millimeter wave radar, remote control, etc., equipped with a communication module.

[0044] The interaction between user terminal 110 and smart device 130 can be achieved through a local area network (LAN) or a wide area network (WAN). In one application scenario, user terminal 110 establishes a wired or wireless communication connection with gateway 150 via router 190, such as Wi-Fi, allowing user terminal 110 and gateway 150 to be deployed on the same LAN, thus enabling user terminal 110 to interact with smart device 130 via the LAN path. In another application scenario, user terminal 110 establishes a wired or wireless communication connection with gateway 150 via server 170, such as 2G, 3G, 4G, 5G, or Wi-Fi, allowing user terminal 110 and gateway 150 to be deployed on the same WAN, thus enabling user terminal 110 to interact with smart device 130 via the WAN path.

[0045] The server-side 170 can also be considered as the cloud, cloud platform, platform side, server side, etc. This server-side 170 can be a single server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers, in order to better provide backend services to a massive number of user terminals 110. For example, backend services include device control services.

[0046] In one application scenario, the device control method can be implemented through interaction between different smart devices 130 (such as a first device, a third device, a fourth device associated with the first device, etc.), a gateway 150 (such as a second device bound to the first device), and a server 170. Specifically, the first device obtains working data under the wake-up target function and sends the working data to the gateway 150, which forwards it to the server 170. The server 170 generates corresponding control commands based on the working data and returns the control commands to the gateway 150, which then forwards the control commands to the third or fourth device to execute the corresponding actions.

[0047] In another application scenario, the device control method can be implemented through interaction between different smart devices 130 (such as a first device, a third device, a fourth device associated with the first device, etc.) and a gateway 150. Specifically, the first device obtains working data under the wake-up target function and sends the working data to the gateway 150. The gateway 150 generates corresponding control commands based on the working data and sends the control commands to the third or fourth device to control the third or fourth device to perform corresponding actions.

[0048] In another application scenario, the device control method can also be implemented through the interaction between different smart devices 130 (such as a first device, a third device, a fourth device associated with the first device, etc.). Specifically, the first device obtains working data under the wake-up target function, generates corresponding control commands based on the working data, and sends the control commands to the third or fourth device to control the third or fourth device to perform corresponding actions.

[0049] Please refer to Figure 2, which is a hardware structure diagram of a device according to an exemplary embodiment. This device is applicable to the smart device 130 or gateway 150, or server 170 in the implementation environment shown in Figure 1.

[0050] It should be noted that this device is merely an example adapted to this disclosure and should not be construed as providing any limitation on the scope of use of this disclosure. Nor should this device be interpreted as requiring or needing to have one or more of the components of the exemplary device 200 shown in Figure 2.

[0051] The hardware structure of the device 200 can vary greatly depending on the configuration or performance. As shown in Figure 2, the device 200 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0052] Specifically, power supply 210 is used to provide operating voltage for the various hardware devices on device 200.

[0053] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, in the implementation environment shown in FIG1, a smart device 130 configured with interface 231 interacts with gateway 150.

[0054] Of course, in other examples adapted to this disclosure, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, as shown in Figure 2, but this is not intended to be a specific limitation.

[0055] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0056] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the device 200, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0057] Application 253 is a computer program formed by computer-readable instructions based on operating system 251 to perform at least one specific task. It may include at least one module (not shown in FIG2), and each module may contain corresponding computer-readable instructions. For example, a device control device can be regarded as application 253 deployed on device 200.

[0058] Data 255 can be photos, pictures, etc. stored on a disk, or it can be voice data, first motion data, second motion data, various automated control schemes, etc., stored in memory 250.

[0059] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on massive amounts of data 255 stored in the memory 250. For example, a device control method may be implemented by the central processing unit 270 reading an application program 253 stored in the memory 250.

[0060] Furthermore, this disclosure can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of this disclosure is not limited to any particular hardware circuit, software, or combination thereof.

[0061] Referring to Figure 3, this disclosure provides a device control method applicable to devices, such as gateway 150 or server 170 in the implementation environment shown in Figure 1, to facilitate data transmission between a first device and other smart devices. For example, other smart devices may be a third device deployed in the gateway, or a fourth device associated with the first device. The device may also be the first device itself, for example, a remote control. The hardware structure of the device can be as shown in Figure 2.

[0062] In the following method embodiments, for ease of description, the execution subject of each step of the method is the device, but this does not constitute a specific limitation.

[0063] As shown in Figure 3, the method may include the following steps:

[0064] Step 310: Obtain the working data of the first device under the target function after wake-up.

[0065] First, it should be noted that the target function refers to the function possessed by the first device and activated. In some embodiments, the target function includes, but is not limited to, voice acquisition function, motion recognition function, and communication function. Specifically, the voice acquisition function means that the first device can pick up the user's dialogue; the motion recognition function means that the first device can detect the user's actions on the first device; and the communication function means that the first device can transmit data with other devices, which can be a gateway, a server, or other smart devices deployed in a gateway. In some embodiments, the working data includes, but is not limited to, voice data, first motion data, and second motion data. It is understood that, to reduce the power consumption of the first device, the first device can first enter a sleep state after power-on. In the sleep state, the first device does not work or operates in a low-power mode until one or more of its functions are activated. Only then can it enter the awakened state and start working in a high-power mode, obtaining the corresponding working data under the activated target function. For example, if the first device is a remote control and the activated target function is the voice acquisition function, then the first device can acquire voice data as working data under the voice acquisition function.

[0066] Voice data is used to instruct the user on their device control intentions. This means the user's intentions must reflect not only which smart device they want to control, but also what they want that device to do. In other words, voice data instructs the smart device to be controlled and the action it needs to perform. It's important to note that the smart device to be controlled can be any smart device different from the first device. For example, in a smart home scenario, the first device could be a remote control, and the smart device to be controlled could be a smart light, smart curtains, smart air conditioner, smart TV, etc.

[0067] The first motion data is data used to wake up the function of the first device; that is, the first motion data is used to indicate the function of the first device to be woken up. In some embodiments, the first motion data includes first feature data and second feature data, wherein the first feature data is used to characterize the amplitude of the movement of the first device, and the second feature data is used to characterize the type of movement of the second device. In some embodiments, the type of movement characterized by the first motion data includes at least one of picking up and shaking.

[0068] The second motion data is used to indicate the user's device control intention; that is, the second motion data is used to indicate the smart device to be controlled and the action to be performed by the smart device. Similarly to voice data, the smart device to be controlled can be any smart device different from the first device. For example, in a smart home scenario, the first device can be a remote control, and the smart device to be controlled can be a smart light, smart curtains, smart air conditioner, smart TV, etc. In some embodiments, the second motion data includes motion data corresponding to the first state, which can also be understood as the overall motion state, referring to the state where the first device as a whole is in motion. For example, for a remote control, a user can rotate, flip, push, vibrate, shake the remote control, and tap other objects such as a table with the remote control. When the remote control is performing the above movements, it can be considered that the remote control is in the first state. In some embodiments, the second motion data includes motion data corresponding to the second state, which can also be understood as a relatively stationary state, referring to the state where the first device as a whole remains stationary while the components configured in the first device are in motion relative to the whole. In other words, the second state can refer to the state where the first device as a whole remains stationary relative to the moving components. For example, a user can tap or rotate a knob on a relatively stationary remote control. As shown in Figure 4, for a remote control equipped with a knob, if the user rotates the knob (401) without rotating the remote control itself, the remote control can be considered to be in a second state, meaning the remote control remains stationary relative to the rotated knob. In some embodiments, the second motion data includes motion parameters of the first device, including but not limited to motion direction, amplitude of movement, motion angle, motion duration, motion speed, acceleration, angular velocity, etc. Thus, by varying the motion parameters, different intelligent devices to be controlled and different actions to be performed by those intelligent devices can be indicated. The action type represented by the second motion data includes at least one of rotation, flipping, pushing, tapping, and vibration.

[0069] In some embodiments, the voice data is collected by a first device, which may be a smart device with voice collection capabilities, such as a remote control, a speaker companion, etc.

[0070] In some embodiments, the first motion data is detected by a first device, which can be a smart device with motion recognition capabilities, such as a remote control, a Rubik's Cube controller, etc. Regarding the first motion data including first feature data and second feature data, the first feature data can be detected by the first device in a sleep state, or it can be understood that the first feature data can be detected when the first device is operating in a low-power mode; the second feature data can be detected by the first device in a wake-up state, that is, the second feature data can be detected when the first device is operating in a high-power mode.

[0071] In some embodiments, the second motion data is also detected by the first device, which can be a smart device with motion recognition capabilities, such as a remote control, a Rubik's Cube controller, etc. The second motion data includes motion data corresponding to the first state and motion data corresponding to the second state. When the first device as a whole is in motion, the motion data corresponding to the first state can be detected by the first device operating in a high-power mode; when the first device as a whole remains stationary, the motion data corresponding to the second state can also be detected by the first device operating in a high-power mode.

[0072] Depending on the application scenario, the environment in which the first device is deployed can also vary. For example, in a smart home scenario, the first device can be deployed in any room the user enters or exits (such as the living room), or in a traffic safety scenario, the first device can be deployed inside the vehicle the user is driving.

[0073] Taking the remote control as an example, the remote control is equipped with a microphone and a six-axis sensor. The microphone can collect voice data, and the six-axis sensor can accurately capture motion, such as identifying the range of motion of the first device, and providing more refined motion recognition capabilities, such as identifying the type of motion of the first device, thereby obtaining first motion data and / or second motion data.

[0074] Secondly, regarding the acquisition of working data, it can come from local storage. For example, if the device is the first device, it can directly retrieve the working data after obtaining and storing it. Alternatively, it can come from data transmission. For example, if the device is a gateway, the first device can send the working data to the gateway after obtaining it, and the gateway can then receive the working data.

[0075] Step 330: Control the first device to provide device control services based on the working data.

[0076] After obtaining the work data, the first device can provide device control services corresponding to the work data to other smart devices. In some embodiments, the device control services include, but are not limited to: controlling a third device to perform actions, controlling a fourth device to perform actions, sending work data to a second device, etc. Here, the third device refers to a smart device deployed on the same gateway as the first device but distinct from the first device; the fourth device refers to a smart device associated with the first device, where the association between the fourth device and the first device is established in advance by the first device through a first or second relationship. The first relationship at least indicates the correspondence between different action types of the first device and at least one fourth device, and the second relationship at least indicates the correspondence between different action types of the first device and actions performed by at least one fourth device; the second device refers to a gateway bound to the first device, which can also be understood as the first device being deployed on the second device, thus establishing a pre-established binding relationship between the first device and the second device, thereby binding the first device and the second device together.

[0077] In some embodiments, the action includes, but is not limited to, at least one of device state adjustment actions and operating parameter adjustment actions. The device state adjustment action includes, but is not limited to, turning on or off, and the operating parameter adjustment action includes, but is not limited to, adjusting the operating parameters of the smart device. For example, the operating parameters of a smart light include brightness, temperature, color, amplitude, etc.

[0078] For example, since the working data includes, but is not limited to, voice data, first motion data, and second motion data, the first device can control the third or fourth device to perform actions based on the voice data. These actions can be device status adjustment actions and / or operating parameter adjustment actions, enabling the user to control the smart device solely based on voice. The first device can also control the third or fourth device to perform actions based on the second motion data. These actions can also be device status adjustment actions and / or operating parameter adjustment actions, enabling the user to control the smart device solely based on actions. Alternatively, the first device can jointly control the third or fourth device to perform actions based on both voice data and second motion data. These actions can be device status adjustment actions and / or operating parameter adjustment actions, thereby enabling the user to jointly control the smart device based on both voice and actions. For example, first, the third device can be controlled to perform a device status adjustment action based on voice data, and then the third device can be controlled to perform a status adjustment action based on the second motion data.

[0079] Through the above process, the user can obtain working data from the first device under different target functions, and is no longer limited to controlling a single smart device. Instead, the user can control any smart device that is different from the first device, greatly improving the flexibility of device control. Moreover, the user can control these smart devices to perform more multi-dimensional actions without having to repeat multiple steps. This is simple and convenient, and can effectively solve the problem of poor convenience and flexibility of device control in related technologies, greatly satisfying the needs of users in different application scenarios.

[0080] As mentioned earlier, in order to reduce the power consumption of the first device, the first device can first enter a sleep state after powering on. In the sleep state, the first device does not work or works in a low-power mode until one or more functions are woken up, at which point it will enter a wake-up state and then start working in a high-power mode.

[0081] Based on this, referring to Figure 5, in an exemplary embodiment, before step 310, the method may further include the following steps:

[0082] Step 410: Obtain the first motion data of the first device.

[0083] The first motion data is data used to wake up the function of the first device; that is, the first motion data is used to indicate the function of the first device to be woken up.

[0084] Understandably, when the first device performs an action, it acquires first motion data corresponding to that action. When the device is the first device itself, such as its main control chip, the main control chip acquires motion data transmitted by the detector within the first device, which is used to detect the device's own actions and generate motion data, and uses this as the first motion data. When the device is a gateway or server, the gateway or server receives the motion data transmitted by the detector based on data transmission and uses it as the first motion data. Here, a detector refers to a detection device capable of detecting and generating motion data, such as a six-axis sensor; the first device refers to a device capable of controlling other intelligent devices, i.e., a device that can control the actions of other intelligent devices, such as a remote control or a thermostat.

[0085] In some embodiments, the first motion data characterizes the amplitude and / or type of movement of the first device. Specifically, the first motion data includes first feature data and second feature data, whereby the first feature data characterizes the amplitude of movement of the first device, and the second feature data characterizes the type of movement of the second device. It should be noted that the first motion data can be generated by the entire first device, meaning the first device as a whole is in motion. In other words, when the first device as a whole moves, the corresponding motion data can be used as the first motion data. This differs from the case where only one or more buttons on the first device are pressed. If only one or more buttons on the first device are pressed, and the first device as a whole remains stationary while only one or more buttons are moving relative to the whole, then it can be considered that the first motion data is generated by one or more buttons rather than the entire first device. That is, the corresponding motion data generated when one or more buttons are pressed can also be used as the first motion data.

[0086] Step 430: Activate the target function of the first device based on the first motion data.

[0087] The target function includes at least one of the first function and the second function. In some embodiments, the first function may be a voice acquisition function, an action recognition function, a communication function, etc. In some embodiments, the second function is different from the first function and includes, but is not limited to, a voice acquisition function, an action recognition function, a communication function, etc.

[0088] In this embodiment, wake-up is achieved by a wake-up operation triggered on the first device. Specifically, wake-up may include the following steps: detecting whether a wake-up operation triggered on the first device exists; if the wake-up operation is detected, obtaining first motion data corresponding to the wake-up operation, and then controlling the first device to enter the wake-up state from the sleep state based on the first motion data, so that at least one function of the first device in the wake-up state is activated. Here, the wake-up operation refers to a user operation that triggers the first device to start working. This wake-up operation can be flexibly set according to the user's actual needs. For example, in some embodiments, the wake-up operation may refer to "shaking" the first device. When the user shakes the first device in the sleep state, the first device can detect the wake-up operation and thus enter the wake-up state from the sleep state. In other embodiments, the wake-up operation may also refer to picking up the first device; no specific limitation is made here. It should be noted that in this case, the first device is still allowed to recognize the set action type (such as shaking or picking up) in low power mode while in sleep mode. Only after the first device is woken up by the set action type and enters the wake-up state from sleep mode is the first device allowed to start working with high power. For example, when the action recognition function or voice acquisition function of the first device is woken up, the first device can detect other set action types (such as rotation, flipping, vibration, tapping, etc.) or acquire voice data.

[0089] Of course, in other embodiments, wake-up can also be achieved by setting voice data. That is, the first device enters the wake-up state from the sleep state, which is not limited to detecting a wake-up operation triggered for the first device, but can also be due to the collection of set voice data for the first device. Moreover, the set voice data can be flexibly set according to the user's actual needs. For example, the set voice data can be a preset wake-up word, such as "Xiao Qiao" or "Xiao Ai", etc. This is not a specific limitation. It should be noted that in this case, the first device is still allowed to collect set voice data (such as "Xiao Qiao") in a low-power mode while in the sleep state. Only after the first device is woken up by the set voice data and enters the wake-up state from the sleep state is the first device allowed to start working with high power. For example, the voice acquisition function or motion recognition function of the first device is awakened, and the first device can obtain voice data, first motion data, and second motion data.

[0090] After acquiring the first motion data, the device will activate the corresponding function of the first device based on the first motion data. In some embodiments, the target function of activating the first device may include the following: First, enabling the first device to generate corresponding working data after it performs an action, i.e., the activated target function is the motion recognition function; Second, enabling the first device to collect corresponding working data, i.e., the activated target function is the voice acquisition function; Third, enabling the first device to transmit working data with other devices, i.e., the activated target function is the communication function.

[0091] Based on this, in some embodiments, the target function to be activated by the first device can vary depending on the content represented by the first motion data. For example, when the amplitude of the motion represented by the first motion data is greater than a set amplitude, the motion recognition function of the first device can be activated; when the motion type represented by the first motion data belongs to a set motion type and the amplitude of the represented motion is greater than the set amplitude, the voice acquisition function of the first device can be activated. Alternatively, the device can also prioritize activating the communication function of the first device when the motion type represented by the first motion data is less than the set amplitude but the motion type represented by the first motion data belongs to the set motion type. Or, when the amplitude of the motion represented by the first motion data is greater than the set amplitude, the voice acquisition function is activated first, and then, based on the activation of the voice acquisition function, the motion recognition function is activated in conjunction with the motion type represented by the first motion data. It can be understood that which target function(s) of the first device need to be activated in different application scenarios can be determined according to the amplitude and / or motion type represented by the first motion data, and this embodiment does not specifically limit this.

[0092] In some embodiments, the order in which the target functions of the first device are activated can differ depending on the content represented by the first motion data. For example, when a user picks up the first device from its dormant state, the first device can detect the picking action based on the action type represented by the first motion data, and then enter the awake state from its dormant state. In this case, for the first device in the awake state, the action recognition function can be activated first based on the picking action. Then, if the user shakes the first device, the first device can detect the shaking action based on the action type represented by the first motion data, and then activate the voice acquisition function based on the shaking action. In other words, when the first device is in a dormant state, the action recognition function of the first device is activated first based on the first motion data, and then the voice acquisition function of the first device is activated.

[0093] Of course, in other embodiments, the wake-up of the target function is not limited to the first motion data, but can also be voice data. For example, when a user picks up the first device in a dormant state, the first device can detect the picking action based on the action type represented by the first motion data, and then enter the wake-up state from the dormant state. At this time, for the first device in the wake-up state, the voice acquisition function can be woken up first based on the picking action. Then, based on the woken voice acquisition function, if the user says "start the action recognition function" or other preset wake-up words, the action recognition function of the first device can be further woken up based on the acquired voice data. That is, when the first device is in a dormant state, the voice acquisition function of the first device is woken up first based on the first motion data, and then the action recognition function of the first device is woken up based on the voice data.

[0094] Under the above embodiments, on the one hand, the device can wake up different target functions of the first device in different application scenarios, thereby laying the foundation for obtaining the working data of the first device under different target functions, which is conducive to realizing the convenience and flexibility of device control, so as to meet the needs of users in different application scenarios; on the other hand, the first device will only start working with high power consumption after being woken up from the sleep state, otherwise it will always maintain low power consumption or not work. This not only helps to reduce device power consumption and effectively extend the service life of the first device, but also helps to reduce the cost of device control.

[0095] In one exemplary embodiment, step 430 may include the following steps;

[0096] Step 431: When the first device is in a sleep state, wake up the first function of the first device based on the first motion data.

[0097] In this embodiment, the first device is initially in a sleep state. For example, if the brightness of the first device's display is off or below a set brightness, it is considered that the first device is in a sleep state. Then, the first device enters a wake-up state from the sleep state, which means that one or more functions of the first device are activated. It should be noted that the first device in the sleep state does not work or works in a low-power mode, while the first device in the wake-up state works in a high-power mode.

[0098] In this embodiment, the first function is a motion recognition function, and the first motion data includes first feature data used to characterize the motion amplitude of the first device. Based on this, if the motion amplitude of the first device characterized by the first feature data is greater than a set amplitude, the motion recognition function of the first device is activated.

[0099] Step 433: When the first function of the first device is activated, the second function of the first device is activated based on the first motion data.

[0100] In this embodiment, the second function is a voice acquisition function, and the first motion data includes second feature data used to characterize the action type of the first device. Based on this, if the action type of the first device characterized by the second feature data matches a set action type, the voice acquisition function of the first device is activated.

[0101] Of course, in other embodiments, the wake-up order of the first function and the second function is not limited to a sequential relationship, but can also be mutually exclusive. This embodiment does not constitute a specific limitation in this regard. For example, if the movement amplitude of the first device represented by the first feature data is greater than a set amplitude, and the movement type of the first device represented by the second feature data does not conform to the set movement type, then the first function of the first device is woken up; if the movement amplitude of the first device represented by the first feature data is greater than the set amplitude, and the movement type of the first device represented by the second feature data conforms to the set movement type, then the second function of the first device is woken up.

[0102] Optionally, in some embodiments, after the first device is woken up, it can also return to a sleep state. Specifically, the return can be achieved in the following ways: if the first device is not connected to a second device bound to it, the first device is controlled to enter a sleep state; if the first device stops collecting voice data or its operation meets the conditions for stopping voice data collection, the first device is controlled to enter a sleep state; if the movement amplitude of the first device is less than or equal to a set amplitude, the first device is controlled to enter a sleep state; if the movement type of the first device does not conform to a set movement type, the first device is controlled to enter a sleep state.

[0103] In this approach, if the first device cannot connect to the second device or the first device detects that the user has no intention to control the device—for example, if the user stops speaking, causing the first device to stop collecting voice data (e.g., if the first device does not collect voice data within 10 seconds, it will stop collecting voice data), or the user causes the first device to stop moving, or the user presses the button to stop collecting voice data (which is considered as the first device's operating condition meeting the condition to stop collecting voice data)—it can be assumed that the user has no intention to control the device. The first device can then re-enter the sleep state from the wake-up state and operate or not operate in a low-power mode. This not only helps reduce power consumption and effectively extends the lifespan of the first device, but also helps reduce the cost of device control.

[0104] Based on the above, the first motion data represents the amplitude and / or type of movement of the first device. Therefore, users can activate the corresponding functions of the first device by controlling its movements, improving the ease of activation. Similarly, the working data is obtained when the first device is activated with its corresponding target function. For example, voice data is obtained when the voice acquisition function is activated. This allows users to generate the first motion data and working data separately by controlling the first device twice, thereby controlling the first device and improving its ease of operation.

[0105] Furthermore, the device control services provided by the first device can vary depending on the type of action. Specifically, under a certain action type, such as rotating, tapping, pushing, or shaking, the fourth device associated with the first device can be controlled to perform actions. This allows the user to directly control the fourth device simply by making the first device move, without the need for the user to press buttons or use voice input, thus improving the ease of control of the fourth device. Additionally, under a certain action type, such as picking up or shaking, the first device can be activated to perform the corresponding target function (such as voice acquisition function), allowing the user to activate the target function of the first device without pressing buttons, thus improving the ease of activating the target function of the first device.

[0106] In one exemplary embodiment, step 310 may include the following steps:

[0107] Step 311: If it is determined that the target function of the first device being woken up is the action recognition function, then detect whether the first device is in the first state.

[0108] The first state can also be understood as the overall motion state, which refers to the state in which the first device as a whole is in motion. For example, for a remote control, a user can rotate, flip, push, vibrate, shake the remote control, and use the remote control to tap other objects such as a table, etc. When the remote control is performing the above movements, it can be considered to be in the first state.

[0109] Step 313: If the first device is detected to be in the first state, then the second motion data of the first device corresponding to the first state is obtained.

[0110] The second motion data includes motion data corresponding to the first state. This motion data refers to the data generated by the first device during its overall movement, including but not limited to motion direction, motion angle, motion duration, speed, and acceleration. Taking a remote control as an example, its configured six-axis sensor can accurately capture motion and provide more refined motion recognition capabilities, thereby obtaining motion data corresponding to the first state.

[0111] Step 315: The second motion data of the first device corresponding to the first state is used as the working data of the first device under the motion recognition function.

[0112] In this remote control mode, users can control other smart devices to perform corresponding actions through the voice acquisition function provided by the first device, and detect the second motion data corresponding to the first state through the motion recognition function provided by the first device. The types of actions represented by the second motion data include, but are not limited to, rotation, flipping, pushing, knocking (such as knocking on other objects), vibration of the first device, etc., to control other smart devices to perform corresponding actions, ultimately realizing dual control of voice and action, and achieving precise control of smart home devices.

[0113] Optionally, to reduce the power consumption of the first device, the method may further include the following steps: if it is detected that the first device is not in the first state, then control the first device to enter a sleep state until the first device re-enters the wake-up state. In this way, the first device always maintains low power consumption when the user does not intend to control other smart devices, which not only helps to reduce power consumption and effectively extend the service life of the first device, but also helps to reduce the cost of device control.

[0114] In one exemplary embodiment, step 310 may include the following steps:

[0115] Step 312: If it is determined that the target function of the first device being woken up is the action recognition function, then detect whether the first device is in the second state.

[0116] The second state can refer to the state in which the first device as a whole remains stationary relative to the moving components. For example, a user can tap (such as tapping the remote control itself) or rotate a knob on a relatively stationary remote control.

[0117] Step 314: If the first device is detected to be in the second state, then detect whether the components configured in the first device have been triggered to perform relative motion operations.

[0118] Among them, relative motion operation refers to the operation that triggers the components configured in the first device to move.

[0119] Referring back to Figure 4, for a remote control equipped with a knob, if the user rotates the knob (401) without rotating the remote control, the remote control can be considered to be in the second state, that is, the remote control remains stationary relative to the knob being rotated. Accordingly, the rotation operation is considered as a relative motion operation triggered by the knob of the remote control.

[0120] Step 316: If it is detected that a relative motion operation is triggered on a component configured in the first device, then the second motion data of the first device corresponding to the second state is obtained.

[0121] The second motion data includes motion data corresponding to the second state. This motion data refers to data showing that the first device as a whole remains stationary relative to the moving components, including but not limited to motion direction, motion angle, motion duration, speed, and acceleration. Taking a remote control as an example, its configured six-axis sensor can accurately capture motion and provide more refined motion recognition capabilities, thereby obtaining motion data corresponding to the second state.

[0122] Step 318: The second motion data of the first device corresponding to the second state is used as the working data of the first device under the motion recognition function.

[0123] In some embodiments, the device can directly detect whether the first device is in the second state, and if the first device is in the second state, continue to detect whether the components configured on the first device are moving. In other embodiments, the device can also continue to detect whether the first device is in the second state when it detects that the first device is not in the first state, and determine whether the components configured on the first device are moving if the first device is in the second state.

[0124] Therefore, the device can determine whether there is second motion data corresponding to the second state of the first device. If there is second motion data corresponding to the second state of the first device, it can use this as working data to control other smart devices to perform corresponding actions.

[0125] In this way, users can control other smart devices to perform corresponding actions through the voice acquisition function provided by the first device, and detect the second motion data corresponding to the second state through the motion recognition function provided by the first device. The types of actions represented by the second motion data include, but are not limited to, rotating and pressing the components configured by the first device (such as knobs and buttons), and control other smart devices to perform corresponding actions, thus realizing dual control of voice and action and achieving precise control of smart home devices.

[0126] Through the combination of the above embodiments, a more natural, intuitive, and multi-dimensional control method is provided, achieving a smooth and multi-dimensional device control experience through only one control device (i.e., the first device).

[0127] In one exemplary embodiment, step 330 may include the following steps:

[0128] Step 510: Determine the third device and action based on the work data, and instruct the third device to perform the action.

[0129] The third device refers to a smart device that is deployed on the same gateway as the first device but is different from the first device.

[0130] In some embodiments, the actions include, but are not limited to, device status adjustment actions and operating parameter adjustment actions. The device status adjustment actions include, but are not limited to, turning the device on and off, while the operating parameter adjustment actions include, but are not limited to, adjusting the operating parameters of the smart device. For example, the operating parameters of a smart light include brightness, temperature, color, amplitude, etc.

[0131] In some embodiments, a first control command is generated based on the third device and actions determined by the working data, instructing the third device to perform a device state adjustment action in response to the first control command. These actions include, but are not limited to, device state adjustment actions and operating parameter adjustment actions. That is, the third device can perform device state adjustment actions or operating parameter adjustment actions based on the working data.

[0132] In some embodiments, a second control command is generated based on the third device and the operating parameter adjustment actions determined by the working data, instructing the third device to respond to the second control command and perform the operating parameter adjustment actions. In other words, the third device can adjust the operating parameters according to the adjustment ratio of the operating parameters based on the working data.

[0133] In some embodiments, a first control command is generated based on the third device and actions determined by the working data, and a second control command is generated based on the third device and operating parameter adjustment actions determined by the working data. This instructs the third device to perform actions in response to the first control command and to perform operating parameter adjustment actions in response to the second control command. That is, the third device can first perform a device state adjustment action based on the working data and then adjust the operating parameters according to the adjustment ratio, or it can adjust the same or different operating parameters by the same or different adjustment ratios based on the working data. Of course, in other embodiments, it is not limited to the third device performing at least two operating parameter adjustment actions; the third device can also first perform a device adjustment action based on the working data and then perform another device adjustment action based on the working data.

[0134] In other words, based on the working data, the first device can independently control the third device to perform equipment status adjustment actions, independently control the third device to perform operating parameter adjustment actions, control the third device to perform equipment status adjustment actions first and then perform operating parameter adjustment actions, or control the third device to perform operating parameter adjustment actions at least twice, or control the third device to perform equipment status adjustment actions at least twice.

[0135] Step 530: Control the fourth device associated with the first device based on the working data to instruct the fourth device to perform an action.

[0136] The fourth device refers to a smart device associated with the first device. The association between the fourth device and the first device is that the first device has pre-established a first relationship or a second relationship for the fourth device. The first relationship is used to indicate the correspondence between different action types of the first device and the fourth device, and the second relationship is used to indicate the correspondence between different action types of the first device and the actions performed by the fourth device.

[0137] In other words, based on the first relationship, the first device can control the fourth device to perform corresponding actions based on the working data; based on the second relationship, the first device can directly control the fourth device to perform corresponding actions. These actions can be device status adjustments or operating parameter adjustments.

[0138] Step 550: Send work data to the second device bound to the first device to instruct the second device to control the third or fourth device to perform actions based on the work data.

[0139] The second device refers to the gateway that is bound to the first device. It can also be understood that the first device is deployed on the second device, so that a binding relationship is pre-established between the first device and the second device.

[0140] Compared to the first device directly controlling other smart devices, such as the first device controlling a third device to perform device state adjustment actions, here, based on the second device bound to the first device, the first device indirectly controls other smart devices, thus making the form of device control more flexible.

[0141] As described in the above embodiments, the equipment can be controlled in various ways, which can easily improve the flexibility and convenience of equipment control.

[0142] Referring to Figure 6, in an exemplary embodiment, step 510 may include the following steps:

[0143] Step 511: If the working data includes voice data, then based on the third device and action determined by the voice data, a first control command is generated to instruct the third device to respond to the first control command and perform an action.

[0144] The first control command is used to instruct the third device to perform a corresponding action. This action can be a device status adjustment action or an operating parameter adjustment action.

[0145] As mentioned earlier, the voice data indicates the smart device to be controlled and the action to be performed by that smart device. In some embodiments, the smart device to be controlled is a third device, and correspondingly, the action to be performed by the smart device to be controlled is a device status adjustment action or an operating parameter adjustment action. In other words, the smart device indicated by the voice data can be explicit. That is, the voice data clearly indicates the user's device control intention. For example, if the voice data is "Turn on living room light A", then the third device is living room light A, and the device status adjustment action is "turn on".

[0146] Furthermore, the inventors also realized that if the voice data explicitly indicates the user's device control intention, this device control method often can only control a single smart device. For example, in the aforementioned example, the user can only control living room light A. If light B is also deployed in the living room, then light B cannot be controlled simultaneously. Therefore, to achieve dynamic selection of a third device, in some embodiments, the smart device indicated by the voice data can be ambiguous. That is, the user's device control intention indicated by the voice data can be ambiguous. For example, if the voice data is "turn on the living room light," and if at least light A and light B are deployed in the living room, then the voice data is considered not to have explicitly indicated which light in the living room the user wishes to control.

[0147] Therefore, if the smart device indicated by the voice data is ambiguous, a third device can be dynamically selected to further determine the third device based on the ambiguous smart device indicated by the voice data. Using the previous example, suppose the ambiguous smart device indicated by the voice data is a "living room light," and there are at least two lights, A and B, in the living room. Then the third device could be light A and light B, and correspondingly, the device status adjustment actions for light A and light B would both be "on."

[0148] After determining the third device and the action, the device can generate a first control command to instruct the third device to perform the action, and send the first control command to the third device. The third device can then receive the first control command and respond to it by performing the corresponding action.

[0149] In this case, the third device can respond to the first control command to perform a device status adjustment action, or the third device can respond to the first control command to perform an operating parameter adjustment action.

[0150] Step 513: If the working data also includes second motion data, then based on the voice data and / or the second motion data, determine the third device and the operating parameter adjustment action, and generate a second control command based on the third device and the operating parameter adjustment action to instruct the third device to respond to the second control command and perform the operating parameter adjustment action.

[0151] The second control command instructs the third device to adjust the operating parameters according to the adjustment ratio corresponding to the operating parameters. These operating parameters refer to the parameter to be adjusted among at least one adjustable parameter of the third device. For example, for a smart light, the adjustable parameters include, but are not limited to, brightness, temperature, color, and adjustment range.

[0152] In this embodiment, the second control command is generated based on the second motion data detected by the first device and / or the voice data collected by the first device. Therefore, the first device not only has voice acquisition capabilities (i.e., it can collect voice data) but also motion recognition capabilities (i.e., it can detect the second motion data). Taking a remote control as an example, this remote control is equipped with a microphone and a six-axis sensor. The microphone can collect voice data, while the six-axis sensor can accurately capture motion and provide more refined motion recognition capabilities, thereby obtaining the second motion data. The second motion data refers to the data generated by the first device when it is in overall motion or relatively stationary, including but not limited to motion direction, motion amplitude, motion angle, motion duration, speed, acceleration, and angular velocity.

[0153] After obtaining voice data and / or second motion data, the device will further determine operating parameters and their corresponding adjustment ratios. In some embodiments, operating parameters can be extracted from voice data containing operating parameters. In some embodiments, the adjustment ratios corresponding to the operating parameters can also be extracted from voice data. In some embodiments, operating parameters can be determined based on the priority of several parameters adjustable by the third device when the voice data does not contain operating parameters. In some embodiments, operating parameters and their corresponding adjustment ratios can be determined based on whether the action type of the first device in the first state, as determined by the second motion data, meets the triggering conditions in the automation scheme. In some embodiments, the adjustment ratios corresponding to the operating parameters can be determined based on at least one of the motion direction, motion amplitude, motion angle, motion duration, motion speed, acceleration, and angular velocity contained in the second motion data.

[0154] Therefore, after determining the operating parameters and their corresponding adjustment ratios, the device can generate a second control command to instruct the third device to adjust the operating parameters based on the third device, the operating parameters and their corresponding adjustment ratios, and send the second control command to the third device. The third device can then receive the second control command and respond to it by performing the corresponding device state adjustment action.

[0155] It is worth mentioning that the device control method is not limited to the user controlling the device by voice first and then by motion, i.e., executing step 511 first and then step 513. The two steps can also be executed simultaneously. That is, it can be considered that the user can perform voice control and motion control at the same time. Specifically, the first device collects voice data and also detects the second motion data. Accordingly, the first control command and the second control command can be generated based on the voice data and the second motion data, respectively. Then, the first control command and the second control command are sent to the third device to control the third device to perform device state adjustment actions and adjust the operating parameters. This embodiment does not constitute a specific limitation in this regard.

[0156] In this scenario, the user first uses voice control and then uses motion control. The user can use the voice acquisition function provided by the first device to control the third device to perform device status adjustments or operating parameter adjustments. Furthermore, the user can use the motion recognition function provided by the first device to control the third device to adjust the corresponding operating parameters according to the adjustment ratio. Ultimately, this achieves dual control via voice and motion, enabling precise operation of smart home devices.

[0157] Step 515: If the working data does not include the second motion data, then based on the third device and operating parameter adjustment actions determined by the voice data, a second control command is generated to instruct the third device to respond to the second control command and perform the operating parameter adjustment actions.

[0158] In this scenario, the user only performs voice control. After the first device controls the third device to perform device status adjustment or operating parameter adjustment actions based on voice data, and no second motion data generated by the first device during movement is detected, the user can continue to control the third device to perform operating parameter adjustment actions based on voice data, ultimately achieving precise voice control of smart home devices.

[0159] In some embodiments, when the user performs voice control only, the voice data may include operating parameters and their corresponding adjustment ratios, thereby generating a second control command.

[0160] In some embodiments, when the user performs only voice control, the voice data may contain only operating parameters, and then a second control command may be generated based on the adjustment ratio corresponding to the operating parameters pre-configured in the automation scheme.

[0161] In some embodiments, when the user performs only voice control, the voice data may only contain the adjustment ratio, and then a second control command is generated based on the pre-configured operating parameters corresponding to the adjustment ratio in the automation scheme.

[0162] Through the above process, with the help of the voice and motion capture functions of the first device, users are no longer limited to controlling a single smart device, but can control any smart device that is different from the first device. This greatly improves the flexibility of device control and allows for more multi-dimensional operations on these smart devices without having to repeat multiple steps. This is simple and convenient, and can effectively solve the problem of poor convenience and flexibility of device control in related technologies, thus greatly satisfying user needs.

[0163] Referring to Figure 7, in an exemplary embodiment, prior to step 513, the method may further include the following steps:

[0164] Step 610: Determine the time interval between the acquisition time of the voice data and the acquisition time of the second motion data.

[0165] As mentioned earlier, for the third device, the user's device control intention can take many forms, such as voice control followed by action control, voice control only, or action control only. In order to determine which form the user's device control intention is, in this embodiment, the time interval = the detection time of the second motion data - the acquisition time of the voice data.

[0166] For the device, during the data transmission with the first device, each received data can be timestamped. For example, the timestamp of the received voice data can be t1, and the timestamp of the received motion data can be t2. In this case, the time interval = t2 - t1.

[0167] If the time interval is less than or equal to the set threshold, it means that the user wants to control the same smart device by combining voice and action control. That is, the user's device control intention is to first control the same smart device by voice and then by action. In this case, if step 630 is executed, the third device can be controlled based on the second motion data after the third device is controlled based on the voice data.

[0168] If the time interval is greater than the set threshold, it means that after the user controls the third device based on voice data, there is no intention to continue controlling the third device. That is, the user's intention to control the device is to control the third device only by voice or action, and to control other smart devices only by voice or action. Here, step 650 is executed so that after controlling the third device based on voice data, other smart devices can be controlled based on voice data or second motion data.

[0169] The time interval represents a duration that can be flexibly adjusted according to actual needs. It represents the maximum time interval between voice data and second motion data, preventing the device from misunderstanding the user's true device control intentions by waiting indefinitely for second motion data after acquiring voice data. For example, the time interval can be 100ms, 200ms, or 250ms; no limitation is set here.

[0170] Step 630: If the time interval is less than or equal to the set threshold, then the control object of the first device is determined to be the third device.

[0171] After determining that the third device is the controlled object, it indicates that the user's device control intention is to combine voice control and motion control. At this time, the device can execute step 513, that is, determine the third device and the adjustment action of the operating parameters based on the voice data and / or the second motion data, and generate a second control command based on the third device and the adjustment action of the operating parameters, so that the third device can adjust the corresponding operating parameters according to the adjustment ratio in response to the second control command.

[0172] In this case, the user wants to control the third device by combining voice and gesture control. That is, after saying "turn on the living room light", the user can continue to rotate the first device to adjust the operating parameters (such as brightness) of the "living room light" based on the adjustment ratio corresponding to the rotation.

[0173] Step 650: If the time interval is greater than the set threshold, then the control object of the first device is determined to be the fifth device.

[0174] The fifth device refers to the intelligent device controlled in the first automation scheme. This controlled intelligent device can be the third device or other intelligent devices that are different from the third device.

[0175] Once the fifth device is identified as the controlled object, it indicates that the user's current intention to control the device is only action control. At this time, the device can generate a third control command based on the first automation scheme and send the third control command to the fifth device so that the fifth device can respond to the third control command and perform the corresponding action.

[0176] The specific process may include the following steps:

[0177] The first step is to determine the action type of the first device based on the second motion data if the time interval is greater than the set threshold.

[0178] Among them, the action type can also be considered as the action type of the first device as a whole when it is in motion. This action type includes, but is not limited to, actions such as rotation, flipping, pushing, striking, and vibration.

[0179] The second step is to extract the action associated with the triggering condition from the first automation scheme if the action type meets the triggering condition in the first automation scheme.

[0180] The first automation scheme controls the fifth device to perform an action by monitoring whether the action type of the first device meets the triggering condition. For example, suppose the first automation scheme = {triggering condition = the action type of the first device is detected as flipping, controlled action = turning on the air conditioner}. That is, if the action type of the first device is detected as flipping, then the fifth device is the air conditioner (unlike the third device in the previous example which was a light), and the action is to turn it on.

[0181] The third step is to generate a third control command based on the fifth device and the action, instructing the fifth device to perform an action in response to the third control command.

[0182] The third control command is used to instruct the fifth device to perform a corresponding action. This fifth device can be distinguished from the third device. The action refers to the pending operation among at least one of the operations that the fifth device can perform. For example, for a smart light, the executable operations include, but are not limited to, turning on, turning off, adjusting brightness, temperature, color, etc.

[0183] Once the fifth device and the action are determined, the device can generate a third control command to instruct the fifth device to perform the action, and send the third control command to the fifth device. The fifth device can then receive the third control command and respond to it by performing the corresponding action.

[0184] In this scenario, the user wants to control a third device to perform actions via voice, while controlling a fifth device, distinct from the third device, to perform actions via gestures.

[0185] Of course, if the control object of the first device cannot be determined in the above situations, the device can also generate a prompt message and output the prompt message according to the set output method (such as voice broadcast, display screen display, alarm buzzer, etc.) to prompt the user to further clarify their device control intention.

[0186] With the combination of the above embodiments, users can arbitrarily combine voice control and action control, which can be used to control the same device or to control different devices, further improving the flexibility of device control and enhancing the user's device control experience.

[0187] In one exemplary embodiment, step 530 may include the following steps:

[0188] Step 531: If the working data includes the second motion data, then determine the action type of the first device based on the second motion data.

[0189] In this embodiment, the first device can control the fourth device associated with the first device based on the action type represented by the second motion data. Therefore, before controlling the fourth device to perform an action, the action type of the first device must first be determined based on the second motion data.

[0190] Step 533: Determine the fourth device associated with the first device based on the action type and the first relationship, or determine the action performed by the fourth device based on the action type and the second relationship.

[0191] In this embodiment, controlling the fourth device includes two scenarios: one where there is only one fourth device, and the other where there are multiple fourth devices. Since the fourth device is a smart device pre-associated with the first device, the number of fourth devices is fixed and known. When there are multiple fourth devices, the fourth device is determined based on the action type and the first relationship; when there is only one fourth device, the action to be executed is determined based on the action type and the second relationship.

[0192] The first relationship includes the correspondence between different action types and the corresponding fourth device, and the second relationship includes the correspondence between different action types and the corresponding actions performed by the fourth device.

[0193] Step 535: If a fourth device associated with the first device is determined, the second motion data is sent to the fourth device to instruct the fourth device to perform a corresponding action based on the second motion data.

[0194] In other words, the first device is pre-associated with multiple fourth devices. In the first relationship, different action types correspond to corresponding fourth devices, so that after the device determines the fourth device according to the first relationship, it sends the second motion data to the determined fourth device, and the fourth device will perform the corresponding action according to the second motion data.

[0195] For example, the fourth device includes a living room air conditioner and a bedroom air conditioner. The first relationship includes associating vibration with the living room air conditioner and pushing with the bedroom air conditioner. If the action type represented by the second motion data is vibration, the second motion data is transmitted to the living room air conditioner, causing the living room air conditioner to perform a corresponding action based on the second motion data. The specific action performed by the fourth device based on the second motion data can be pre-configured through a corresponding correspondence (i.e., the correspondence between the action type of the first device and the action performed by the fourth device), and this embodiment does not specifically limit this. For example, the living room air conditioner is pre-configured to start when the received second motion data represents a vibration action.

[0196] Step 537: If the action to be performed by the fourth device is determined, a fourth control command is generated based on the fourth device and the action to instruct the fourth device to respond to the fourth control command and perform the corresponding action.

[0197] In other words, the first device is associated with only one fourth device. In the second relationship, different action types correspond to different actions to be performed by the fourth device. This allows the device to determine the fourth device and corresponding actions based on the second relationship to generate a fourth control command, which is then sent to the fourth device, causing the fourth device to perform the corresponding action. In other words, the fourth device has multiple actions to perform, and the actions to be performed by the fourth device differ depending on the action type represented by the second motion data.

[0198] For example, a remote control is the first device, and the control chip within the remote control is the fourth device. The living room air conditioner can perform actions including turning on / off, heating up, cooling down, increasing airflow, and decreasing airflow. Shaking is associated with turning on / off, tapping with heating up, rotating with cooling down, vibrating with increasing airflow, and translating with decreasing airflow. The remote control is only pre-associated with the living room air conditioner. Therefore, when the remote control generates second motion data, the control chip inside the remote control will identify the type of action represented by the second motion data. If the action type is shaking, the control chip generates a fourth control command to control the living room air conditioner to turn on or off; if the action type is vibration, the control chip generates a fourth control command to control the living room air conditioner to increase airflow.

[0199] Based on the above, during device control, the second motion data representing different action types can correspond to different actions of at least one fourth device, enabling the differentiation of multiple situations and allowing users to control the fourth device more flexibly, thus improving the convenience of device control.

[0200] Referring to Figure 8, in an exemplary embodiment, step 511 may include the following steps:

[0201] Step 5111: Perform speech recognition processing on the speech data to obtain the candidate devices and actions indicated by the speech data.

[0202] In this embodiment, the candidate device refers to the smart device indicated by the voice data. Actions include, but are not limited to, device status adjustment actions and operating parameter adjustment actions.

[0203] As mentioned earlier, in order to achieve dynamic selection of the third device, the smart device pointed to by the voice data can be clear or blurry. Therefore, in order to determine whether the smart device pointed to by the voice data is clear, voice recognition processing technology is used.

[0204] In some embodiments, speech recognition processing includes, but is not limited to, speech conversion and semantic analysis. Specifically, speech recognition processing may include the following steps: First, converting speech data into text; second, performing semantic analysis on the text to obtain candidate devices and device state adjustment actions indicated by the speech data.

[0205] For example, if the voice data is "turn on the light", then through voice recognition processing, the candidate device pointed to by the voice data can be determined to be "light", and the action is to turn it on.

[0206] After identifying the candidate device indicated by the voice data, a third device can be further determined based on whether the candidate device is clear or blurry. This third device refers to a smart device with the same device attributes as the candidate device. Device attributes can also be understood as the functions possessed by the smart device. For example, the device attribute of an air conditioner refers to its temperature regulation function, while the device attribute of a temperature sensor refers to its temperature detection function.

[0207] If the candidate device pointed to by the voice data is ambiguous, then the third device can be obtained by associating it with the ambiguous candidate device pointed to by the voice data, i.e., step 5113 is executed.

[0208] Conversely, if the candidate device pointed to by the voice data is clear, then the third device can be uniquely determined by the clear candidate device pointed to by the voice data, i.e., step 5115 is executed.

[0209] Step 5113: If the clarity of the candidate device pointed to by the voice data is blurry, then according to the set association conditions, the third device is obtained by associating the candidate device pointed to by the voice data.

[0210] Clarity is used to indicate whether the candidate device to which the voice data is directed is clear.

[0211] If the clarity of the candidate device indicated by the voice data is ambiguous, it means that the voice data does not clearly indicate the user's intention to control the device. In this case, the ambiguous candidate device indicated by the voice data can be associated with it. For example, if the voice data is "turn on the living room light," then the candidate device indicated by the voice data is "living room light." If the lights deployed in the living room include at least light A and light B, then the clarity of the candidate device "living room light" is ambiguous. In this case, by associating the candidate device "living room light," it can be determined that the third device can be at least light A and light B.

[0212] In this embodiment, the association is achieved based on setting association conditions. These association conditions refer to the criteria used to associate with candidate devices. These conditions can be flexibly adjusted according to the actual needs of the application scenario and are not limited here. For example, the association conditions can be based on device type, device location / user location space, or user historical operations, etc.

[0213] The following example illustrates how to set association conditions based on the location of a device:

[0214] Specifically, the location of the device is determined based on the set association conditions.

[0215] Once the association conditions are determined to be based on the room where the device is located, the device will locate the first device to determine its position, i.e., the space where the device is located.

[0216] The candidate device pointed to by the voice data is associated with the space where the device is located to obtain the third device.

[0217] Therefore, based on the spatial association of devices, a third device refers to a candidate device located in the same space as the first device. For example, if the first device is in the living room, and the candidate device is a "lamp," then any lamp located in the living room can be considered a third device.

[0218] Similarly, in associations based on device type, the third device refers to a smart device with the same device type as the candidate device. For example, if the candidate device is "air conditioner," then the third device could be an air conditioner, but not a fan.

[0219] The association is based on the user's historical operations. The third device refers to the historical smart device that the user previously associated with the candidate device during the last control. For example, if the user's last voice command was "turn off the light", then the candidate device is "light". Assuming that the historical smart device associated with the candidate device "light" is living room light A, then the third device is living room light A.

[0220] Of course, in other application scenarios, the association conditions can also be set based on the space where the device is connected to the gateway, or based on the target space (such as the whole house in a smart home scenario), etc. These will not be described in detail here. The association conditions can be pre-configured by the user in the client. For example, the client can display the available association conditions to the user. Once the user selects any one of the association conditions, the configuration of that association condition is considered complete.

[0221] Step 5115: If the clarity of the candidate device pointed to by the voice data is clear, then the candidate device is selected as the third device.

[0222] If the clarity of the candidate device indicated by the voice data is clear, it means that the voice data has clearly indicated the user's intention to control the device. Therefore, the third device can be uniquely identified by the candidate device indicated by the voice data. For example, if the voice data is "Turn on living room light A", then the candidate device indicated by the voice data is "living room light A". It can be seen that the clarity of the candidate device "living room light A" is clear, therefore, the candidate device "living room light A" is the third device.

[0223] Step 5117: Based on the third device and the action, generate a first control command instructing the third device to perform the action.

[0224] After determining the third device and the action, the device can generate a first control command to instruct the third device to perform the action, and send the first control command to the third device. The third device can then receive the first control command and respond to it by performing the corresponding action.

[0225] The above embodiments enable the dynamic selection of a third device, allowing device control to move beyond a single device and obtain multiple devices through the association of candidate devices, thus greatly expanding the flexibility of device control.

[0226] Referring to Figure 9, in an exemplary embodiment, step 513 may include the following steps:

[0227] Step 5131: Determine the operating parameters to be adjusted for the third device based on the voice data and / or the second motion data.

[0228] Among them, the operating parameter refers to the parameter to be adjusted among at least one adjustable parameter of the third device.

[0229] In some embodiments, the operating parameters to be adjusted by the third device can be directly determined based on voice data. Specifically, through voice analysis processing of the voice data, if the voice data contains operating parameters, then the operating parameters are extracted from the voice data. For example, if the voice data is "adjust the brightness of the living room light," then the operating parameter is brightness.

[0230] In some embodiments, when the voice data does not contain operating parameters, the operating parameters to be adjusted for the third device can be determined based on the second motion data. The determination process can specifically include the following steps: First, determine the action type of the first device in its first state based on the second motion data. The action type can also be considered as the action of the first device as a whole during movement, including but not limited to rotation, flipping, pushing, tapping, and vibration. Second, if the action type of the first device is detected to meet the trigger condition in the configured second automation scheme, then the operating parameters associated with the met trigger condition are extracted from the second automation scheme. The second automation scheme controls the third device to adjust its operating parameters by monitoring whether the action type of the first device meets the trigger condition. For example, suppose the second automation scheme = {trigger condition = detected action type of the first device is rotation, controlled action = light color adjustment}, that is, if the action type of the first device is detected to be rotation, then the third device is the light, and the operating parameter is color. It should be noted that, unlike the first automation scheme which is used to control the fifth device to perform device state adjustment actions or operating parameter adjustment actions, the second automation scheme controls the third device to perform corresponding operating parameter adjustment actions by monitoring whether the action type of the first device meets the trigger condition.

[0231] In some embodiments, when the voice data does not contain operating parameters, the priority of candidate parameters adjustable by the third device can also be used to determine the operating parameters. The determination process can specifically include the following steps: First, determine at least one candidate parameter adjustable by the third device. Here, a candidate parameter refers to an adjustable parameter of the third device. Taking a lamp as an example, the candidate parameters of the lamp can be brightness, temperature, or color. Second, obtain the configured priority of each candidate parameter. The priority can be pre-configured by the user on the client side, or by the administrator on the control terminal (such as the server 170 in the implementation environment shown in Figure 1) based on the user's daily control frequency. For example, for a lamp, the priority of brightness can be greater than the priority of color, and the priority of color can be greater than the priority of temperature. Third, use the candidate parameter with the highest priority as the operating parameter. Using the aforementioned example, the operating parameter is the brightness, which has the highest priority.

[0232] Step 5133: Determine the adjustment ratio corresponding to the operating parameters based on the voice data and / or the second motion data.

[0233] In this embodiment, the adjustment ratio corresponding to the operating parameter is determined based on the second motion data. Of course, in other embodiments, the adjustment ratio corresponding to the operating parameter can also be determined based on the voice data. For example, if the voice data contains the adjustment ratio corresponding to the operating parameter, the adjustment ratio corresponding to the operating parameter can be directly extracted from the voice data. Alternatively, based on the operating parameter determined by the voice data and / or the second motion data in step 5131, it can be further determined according to the adjustment ratio corresponding to the operating parameter pre-configured in the automation scheme. This is not intended to constitute a specific limitation.

[0234] It should be understood that for some operating parameters that can change gradually, adjustment is actually a gradual process. Taking the third device as a curtain as an example, if the curtain's opening and closing range is 0-100%, then the curtain can be adjusted to open / close by 10% each time. As mentioned above, the second motion data includes, but is not limited to, motion direction, amplitude of motion, motion angle, motion duration, motion speed, acceleration, and angular velocity. Based on this, in this embodiment, the adjustment ratio corresponding to the operating parameter can be determined according to at least one of the motion direction, amplitude of motion, motion angle, motion duration, motion speed, and acceleration included in the second motion data. It should be noted that the amplitude of motion can refer to the amplitude when the first device shakes or pushes, while the motion angle can refer to the angle when the first device rotates or flips. Correspondingly, the motion direction when the first device shakes or pushes can refer to directions such as up and down, left and right, and forward and backward, while when the first device rotates or flips, it can refer to directions such as clockwise and counterclockwise.

[0235] In some embodiments, the adjustment ratio is related to the direction of motion and / or the amplitude of movement of the first device. Specifically, step 5133 may include the following steps: acquiring the motion parameters of the first device from the second motion data, the motion parameters including one or more of the following: direction of motion, amplitude of movement, angle of motion, duration of motion, speed of motion, acceleration, and angular velocity; and determining the adjustment ratio corresponding to the operating parameters based on the motion parameters.

[0236] For example, the adjustment ratio can be determined based on the direction and angle of motion. Specifically, rotating counterclockwise can decrease the speed, while rotating clockwise can increase it. The percentage decrease / increase is further determined by the angle of motion.

[0237] Alternatively, the adjustment ratio can be determined based on the angle of motion. Specifically, the angle of motion refers to the angle of rotation. In this case, the adjustment ratio is distributed evenly according to the rotation angle by default. For example, if the brightness of the lamp is 0-100%, it corresponds to a rotation angle of 0-360 degrees.

[0238] This can also be understood as follows: a mapping relationship is pre-set between the motion parameters and the adjustment ratios corresponding to the operating parameters of the first device. Based on this mapping relationship, the adjustment ratios with the mapping relationship can be obtained from the motion parameters of the first device.

[0239] Of course, in other embodiments, the adjustment ratio can also be determined based on the range of motion, duration of motion, speed of motion, acceleration, angular velocity, etc., which is not a specific limitation.

[0240] Step 5135: Generate a second control command instructing the third device to adjust its operating parameters according to the adjustment ratio.

[0241] Using the example above, suppose the third device is a lamp, the operating parameter is brightness, and the adjustment ratio is 30% based on a rotation angle of 30 degrees. Thus, the second control command can be used to adjust the brightness of the indicator light by 30%.

[0242] In the above process, adjustable parameters of the third device are controlled, which can control different parameters of the third device, as well as control different adjustment ratios of the same parameter, further improving the flexibility of device control and enhancing the user's device control experience.

[0243] Referring to Figure 10, in an exemplary embodiment, prior to step 330, the method may further include the following steps:

[0244] Step 710: Collect biological data and / or environmental data of the target object in the target authentication scenario.

[0245] Among them, the target authentication scenario is used to indicate the target object's control permissions over the first device.

[0246] Authentication methods may include voice recognition and / or fingerprint recognition. The target object refers to the object that needs to operate the control device. The target authentication scenario may be a voice authentication scenario, a fingerprint authentication scenario, or a dual authentication scenario using both voice and fingerprint.

[0247] In voice authentication scenarios, voice information used to verify the identity of a target object is pre-stored within the control device before its use, indicating the target object's voice control permissions over the device. For example, the target object uses voice to control a TV remote to turn the TV on and off, change channels, and perform other similar operations.

[0248] In fingerprint authentication scenarios, fingerprint information used to verify the identity of a target object is pre-stored within the control device before its use, representing the target object's fingerprint control permissions. For example, a target object can control an air conditioner remote control via fingerprint, allowing any user to adjust the air conditioner's temperature via voice.

[0249] In scenarios requiring dual authentication via voice and fingerprint, voice and fingerprint information are pre-stored within the control device before use to verify the identity of the target user. This information represents the target user's fingerprint and voice control permissions to the control device. For example, the target user can activate an air conditioner remote control via fingerprint and then use voice control to turn the air conditioner on / off and adjust the temperature.

[0250] In some embodiments, the first device may be a remote control. The remote control includes a selection control for selecting an authentication method, which may be a physical button or a virtual button. When the selection control is a physical button, the authentication method selection operation can be the operation of a target object pressing the physical button. For example, when the target object presses the physical button once, the target authentication scenario is determined to be a voice authentication scenario corresponding to voice recognition; when the target object presses the physical button twice consecutively, the target authentication scenario is determined to be a fingerprint authentication scenario; when the target object presses the physical button three times consecutively, the target authentication scenario is determined to be a dual authentication scenario of voice and fingerprint. When the selection control is a virtual button, the authentication method selection operation can be the operation of the target object touching the virtual button. For example, when the target object touches the virtual button once, the target authentication scenario is determined to be a voice authentication scenario corresponding to voice recognition; when the target object touches the virtual button twice consecutively, the target authentication scenario is determined to be a fingerprint authentication scenario; when the target object touches the virtual button three times consecutively, the target authentication scenario is determined to be a dual authentication scenario of voice and fingerprint.

[0251] In some embodiments, biometric data refers to data used to identify a target object. This biometric data includes, but is not limited to, physiological characteristics such as fingerprints, faces, irises, voiceprints, hand shapes, finger veins / palm veins, retina, DNA, and palm prints, as well as behavioral characteristics such as gait, signature, and voice. Optionally, biometric data includes at least one of voiceprint data and fingerprint data.

[0252] Step 730: Authenticate the target object based on biological data and / or environmental data.

[0253] In some embodiments, target object authentication is achieved based on biometric data. Specifically, biometric data is matched against pre-stored biometric data; if a match is successful, authentication is successful; if a match fails, authentication fails, and the target object is prohibited from controlling other smart devices through the first device. In some embodiments, target object authentication can be achieved by invoking a biometric model corresponding to the biometric data. This biometric model includes, but is not limited to, a speech recognition model and a fingerprint recognition model. The speech recognition model can be implemented using deep learning techniques, such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs); the fingerprint recognition model can be implemented using deep learning techniques, such as convolutional neural networks (CNNs) and generative adversarial networks (GANs).

[0254] In some embodiments, biometric data includes voice data. The voice data is matched with pre-stored voice data. If the match is successful, the authentication is successful, and the first device is activated (e.g., the first device is powered on). After that, any user can use the first device to control other smart devices (e.g., air conditioners, televisions, etc.).

[0255] In some embodiments, biometric data includes fingerprint data. The fingerprint data is matched with pre-stored fingerprint data. If the match is successful, authentication is successful, the first device is activated, and then any user can use the first device to control other smart devices.

[0256] In some embodiments, biometric data includes voice data and fingerprint data. The voice data and fingerprint data are matched against pre-stored voice data and fingerprint data, respectively. If a match is successful, authentication is successful, and the first device is activated. Subsequently, any user can use the first device to control other smart devices. This approach is suitable for scenarios with high security permission requirements, or where users and administrators are separated or need to mutually restrain each other. For example, if a user relies solely on fingerprint recognition, there is a risk of misuse in a passive or disabled state.

[0257] In some embodiments, voice data and fingerprint data can originate from the same target object or from different target objects. Specifically, the voice data and fingerprint data of the target object can be verified simultaneously. For example, when a target object wants to control a first device, the first device collects the target object's voice data and fingerprint data, and matches the voice data with pre-stored voice data in the first device. Simultaneously, the fingerprint data is matched with pre-stored fingerprint data in the first device. If both the voice data and fingerprint data match successfully, the target object's authentication is successful, allowing the target object to control the first device via voice; otherwise, the target object's authentication fails, prohibiting the target object from controlling the first device via voice. Based on this, by simultaneously verifying the target object's fingerprint data and voice data before voice control can be performed, security is greatly improved.

[0258] In some embodiments, the authentication of the target object is achieved based on biometric data and environmental data. Environmental data may refer to the ambient noise data of the space where the target object is located, and biometric data includes at least voice data. In some embodiments, voice data may contain the target object's voice commands (e.g., turning on a light), tone of voice (e.g., excitement), and other vocal characteristics, which can be represented in the form of digital signals, analog signals, etc. It is understood that the target object's voice characteristics may differ when using the first device (e.g., a remote control) in the morning and evening.

[0259] Specifically, the voice data is matched with the voice data pre-stored in the first device. At the same time, the environmental data is analyzed. If the environmental data is determined to match the expected environmental noise, the target object authentication is successful; otherwise, the target object authentication fails.

[0260] For example, suppose the target is in a study, and the voice data of the target in the study, "turn on the light," and the ambient noise in the study are collected. When the voice data matches the voice data pre-stored in the first device, and at the same time, the ambient noise matches the ambient noise of the study pre-stored in the first device, then the target's identity is verified.

[0261] In this approach, verifying the identity of the target object simultaneously through environmental noise and voice data can effectively improve the quality of the voice signal, remove background noise and other interference, ensure that only specific voiceprints in specific environments can pass verification, prevent unauthorized access, and make the device control services provided by the first device more accurate.

[0262] Step 750: If the target object passes the authentication, obtain the working data corresponding to the target object, and control the first device to provide device control services based on the working data corresponding to the target object.

[0263] As mentioned above, if the target object passes authentication, it can start the first device and then control the first device to enter the wake-up state from the sleep state, so that the device can obtain the working data of the first device under the target function under the wake-up state corresponding to the target object, so as to control the first device to provide device control services based on the working data.

[0264] For example, the first device is a remote control for controlling an air conditioner. The remote control has a temperature adjustment button for adjusting the air conditioner temperature. When the target object is authenticated, it means that the remote control can be used by any user. When the target object or any user other than the target object presses the temperature adjustment button on the remote control, the device can obtain the corresponding second motion data, generate a temperature adjustment command based on the second motion data, and then send the temperature adjustment command to the air conditioner so that the air conditioner responds to the temperature adjustment command and adjusts the current air conditioner temperature.

[0265] The above embodiments not only solve the problem that the traditional first device does not restrict the user's identity, but also avoid the problem that the security identification method is easy to be copied and cracked due to the single security identification method, thus avoiding the problem of low security of smart home system due to security risks.

[0266] In one exemplary embodiment, the target authentication scenario includes a single-user authentication scenario and a multi-user authentication scenario. The single-user authentication scenario represents a scenario where voice authentication, fingerprint authentication, or dual authentication (voice and fingerprint) is used to authenticate a target object and the associated operation permissions of that target object. The multi-user authentication scenario represents a scenario where voice authentication, fingerprint authentication, or dual authentication (voice and fingerprint) is used to authenticate at least two target objects and the associated operation permissions of each target object. In some embodiments, when the target authentication scenario is a multi-user authentication scenario, it may include a first target object and a second target object.

[0267] Based on this, prior to step 330, the method may further include the following steps:

[0268] Step 810: If both the first target object and the second target object are authenticated, then compare the control priorities of the first target object and the second target object.

[0269] Step 830: Obtain the working data corresponding to the target object with higher control priority, so as to control the first device to provide device control services based on the working data corresponding to the target object with higher control priority.

[0270] In this scenario, target authentication can include multiple target objects and the control permissions for each target object. For example, a first target object and its control permissions, a second target object and its control permissions. Control priority refers to the priority at which a target object controls the control device. Optionally, the priority can be determined based on the target object's identity; for example, administrators have higher priority, while ordinary users have lower priority.

[0271] Specifically, the first device can collect the voiceprint and / or fingerprint of a first target object and the voiceprint and / or fingerprint of a second target object. Based on a preset verification priority, the voiceprint and / or fingerprint of the first target object can be matched with a pre-stored voiceprint and / or fingerprint template in the first device. If the match is successful, the first target object is authenticated. Then, the voiceprint and / or fingerprint of the second target object can be matched with a pre-stored voiceprint and / or fingerprint template in the first device. If the match is successful, the second target object is authenticated. Next, the control priorities of the first target object and the second target object are compared. Finally, if the control priority of the first target object is higher than that of the second target object, the voice data of the first target object is acquired and used as working data.

[0272] For example, suppose there is an administrator and a regular user, and the administrator has higher verification and control priorities than the regular user. The device control command corresponding to the administrator's voice data is "lower the air conditioner temperature," while the device control command corresponding to the regular user's voice data is "turn off the air conditioner." The system collects the administrator's voiceprint and the regular user's voiceprint using a first device. First, the administrator's voiceprint is matched against a pre-stored voiceprint template in the first device. Then, the regular user's voiceprint is matched against the pre-stored voiceprint template in the first device. If both the administrator's and regular user's voiceprints match successfully, since the administrator's control priority is higher than the regular user's, the administrator's voice data is acquired and used as working data to generate the corresponding device control command "lower the air conditioner temperature."

[0273] In this embodiment, by authenticating the first target object and the second target object, dual authentication is achieved for different objects, which increases the difficulty for unauthorized users to control the first device and improves security. Furthermore, by controlling the priority, the corresponding device control commands are determined, which can flexibly respond to objects of different levels, avoid control conflicts between different objects, and ensure the stability and reliability of the control of the first device.

[0274] Referring to Figure 11, in an exemplary embodiment, a device includes a processing unit 1, an action recognition unit 2, and a voice acquisition unit 3. The action recognition unit 2 and the voice acquisition unit 3 are electrically connected to the processing unit 1.

[0275] Specifically, the action recognition unit 2 is used to acquire corresponding working data when the target function of the first device being woken up is the action recognition function.

[0276] In some embodiments, the motion recognition unit 2 includes a six-axis sensor.

[0277] In some embodiments, the six-axis sensor is always active. Therefore, when the first device performs an action, the six-axis sensor detects the corresponding action, and the first device determines the motion data generated by the six-axis sensor as either first motion data or second motion data. For example, when the six-axis sensor operates in low-power mode, it can detect the amplitude of the first device's movement, and the resulting motion data is the first feature data. When the six-axis sensor operates in high-power mode, it can detect different types of the first device's movements, and the resulting motion data is the second feature data or second motion data. It should be noted that the second feature data is distinct from the second motion data. The movement type of the first device represented by the second feature data includes at least one of picking up and shaking, while the movement type of the first device represented by the second motion data includes at least one of rotating, flipping, pushing, tapping, and vibrating. In this approach, the always-on six-axis sensor can detect both the acceleration and angular velocity of the first device during its movement. This helps to increase the amount and coverage of the first and second motion data. It also helps to reduce the number of hardware components actually activated in the first device operating in low-power mode. For example, the voice acquisition unit is turned off when the first device is operating in low-power mode, thereby reducing the cost of the control device.

[0278] The voice acquisition unit 3 is used to acquire the corresponding working data when the target function of the first device is voice acquisition function.

[0279] In some embodiments, the voice acquisition unit 3 includes a microphone.

[0280] In some embodiments, when the first device activates the microphone, it is considered that the voice acquisition function is awakened. At this time, the first device can use the microphone to collect voice and generate voice data, which is regarded as the working data under the voice acquisition function.

[0281] The processing unit 1 is used to acquire the working data obtained by the first device under the target function of wake-up, and to control the first device to provide device control services based on the working data.

[0282] In some embodiments, the first device further includes a communication unit for transmitting operational data with other devices. These other devices include, but are not limited to, gateways, servers, and other smart devices.

[0283] In some embodiments, the first device further includes at least one of an indicator light and / or a buzzer, both of which can be electrically connected to the processing unit. In this case, the processing unit 1 is also used to control the indicator light and / or buzzer to start when the device status and / or device data of the first device meet preset prompting conditions. The device status may include the first device's offline status, online status, normal working status, abnormal working status, fault status, etc., and the device data may refer to data such as power and battery level used to indicate the performance / power consumption of the first device; no specific limitation is made here. It should be noted that no specific limitation is made to the prompting conditions here, and the prompting conditions can be flexibly set according to actual needs. For example, a prompt can be made when the voice acquisition unit or motion recognition unit cannot be started; or, a prompt can also be made when the type of working data cannot be identified; or, for a rechargeable first device, a prompt can also be made when the battery level of the first device is lower than a set value.

[0284] In some embodiments, the first device can be configured to different modes so that different target functions can be activated in different modes, thereby obtaining working data under the activated target functions.

[0285] Figure 12 illustrates the process of the first device switching between different modes. In Figure 12, the modes that the first device can be configured with include: sleep mode, recognition mode, voice acquisition mode, and motion control mode.

[0286] As shown in Figure 12, specifically, when the first device is in sleep mode (which can be considered as being in hibernation), the motion recognition unit operates in low-power mode, and the voice acquisition unit is either off or operates in low-power mode. Specifically, the motion recognition unit operating in low-power mode can detect the amplitude of a movement when the first device performs a movement to generate first motion data. At this time, the first device can determine whether to switch from sleep mode to recognition mode based on the amplitude of the movement represented by the first motion data. Specifically, when the amplitude of the movement represented by the first motion data is greater than a set amplitude, the device switches from sleep mode to recognition mode.

[0287] When the first device is in recognition mode, the motion recognition unit operates in high-power mode, while the voice acquisition unit remains off or operates in low-power mode. In high-power mode, the motion recognition unit can detect different motion types when the first device performs an action, and then generate first motion data or second motion data based on these different motion types. At this time, the first device determines whether to switch the recognition mode to voice acquisition mode or motion control mode based on the motion type represented by the first or second motion data. Specifically, when the motion type represented by the first motion data is picking up or shaking, the device switches from recognition mode to voice acquisition mode; when the motion type represented by the second motion data is rotating, tapping, pushing, or vibrating, the device switches from recognition mode to motion control mode. It is worth noting that the rotation action here can refer to the entire first device rotating, in which case the second motion data corresponds to the motion data of the first state; or it can refer to the entire first device remaining stationary while the knob component of the first device rotates relative to the whole, in which case the second motion data corresponds to the motion data of the second state.

[0288] Understandably, if the first device stops moving, stops collecting voice data, or its movement amplitude decreases to less than a set amplitude, it can switch back to sleep mode. In sleep mode, the overall power consumption of the first device is lower, which can be achieved by reducing the power consumption of the motion recognition unit and shutting down the voice acquisition unit and communication unit.

[0289] To facilitate understanding, let's take a remote control as the first device as an example. When a user picks up the remote control, the remote control determines, based on first motion data, that the amplitude of the movement meets the conditions for switching to recognition mode (e.g., the amplitude exceeds a set amplitude). Therefore, the remote control switches from sleep mode to recognition mode. Then, when the user rotates the remote control, the remote control determines the movement type as rotation based on second motion data. Since the remote control has pre-configured the movement type as rotation, it transitions from recognition mode to motion control mode. After entering motion control mode, the remote control can send the second motion data to the washing machine (depending on the first relationship), causing the washing machine to perform actions such as turning on, turning off, or heating up. Alternatively, it can determine the action to be performed by the washing machine based on the second motion data (depending on the second relationship), and then control the washing machine to perform that action. This depends on whether the washing machine associated with the first device has a pre-established first or second relationship. Similarly, if the user picks up the remote control, the remote control determines the movement type as picking up based on the first motion data and transitions from recognition mode to voice acquisition mode. In this case, the user can control the washing machine to perform actions such as turning on, turning off, or heating up via voice.

[0290] Through the above, the first device can switch between different modes to wake up different target functions in different modes, thereby obtaining the working data under the woken target function. This is convenient and quick, and helps to improve the ease of operation of the first device, as well as the ease of control and flexibility of other intelligent devices.

[0291] The following explanation will be based on the application of the above device control method to a smart home scenario. In this smart home scenario, the first device is a remote control, which has at least voice acquisition, motion recognition, and communication functions:

[0292] In the traditional way, users need to press physical buttons on the remote control to control smart devices such as air conditioners, lights, and curtains.

[0293] With the development of science and technology, the emergence of speech recognition technology has promoted the development of artificial intelligence. Speech recognition enables machines to understand human speech and perform corresponding operations according to verbal commands, realizing direct human-machine language communication and control. For example, wireless remote control devices, through the speech recognition function of the remote control, achieve the effect of voice control of intelligent devices.

[0294] Wireless remote control technology is primarily based on speech recognition, speech synthesis, and intelligent control technologies. These technologies collectively provide users with a more convenient and user-friendly operating experience.

[0295] In smart home scenarios, voice control has been widely adopted. For example, users can use voice commands to turn on lights, adjust light brightness, start a home theater, control air conditioning, and switch audio and video channels, taking home automation a significant step forward.

[0296] However, in existing voice control solutions, some remote controls that control smart devices via voice recognition require pressing a physical button to activate the voice acquisition function, and commands can only be sent via voice. This limits existing remote control solutions to directly controlling smart devices using physical buttons, and each button can only correspond to one type of command. Furthermore, some remote controls that control smart devices via voice recognition require picking up the remote and pressing a physical button to activate the voice acquisition function, reducing the convenience of device control.

[0297] Therefore, the device control methods provided in the embodiments of this disclosure can be applied to remote controls, which can emit wireless signals to remotely control smart devices and can control smart devices through a combination of voice and action control. When a user picks up the remote control, the voice acquisition function is activated. At this time, the user issues commands through voice to control the specified smart device, and then continues to trigger remote control actions to adjust the state of the specified smart device.

[0298] For example, the following scenarios:

[0299] Scenario 1: After picking up the remote control, the user activates its voice control function. The user says, "Adjust the brightness of the light," and then rotates the remote control to adjust the brightness of the light; rotating clockwise increases brightness, and rotating counterclockwise decreases brightness. In other words, the user's action of "picking up" the remote control first activates its voice control function, and then the user controls the designated device via voice.

[0300] It is understandable that in scenario 1, the voice collection function is activated by first controlling the first device (i.e., the remote control) to perform an action (i.e., picking it up), then the third device (i.e., the light) is determined based on the voice data (i.e., adjusting the brightness of the light), and finally the operating parameters (i.e., brightness) of the light are adjusted based on the second motion data (i.e., rotating right / left), so that the third device (i.e., the light) performs the brightness adjustment action.

[0301] Scenario 2: After picking up the remote control, activate its voice capture function and say "adjust volume." Then, rotate the remote control to adjust the volume of the sound played from the speaker; rotating clockwise increases the volume, and rotating counterclockwise decreases it. In other words, the user's "picking up" action first activates the remote control's voice capture function, and then the user controls the designated device via voice.

[0302] It is understandable that scenario 2 involves first controlling the first device (i.e., the remote control) to activate the voice acquisition function, then determining the operating parameters (i.e., volume) to be adjusted based on the voice data (i.e., adjusting the volume), and finally adjusting the operating parameters (i.e., brightness) of the associated fourth device (i.e., the speaker) based on the second motion data (i.e., rotating right / left), so that the fourth device (i.e., the speaker) performs the volume adjustment action.

[0303] The difference from Scenario 1 is that the third device in Scenario 1 is determined based on voice data, while the fourth device in Scenario 2 has a pre-established association with the first device.

[0304] Scenario 3: After picking up the remote control, activate its voice capture function and say "adjust the curtains." Then, rotate the remote control to adjust the opening and closing of the curtains; rotating clockwise gradually opens the curtains, and rotating counterclockwise gradually closes them. In other words, the user's action of "picking up" the remote control first activates its voice capture function, and then the user controls the designated device via voice.

[0305] It is understandable that scenario 3 involves first controlling the first device (i.e., the remote control) to activate the voice collection function, then determining the third device (i.e., the curtains) to be controlled based on the voice data (i.e., adjusting the curtains), and finally adjusting the corresponding operating parameters (i.e., the degree of opening and closing) of the third device based on the second motion data (i.e., right rotation / left rotation), so that the third device (i.e., the curtains) performs the opening and closing adjustment action.

[0306] The difference from Scenario 1 is that the operating parameters in Scenario 1 are determined based on voice data, while the operating parameters in Scenario 3 are preset. For example, the operating parameters of the curtains are preset to the degree of opening and closing, so that when the third device is identified as the curtains, the degree of opening and closing is adjusted accordingly.

[0307] Scenario 4: After picking up the remote control, activate its voice capture function and say "turn on the light," and the light will turn on. This means you can directly control a specific device via voice without any further action; it's simply voice control.

[0308] It is understandable that scenario 4 involves first controlling the first device (i.e., the remote control) to activate the voice collection function, and then determining the third device (i.e., the light) and the action to be performed (i.e., the on / off state) based on the voice data (i.e., turning on the light), so that the third device (i.e., the light) performs the on / off state action.

[0309] The difference from scenarios 1-3 is that in scenario 4, after obtaining the voice data, the second smart device and the action to be performed can be determined simultaneously. Therefore, the user does not need to operate the first device again, which means there is no need to generate second motion data.

[0310] Scenario 5: After configuring the automation scheme, rotating the wireless remote control and shaking it (or performing other actions such as reversing or pushing) will turn on the light. This means that the state of a specific device can be directly adjusted simply by using an action, i.e., motion control alone.

[0311] Scenario 5 can be understood as follows: First, the motion recognition function is activated by manipulating the first device (i.e., the remote control) to perform an action (i.e., rotation). Then, based on the action type represented by the second motion data (i.e., shaking), the third device (i.e., the light) and the action to be performed (i.e., on / off state) are determined, causing the third device (i.e., the light) to perform the on / off action. It should be noted that if the user taps the remote control after activating the motion recognition function, the air conditioner will turn on. This means that different action types represented by the second motion data can result in different actions for the third device and / or the action to be performed. The specific actions to be performed by the different third devices can be determined based on a pre-configured automation scheme.

[0312] The difference from scenarios 1-4 is that in scenario 5, different action types are used to make the remote control no longer activate the voice function, but instead execute the action recognition function, and then perform action control to achieve control of the designated first smart device.

[0313] The above scenarios are merely examples; specific control can be implemented based on the user's actual automation configuration. In other words, if the remote control is pre-configured with an automation scheme between actions and smart devices, the remote control can perform the action according to the configured automation scheme after a corresponding action occurs. For instance, in scenario 5 above, because of the pre-configured automation scheme, shaking the remote control will turn on the light. Conversely, if the automation scheme associates shaking with air conditioning, shaking the remote control will turn on the air conditioner, not the light.

[0314] Figure 13 is a schematic diagram of a specific implementation of a device control method in an application scenario. As shown in Figure 13, this application scenario is a smart home scenario, which includes a cloud (i.e., the server 170 in the implementation environment shown in Figure 1), a gateway (i.e., the gateway 150 in the implementation environment shown in Figure 1, which can also be understood as a second device bound to the first device), and the first device and other smart devices deployed in the gateway (i.e., smart devices 130 in the implementation environment shown in Figure 1, such as the third device, the fourth device, etc.).

[0315] In this smart home scenario, the first device can be a remote control with voice capture and motion recognition capabilities. This remote control is equipped with a microphone for capturing voice and a six-axis sensor for capturing motion, thus enabling dual voice and motion control of other smart devices. The motion recognition function can be activated based on the amplitude of the motion represented by first motion data (i.e., first feature data), while the voice capture function can be activated based on the type of motion represented by the first motion data (i.e., second feature data). Optionally, the first feature data can be detected when the first device operates in low-power mode, and the second feature data can be obtained when the first device operates in high-power mode.

[0316] Given that the target functions for waking up the first device include voice acquisition and motion recognition, the device control method in this smart home scenario will now be described in conjunction with Figure 13 and various embodiments of this disclosure:

[0317] Scenario 1: Smart Light Control

[0318] 1. The user issues a command via remote control: "Turn on the living room lights."

[0319] 2. The remote control's microphone captures voice data and converts it into text using built-in voice recognition software.

[0320] 3. The six-axis sensor simultaneously detects the motion data corresponding to the user's rotation of the remote control, and uses the built-in motion recognition algorithm to analyze the rotation of the remote control and convert the rotation angle into a brightness adjustment ratio.

[0321] 4. The gateway receives voice data and motion data and uploads them to the cloud.

[0322] 5. Semantic analysis is performed in the cloud, combining voice and action data to determine the user's intent to turn on the living room light and adjust its brightness. It's worth noting that if there's only one smart light A in the living room, the user's intent is determined to be turning on light A. If there are multiple smart lights in the living room, such as smart light A and smart light B, the user's intent is determined to be turning on both lights A and B. This achieves dynamic selection of smart devices, effectively avoiding the limitation of single-device control.

[0323] 6. Generate a first control command and a second control command in the cloud. The first control command is used to turn on the living room light, and the second control command is used to adjust the brightness of the living room light to the user's desired adjustment ratio. Send the above control commands back to the gateway.

[0324] 7. The gateway receives these instructions and sends them to the living room light. First, it turns on the living room light, and then adjusts the brightness of the living room light according to the adjustment ratio, so as to achieve a smooth transition of the brightness of the living room light.

[0325] Scenario 2: Smart Curtain Control

[0326] 1. The user issues a command via remote control: "Open the curtains."

[0327] 2. The above voice commands are captured by the remote control and converted into text.

[0328] 3. Users can also rotate the knob on the remote control to adjust the opening and closing ratio of the curtains. Accordingly, the six-axis sensor can detect the rotation of the knob and convert the rotation angle into the adjustment ratio of the curtain opening and closing.

[0329] 4. The gateway uploads voice data and second motion data to the cloud.

[0330] 5. Semantic analysis is performed in the cloud to generate a first control command and a second control command to control the opening and closing of the curtains. The first control command opens the curtains, and the second control command indicates the specific degree to which the curtains are opened. It should be noted that in smart home scenarios, curtains can often be deployed in more than one room. In this case, the user's room (e.g., the living room) can be used as a setting association condition. The "curtains" in the voice data can be associated with a specific room, thus determining the user's intention as opening the living room curtains. This enables dynamic selection of smart devices and effectively avoids the limitation of single-device control.

[0331] 6. The cloud sends the above control commands to the gateway.

[0332] 7. The gateway receives these instructions and sends them to the smart curtains, which then open the curtains according to the adjustment ratio.

[0333] In this application scenario, the combination of voice and gestures via remote control enables users to control smart devices in multiple dimensions. This not only improves the flexibility and accuracy of smart home control and greatly enhances the convenience of user control, but also allows users to control any smart device in the smart home scenario more naturally and intuitively, thus improving the user experience.

[0334] Of course, in other application scenarios, the use of voice and actions provided by the remote control is not limited to voice controlling a third device to perform device adjustment actions and actions controlling the third device to adjust operating parameters. It can also be that voice is used to control the third device to adjust one operating parameter, such as the color temperature of the living room light, and actions are used to control the third device to adjust another operating parameter, such as the brightness of the living room light. Alternatively, voice can be used to control the third device to perform one device adjustment action, such as turning on the living room light, and actions can be used to control the third device to perform another device adjustment action, such as turning off the living room light. The device control principle is basically the same, and this application scenario does not constitute a specific limitation.

[0335] Figure 14 is a hardware structure block diagram of the first device in an application scenario. In Figure 14, the first device includes a microphone (i.e., a voice acquisition unit), a main controller (i.e., a processing unit), a memory, a six-axis sensor (i.e., a motion recognition unit), and a battery. The battery can be a button cell battery, which powers the entire first device. The main controller, or main control chip, is used to control the communication, calculation, and control functions of the entire first device. The six-axis sensor communicates with the main controller via an IIC interface, and the sensor's signals are used for motion recognition. Generally, the six-axis sensor operates in a low-power mode. The microphone is used to acquire voice signals. There can be one or more microphones; this application scenario is not limited to this.

[0336] As shown in Figure 15, the first device can switch between the following four operating modes:

[0337] (1) Under normal circumstances, the first device is in sleep mode. Sleep mode means that the main controller enters a sleep state, the microphone power is turned off, and the memory is disabled; only the six-axis sensor measures at a low rate. The six-axis sensor includes a three-axis accelerometer and a gyroscope. In sleep mode, the three-axis accelerometer operates in a low-power mode, and the gyroscope is powered off.

[0338] (2) When the first device is moved, if the measured value of the 3-axis accelerometer is greater than the threshold, the main controller wakes up from sleep mode, and the first device enters recognition mode. Recognition mode refers to the working mode in which the main controller reads the measured values ​​of the six-axis sensor and performs calculations and recognition actions. The specific workflow includes:

[0339] First, the six-axis sensor turns on the power to the internal gyroscope, which then starts working and increases the sampling rate of the three-axis accelerometer.

[0340] Secondly, the six-axis sensor simultaneously samples the values ​​from the three-axis accelerometer and gyroscope and writes the sampled values ​​into the built-in memory of the six-axis sensor. When the built-in memory of the six-axis sensor is full, an interrupt is output to the main controller. The main controller reads the measurement data from the three-axis accelerometer and gyroscope in the built-in memory of the six-axis sensor. After the main controller has read enough data (the amount of data to be read is defined when the software and algorithm are written), it notifies the six-axis sensor to enter sleep mode.

[0341] The main control chip then uses the read data to calculate and identify the action type, which can include at least six action types: picking up, shaking, rotating, tapping, pushing, and vibrating.

[0342] Finally, the device is determined to be in sleep mode, motion control mode, or voice acquisition mode based on the type of action detected.

[0343] (3) In recognition mode, if a "pick up" or "shake" action is detected, the first device enters voice acquisition mode. Voice acquisition mode refers to the working mode that sends voice data to the gateway. The specific workflow includes:

[0344] First, the main controller starts up and attempts to connect to the gateway;

[0345] Next, the speech recognition unit begins to pick up audio and transmits the acquired speech data to the main control unit;

[0346] Secondly, the main controller determines whether it has connected to the gateway. If it has, it sends the voice data to the gateway. If the device has not connected to the gateway, it stores the voice data in the chip until the first device connects to the gateway and sends all the voice data to the gateway. If the main controller cannot connect to the gateway within 3 seconds, it discards the voice data sampled and stored this time and directly enters sleep mode.

[0347] Finally, if the first device receives a recognition success flag from the gateway, it indicates that the voice data transmission has ended. It then turns off the power of the voice recognition unit, stops reading and sending voice data, and enters sleep mode. If it does not receive a recognition success flag from the gateway, it continues to read, compress, and send voice data to the gateway for a total of 10 seconds (the duration can be adjusted according to actual needs) for the voice recognition unit to sample voice data. After that, the first device automatically enters sleep mode.

[0348] It is worth mentioning that after recognizing the "pick up" or "shake" action, the action type can be further judged. This judgment is a further judgment after the first "pick up" or "shake" action. If one of the actions of rotation, knocking, pushing and vibration is further recognized, the first device can directly control the third or fourth device to perform the corresponding action, that is, enter the action control mode.

[0349] (4) In the recognition mode, if at least one of the following actions is detected: rotation, tapping, pushing and vibration, the first device enters the motion control mode until the first device stops moving or the motion amplitude is less than the set amplitude, then the first device returns to the sleep mode.

[0350] (5) If an undefined action type is detected, the first device returns to sleep mode.

[0351] This enables the first device to switch between four operating modes.

[0352] As shown in Figure 16, the device control process of the first device controlling other smart devices with the help of the second device (gateway) may include the following steps:

[0353] In the first step, the first device is initially in sleep mode and the six-axis sensor measures the amplitude of the first device's movements in low-power mode;

[0354] The second step is that if the measured motion amplitude does not exceed the set amplitude, the first device remains in sleep mode (the main controller operates at low power); when the motion amplitude exceeds the set amplitude, the main controller is awakened (operates at high power) and the six-axis sensor is configured to turn on the gyroscope power and the accelerometer is configured to measure and read their values ​​(motion parameters) at high speed to calculate the motion type, which is considered as entering the recognition mode, and the first motion data or the second motion data can be obtained.

[0355] Third, if the action type is one of "vibration", "flipping", "tapping (twice)", "pushing", and "rotating", the main controller starts the protocol to connect to the gateway (the bound gateway or parent node, which supports the protocol); if the gateway is connected, the identified action type is sent to the gateway (the gateway executes the corresponding action according to the preset operation rules) and then the first step is automatically entered; if the gateway is not connected, a reminder is given and then the first step is automatically entered; the gateway can control the corresponding smart device to perform the corresponding action based on the received second motion data.

[0356] Fourth, if the action type is "pick up" or "shake", the main controller initiates the protocol to connect to the gateway (the bound gateway or parent node, supporting the protocol), simultaneously turning on the power of both microphones and reading and compressing the voice data. If connected to the gateway, the main controller sends the compressed voice data to the gateway via wireless communication while simultaneously reading and compressing the voice data from both microphones. Reading, compression, and wireless transmission of voice data cease immediately upon receiving a "successful recognition" flag from the gateway, and the process automatically proceeds to the first step. Otherwise, reading, compression, and wireless transmission of voice data continue until a "successful recognition" flag is received from the gateway after 10 seconds, at which point the process automatically stops and proceeds to the first step. Based on the received first motion data, the gateway can control the corresponding smart device to perform the appropriate action.

[0357] This enables the first device to control other smart devices using its deployed gateway.

[0358] Compared to related technologies, 1) Dual voice and action control: This provides a more precise dual control method for smart homes through voice and action, offering a more natural, intuitive, and multi-dimensional control approach; 2) Dynamic selection of smart devices: Voice recognition technology can analyze user voice to determine user intent and the smart devices the user needs to control, overcoming the limitation of existing technologies that can only specify a single device; 3) Multi-dimensional device control: This provides a smooth, multi-dimensional device control experience through the same remote control, allowing users to specify actions to be performed by smart devices via voice and also to specify operating parameters to be adjusted by capturing actions through a six-axis sensor; 4) Intelligent semantic analysis: Semantic analysis is performed locally (i.e., on the remote control) or in the cloud to determine user intent through voice and / or action, generating control commands and enabling real-time identification of the devices the user needs to control, increasing control accuracy.

[0359] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0360] The following are embodiments of the apparatus disclosed herein, which can be used to execute the device control method involved in this disclosure. For details not disclosed in the embodiments of the apparatus disclosed herein, please refer to the method embodiments of the device control method involved in this disclosure.

[0361] Please refer to Figure 17. In this embodiment of the disclosure, a device control device 900 is provided, including but not limited to: a data acquisition module 910 and a device control module 930.

[0362] The data acquisition module 910 is used to acquire the working data obtained by the first device under the target function of wake-up.

[0363] The equipment control module 930 is used to control the first equipment and provide equipment control services based on the working data.

[0364] It should be noted that the device control device provided in the above embodiments is only illustrated by the division of the above functional modules when controlling the device. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the device control device will be divided into different functional modules to complete all or part of the functions described above.

[0365] Furthermore, the device control apparatus and device control method embodiments provided in the above embodiments belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0366] Please refer to Figure 18. In this embodiment of the disclosure, a device 4000 is provided, which may include: a remote control, a gateway, a server, etc.

[0367] In Figure 18, the device 4000 includes at least one processor 4001 and at least one memory 4003.

[0368] Data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not indicate that there is only one bus or one type of bus.

[0369] Optionally, the device 4000 may further include a transceiver 4004, which can be used for data interaction between the device and other devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0370] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0371] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing computer programs having instruction or data structure forms and accessible by the device 4000, but not limited to these.

[0372] The memory 4003 stores a computer program, and the processor 4001 can read the computer program stored in the memory 4003 through the communication bus 4002.

[0373] The computer program is executed by one or more processors 4001 to implement the device control methods in the above embodiments.

[0374] Furthermore, this disclosure provides a storage medium storing a computer program that is executed by one or more processors to implement the device control method described above.

[0375] This disclosure provides a computer program product, including a computer program that is executed by one or more processors to implement the device control method described above.

[0376] Compared with related technologies, firstly, this disclosure can wake up different target functions of the first device based on the first motion data, which improves the ease of waking up the control device; secondly, under the target function of the first device, this disclosure can obtain corresponding working data, which enables users to control any intelligent device different from the first device based on the working data. At the same time, different action types can control the first device to provide corresponding device control services to other intelligent devices, which improves the ease of operation of the control device.

[0377] The above description is only a partial embodiment of this disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure.

Claims

1. A device control method in which, include: Acquire the working data obtained by the first device under the target function that has been activated; Based on the working data, the first device is controlled to provide device control services.

2. The method of claim 1, wherein, Before acquiring the working data obtained by the first device under the target function after wake-up, the method includes: Acquire the first motion data of the first device; The first device is activated based on the first motion data, and the target function includes at least one of a first function and a second function.

3. The method of claim 2, wherein, The step of waking up the target function of the first device based on the first motion data includes: When the first device is in a sleep state, the first function of the first device is activated based on the first motion data. When the first function of the first device is activated, the second function of the first device is activated based on the first motion data.

4. The method of claim 3, wherein, The first motion data includes first feature data for characterizing the range of motion of the first device; The step of waking up the first function of the first device based on the first motion data includes: If the range of motion of the first device represented by the first feature data is greater than a set range, then the first function of the first device is activated.

5. The method of claim 3, wherein, The first motion data includes second feature data used to characterize the type of motion of the first device; The step of waking up the second function of the first device based on the first motion data includes: If the action type of the first device represented by the second feature data matches the set action type, then the second function of the first device is activated.

6. The method of claim 3, wherein, The first function is motion recognition, and the second function is voice acquisition.

7. The method of claim 2, wherein, The first motion data includes first feature data for characterizing the amplitude of the first device's movement and second feature data for characterizing the type of movement of the first device; The step of waking up the target function of the first device based on the first motion data includes: If the range of motion of the first device represented by the first feature data is greater than the set range, and the type of motion of the first device represented by the second feature data does not conform to the set type of motion, then the first function of the first device is activated. If the range of motion of the first device represented by the first feature data is greater than a set range, and the type of motion of the first device represented by the second feature data matches the set type of motion, then the second function of the first device is activated.

8. The method of claim 2, wherein, After waking up the target function of the first device based on the first motion data, the method further includes: If the first device is not connected to the second device bound to it, then the first device is controlled to enter a sleep state; If the first device stops collecting voice data or the operating status of the first device meets the conditions for stopping voice data collection, then the first device is controlled to enter the sleep state. If the movement range of the first device is less than or equal to the set range, then the first device is controlled to enter the sleep state; If the action type of the first device does not match the set action type, then the first device is controlled to enter the sleep state.

9. The method of claim 1, wherein, The step of acquiring the working data obtained by the first device under the awakened target function includes: If it is determined that the target function of the first device being woken up is the action recognition function, then it is detected whether the first device is in a first state; If the first device is detected to be in the first state, then the second motion data of the first device corresponding to the first state is obtained; The second motion data of the first device corresponding to the first state is used as the working data of the first device under the motion recognition function.

10. The method of claim 1, wherein, The step of acquiring the working data obtained by the first device under the awakened target function includes: If it is determined that the target function of the first device being woken up is the action recognition function, then it is detected whether the first device is in the second state; If the first device is detected to be in the second state, then it is detected whether the components configured in the first device have been triggered to perform a relative motion operation; If a relative motion operation is triggered on a component configured in the first device, then the second motion data of the first device corresponding to the second state is obtained; The second motion data of the first device corresponding to the second state is used as the working data of the first device under the motion recognition function.

11. The method of claim 1, wherein, The step of controlling the first device to provide device control services based on the working data includes: Based on the work data, a third device and an action are determined to instruct the third device to perform the action; or Control a fourth device associated with the first device based on the work data, and instruct the fourth device to perform the action; or The working data is sent to a second device bound to the first device to instruct the second device to control the third device or the fourth device to perform the action based on the working data.

12. The method of claim 11, wherein, The action includes at least one of the following: equipment status adjustment action and operating parameter adjustment action.

13. The method of claim 11, wherein, The step of determining a third device and an action based on the working data, and instructing the third device to perform the action, includes: Based on the third device determined from the working data and the action, a first control command is generated to instruct the third device to perform the action in response to the first control command; and / or Based on the working data and the determined adjustment action of the third device and operating parameters, a second control command is generated to instruct the third device to respond to the second control command and perform the adjustment action of the operating parameters.

14. The method of claim 13, wherein, The step of determining a third device and an action based on the working data, and instructing the third device to perform the action, includes: If the working data includes voice data, then based on the third device determined by the voice data and the action, the first control command is generated to instruct the third device to respond to the first control command and execute the action; If the working data also includes second motion data, then based on the voice data and / or the second motion data, the third device and the operating parameter adjustment action are determined, and based on the third device and the operating parameter adjustment action, the second control command is generated to instruct the third device to respond to the second control command and execute the operating parameter adjustment action.

15. The method of claim 14, wherein, Before determining the third device and the operating parameter adjustment action based on the voice data and / or the second motion data, and generating the second control command based on the third device and the operating parameter adjustment action, the method further includes: Determine the time interval between the acquisition time of the voice data and the acquisition time of the second motion data; If the time interval is less than or equal to a set threshold, then the control object of the first device is determined to be the third device, and the steps of determining the third device and the operation parameter adjustment action based on the voice data and / or the second motion data, and generating the second control command based on the third device and the operation parameter adjustment action are executed.

16. The method of claim 14, wherein, Before determining the third device and the operating parameter adjustment action based on the voice data and / or the second motion data, and generating the second control command based on the third device and the operating parameter adjustment action, the method further includes: Determine the time interval between the acquisition time of the voice data and the acquisition time of the second motion data; If the time interval is greater than a set threshold, the action type of the first device is determined based on the second motion data; If the action type meets the triggering condition in the first automation scheme, then the action associated with the triggering condition is extracted from the first automation scheme; the first automation scheme controls the fifth device to execute the action by monitoring whether the action type meets the triggering condition. A third control command is generated based on the fifth device and the action to instruct the fifth device to perform the action in response to the third control command.

17. The method of claim 14, wherein, After generating the first control command based on the third device determined from the voice data and the action, the method further includes: If the working data does not include the second motion data, then based on the third device determined by the voice data and the operating parameter adjustment action, a second control command is generated to instruct the third device to respond to the second control command and execute the operating parameter adjustment action.

18. The method of claim 14, wherein, The step of generating the first control command based on the third device determined from the voice data and the action includes: The voice data is processed by speech recognition to obtain the candidate device indicated by the voice data and the action. If the clarity of the candidate device pointed to by the voice data is blurry, then according to the set association conditions, the third device is obtained by associating the candidate device pointed to by the voice data; the third device is an intelligent device with the same device attributes as the candidate device. Based on the third device and the action, a first control command is generated to instruct the third device to perform the action.

19. The method of claim 18, wherein, The step of associating the third device with the candidate device pointed to by the voice data according to the set association conditions includes: The location of the device is determined based on the aforementioned associated conditions; The candidate device pointed to by the voice data is associated with the space where the device is located to obtain the third device; the third device refers to the candidate device located in the space where the device is located.

20. The method of claim 14, wherein, The step of determining the third device and the operating parameter adjustment action based on the voice data and / or the second motion data, and generating the second control command based on the third device and the operating parameter adjustment action, includes: The operating parameters to be adjusted for the third device are determined based on the voice data and / or the second motion data. The adjustment ratio corresponding to the operating parameters is determined based on the voice data and / or the second motion data; Generate a second control command to instruct the third device to adjust the operating parameters according to the adjustment ratio.

21. The method of claim 20, wherein, The step of determining the operating parameters to be adjusted for the third device based on the voice data and / or the second motion data includes: If the voice data does not contain the operating parameters, the action type of the first device is determined based on the second motion data; If the action type is detected to meet the trigger condition in the configured second automation scheme, the operating parameters associated with the trigger condition are extracted from the second automation scheme; the second automation scheme controls the third device to adjust the operating parameters by monitoring whether the action type meets the trigger condition.

22. The method of claim 20, wherein, The step of determining the operating parameters to be adjusted for the third device based on the voice data and / or the second motion data includes: If the voice data does not contain the operating parameters, then at least one candidate parameter that the third device can adjust is determined, and the priority of each candidate parameter is obtained. The candidate parameter with the highest priority is used as the running parameter.

23. The method of claim 20, wherein, Determining the adjustment ratio corresponding to the operating parameters based on the voice data and / or the second motion data includes: Obtain the motion parameters of the first device from the second motion data. The motion parameters include one or more of the following: motion direction, motion amplitude, motion angle, motion duration, motion speed, acceleration, and angular velocity. Based on the motion parameters, determine the adjustment ratio corresponding to the operating parameters.

24. The method of claim 11, wherein, The step of controlling a fourth device associated with the first device based on the working data to instruct the fourth device to perform the action includes: If the working data includes second motion data, then the action type of the first device is determined based on the second motion data; The fourth device associated with the first device is determined based on the action type and the first relationship, or the action performed by the fourth device is determined based on the action type and the second relationship; wherein, the first relationship includes the correspondence between different action types and the corresponding fourth devices, and the second relationship includes the correspondence between different action types and the corresponding actions performed by the fourth device; If the fourth device associated with the first device is determined, the second motion data is sent to the fourth device to instruct the fourth device to perform the corresponding action based on the second motion data; If the action performed by the fourth device is determined, a fourth control command is generated based on the fourth device and the action to instruct the fourth device to perform the corresponding action in response to the fourth control command.

25. The method of claim 1, wherein, Before controlling the first device to provide device control services based on the working data, the method further includes: Collect biological and / or environmental data of the target object in a target authentication scenario; the target authentication scenario is used to indicate the target object's control permissions over the first device. The target object is authenticated based on the biological data and / or the environmental data; If the target object passes the authentication, the working data corresponding to the target object is obtained, so as to control the first device to provide device control services based on the working data corresponding to the target object.

26. The method of claim 25, wherein, The target authentication scenario includes a first target object and a second target object; Before controlling the first device to provide device control services based on the working data, the method further includes: If both the first target object and the second target object pass the authentication, then compare the control priorities of the first target object and the second target object; The working data corresponding to the target object with higher control priority is obtained, so as to control the first device to provide device control services based on the working data corresponding to the target object with higher control priority.

27. The method of claim 25, wherein, The biometric data includes at least one of voiceprint data and fingerprint data.

28. An apparatus, comprising: It includes a processing unit, an action recognition unit, and a voice acquisition unit; the action recognition unit and the voice acquisition unit are respectively electrically connected to the processing unit; The motion recognition unit is used to acquire corresponding working data when the target function of the device being woken up is the motion recognition function; The voice acquisition unit is used to acquire corresponding working data when the target function of the device is awakened is the voice acquisition function; The processing unit is used to acquire the working data obtained by the device under the activated target function; and The processing unit is also used to control the device to provide device control services based on the working data.

29. The apparatus of claim 28, wherein, The motion recognition unit includes a six-axis sensor; the voice acquisition unit includes a microphone.

30. The apparatus of claim 28, wherein, The working data includes at least one of the following: first motion data, second motion data, and voice data; The first motion data represents the type of action of the device, including at least one of picking up and shaking; the second motion data represents the type of action of the device, including at least one of rotating, flipping, pushing, striking, and vibrating.

31. An apparatus, wherein, It includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, implements the device control method as described in any one of claims 1 to 27.

Images