Intelligent device control method and device, interactive device and storage medium

Abstract

The present invention provides a control method and device for an intelligent device, an interactive device and a storage medium. According to the method, when the interactive device detects a hand motion trajectory generated by a user in a wake-up state, a target sensing area can be determined in multiple sensing areas set on the interactive device, and based on the output signal value generated by each distance sensor pre-set on the target sensing area, the target operation gesture corresponding to the hand motion trajectory is determined in multiple preset operation gestures. Then, the control instruction corresponding to the target operation gesture is determined, and the intelligent device associated with the interactive device is controlled according to the control instruction. By applying the method of the present invention, the user can trigger the interactive device to perform gesture recognition through hand movement, and then control the intelligent device, without having to perform cumbersome control operations through mobile terminals such as mobile phones, which can save and simplify the user's operation process and improve control efficiency.

Description

Technical Field

[0001] The present invention relates to the field of interactive control technology, and in particular to a control method and device for an intelligent device, an interactive device and a storage medium. Background Art

[0002] With the development of Internet of Things technology, many smart devices have gradually entered people's field of vision, such as various smart homes. Users can remotely control smart devices, such as remotely controlling smart lamps to start or end lighting, without using the physical control device of the device, which brings great convenience to people's lives.

[0003] As people's living standards continue to improve, the number of smart devices used in daily life is also increasing. At present, the remote control functions of various smart devices are usually bound to their corresponding application software, so users usually need to install multiple application software on mobile terminals such as mobile phones. When a user controls a smart device, he needs to use a mobile terminal to open the corresponding application software, connect the corresponding smart device, and perform corresponding control operations in the application software. This makes the process of users controlling smart devices more cumbersome, takes too much time and energy, and has low control efficiency. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method for controlling a smart device to solve the problem that the process of controlling the smart device by a user is relatively complicated and the operation efficiency is low.

[0005] The embodiment of the present invention also provides a control device for an intelligent device to ensure the practical implementation and application of the above method.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] A method for controlling an intelligent device, comprising:

[0008] When a hand motion trajectory of a user is detected relative to a preset interactive device, it is determined whether the interactive device is currently in an awake state; a plurality of sensing areas are provided in the interactive device, and a plurality of distance sensors are provided in each of the sensing areas according to the device form corresponding to the sensing area in the interactive device;

[0009] If the interactive device is currently in an awake state, then, among the multiple sensing areas, determining a target sensing area corresponding to the area where the hand motion trajectory is generated, and acquiring multiple output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory;

[0010] Determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, each of the distance sensor sets including a plurality of distance sensors for identifying the corresponding operation gesture;

[0011] Determining a target operation gesture corresponding to the hand motion trajectory based on a time series change of a plurality of output signal values ​​generated by each distance sensor in each of the distance sensor sets;

[0012] Determine a control instruction corresponding to the target operation gesture, and control the smart device associated with the interactive device according to the control instruction.

[0013] The above method may optionally further include:

[0014] If the interactive device is not currently in a wake-up state, determining whether the hand motion trajectory matches a preset wake-up gesture;

[0015] If the hand motion trajectory matches the wake-up gesture, the interactive device is switched from a current non-awakening state to a wake-up state.

[0016] In the above method, optionally, the determining whether the interactive device is currently in an awake state includes:

[0017] Determine an enable signal corresponding to the current interactive device;

[0018] Determining whether the enable signal is an enable signal that triggers the interactive device to be in a wake-up state;

[0019] If the enable signal is an enable signal that triggers the interactive device to be in the awake state, it is determined that the interactive device is currently in the awake state.

[0020] In the above method, optionally, determining, among the multiple sensing areas, a target sensing area corresponding to the area where the hand motion trajectory is generated, includes:

[0021] Determine the detection signal value corresponding to each of the sensing areas;

[0022] Determine a target detection signal value among the detection signal values, wherein the target detection signal value is the detection signal value with the largest value among the detection signal values;

[0023] The sensing area corresponding to the target detection signal value is used as the target sensing area.

[0024] Optionally, the above method, based on the time series changes of multiple output signal values ​​generated by each distance sensor in each distance sensor set, determines the target operation gesture corresponding to the hand motion trajectory, including:

[0025] For each of the operation gestures, constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture;

[0026] Determine a convolution kernel set corresponding to each of the operation gestures, wherein the convolution kernel set corresponding to each of the operation gestures includes at least one preset convolution kernel;

[0027] Performing a product operation on the signal value matrix corresponding to each operation gesture and each convolution kernel in the corresponding convolution kernel set to obtain an output matrix corresponding to each operation gesture;

[0028] For each of the operation gestures, compare the output matrix corresponding to the operation gesture with the preset matching matrix corresponding to the operation gesture, and if the output matrix corresponding to the operation gesture is the same as the preset matching matrix corresponding to the operation gesture, determine that the operation gesture matches the hand motion trajectory;

[0029] If there is only one operation gesture among the operation gestures that matches the hand motion trajectory, the operation gesture that matches the hand motion trajectory is used as the target operation gesture.

[0030] Optionally, the method of constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture includes:

[0031] For each distance sensor in the distance sensor set corresponding to the operation gesture, based on the timing of the distance sensor generating multiple output signal values, among the multiple output signal values ​​generated by the distance sensor, determine multiple output signal values ​​corresponding to each preset time interval, and determine an interval signal value set corresponding to the distance sensor, the interval signal value set including the interval signal value corresponding to each preset time interval, each of the interval signal values ​​being an average value of the multiple output signal values ​​corresponding to the corresponding preset time interval;

[0032] Each interval signal value in the interval signal value set corresponding to each distance sensor in the distance sensor set corresponding to the operation gesture is used as a matrix element to construct a first matrix, and the first matrix is ​​used as the signal value matrix corresponding to the operation gesture.

[0033] Optionally, the method of constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture includes:

[0034] Using each output signal value generated by each distance sensor in the distance sensor set corresponding to the operation gesture as a matrix element, constructing a second matrix;

[0035] The preset matrix dimension corresponding to the operation gesture is determined, and the second matrix is ​​converted into a third matrix corresponding to the preset matrix dimension, and the third matrix is ​​used as the signal value matrix corresponding to the operation gesture.

[0036] In the above method, optionally, the process of setting each convolution kernel in the convolution kernel set corresponding to each operation gesture includes:

[0037] Determine a plurality of signal samples corresponding to each of the operation gestures;

[0038] Determine each preset convolution matrix corresponding to each of the operation gestures;

[0039] For each of the operation gestures, based on the preset matching matrix corresponding to the operation gesture, the signal samples corresponding to the operation gesture and the preset gradient descent algorithm, the elements in each preset convolution matrix corresponding to the operation gesture are updated, and each preset convolution matrix that has been updated is used as the convolution kernel corresponding to the operation gesture.

[0040] In the above method, optionally, the interactive device has a cylindrical shape, and when the target sensing area is a side surface of the interactive device, a plurality of distance sensors in the target sensing area are evenly distributed in a circle in the target sensing area with respect to a center position point of the interactive device, and the center position point is any point on the center line of the interactive device;

[0041] In the process of determining the target operation gesture corresponding to the hand motion trajectory, if the hand motion trajectory contains a first sub-trajectory, a second sub-trajectory and a third sub-trajectory generated in sequence, and the first sub-trajectory matches the first operation gesture, the second sub-trajectory matches the second operation gesture, and the third sub-trajectory matches the third operation gesture, then it is determined whether the movement direction corresponding to the second operation gesture and the movement direction corresponding to the first operation gesture are opposite directions to each other; the first operation gesture, the second operation gesture and the third operation gesture are respectively operation gestures among the multiple operation gestures;

[0042] If the movement direction corresponding to the second operation gesture and the movement direction corresponding to the first operation gesture are opposite to each other, determining whether the movement direction corresponding to the third operation gesture is the same as the movement direction corresponding to the first operation gesture;

[0043] If the movement direction corresponding to the third operation gesture is the same as the movement direction corresponding to the first operation gesture, determining the operation duration corresponding to the second operation gesture, and judging whether the operation duration is less than a preset threshold;

[0044] If the operation duration is less than the preset threshold, the first operation gesture and the third operation gesture are superimposed, and the superimposed operation gesture is used as the target operation gesture corresponding to the hand motion trajectory.

[0045] A control device for an intelligent device, comprising:

[0046] A judgment unit, configured to judge whether the interactive device is currently in an awake state when detecting that a user generates a hand motion trajectory relative to a preset interactive device; the interactive device is provided with a plurality of sensing areas, and each of the sensing areas is provided with a plurality of distance sensors according to a device form corresponding to the sensing area in the interactive device;

[0047] A first determining unit is configured to determine, if the interactive device is currently in an awake state, a target sensing area corresponding to the area where the hand motion trajectory is generated among the multiple sensing areas, and obtain a plurality of output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory;

[0048] A second determining unit, configured to determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, wherein each of the distance sensor sets includes a plurality of distance sensors for identifying the corresponding operation gesture;

[0049] A third determining unit, configured to determine a target operation gesture corresponding to the hand motion trajectory based on a time series change of a plurality of output signal values ​​generated by each distance sensor in each of the distance sensor sets;

[0050] The control unit is used to determine the control instruction corresponding to the target operation gesture, and control the smart device associated with the interactive device according to the control instruction.

[0051] An interactive device, comprising:

[0052] An interactive device body, wherein the interactive device body is a cylindrical structure, and a front view surface of the interactive device body is a circular surface;

[0053] A circular display screen is provided on the front surface of the interactive device body, and the center point of the circular display screen coincides with the center point of the front surface of the interactive device body;

[0054] A plurality of distance sensors are arranged on the front surface of the interactive device body, and the distance sensors arranged on the front surface of the interactive device body are evenly distributed in a circle about the center point of the front surface of the interactive device body;

[0055] A plurality of indicator lights are arranged on the front surface of the interactive device body, and the plurality of indicator lights are evenly distributed in a circle about the center point of the front surface of the interactive device body;

[0056] A plurality of distance sensors are arranged on the side of the interactive device body, and the distance sensors arranged on the side of the interactive device body are evenly distributed in a circle about a target position point, and the target position point is any point on the center line of the interactive device body;

[0057] The interactive device body has a built-in control chip, and each distance sensor provided on the front surface and the side surface of the interactive device body is connected to the control chip via a serial interface;

[0058] The interactive device body has a built-in communication device.

[0059] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the control method of the smart device as described above.

[0060] Based on the above-mentioned embodiment of the present invention, a control method for an intelligent device is provided, the method comprising: when detecting that a user generates a hand motion trajectory relative to a preset interactive device, determining whether the interactive device is currently in a wake-up state, the interactive device is provided with multiple sensing areas, and each sensing area is provided with multiple distance sensors according to the device form corresponding to the sensing area in the interactive device; if the interactive device is currently in a wake-up state, determining the target sensing area corresponding to the area where the hand motion trajectory is generated, and obtaining multiple output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory; determining multiple operation gestures preset in the interactive device, and determining the distance sensor set corresponding to each operation gesture in the target sensing area, each distance sensor set including multiple distance sensors for identifying the corresponding operation gesture; determining the target operation gesture corresponding to the hand motion trajectory based on the time sequence change of the multiple output signal values ​​generated by each distance sensor in each distance sensor set; determining the control instruction corresponding to the target operation gesture, and controlling the intelligent device associated with the interactive device according to the control instruction. Applying the method provided by the embodiment of the present invention, the interactive device can identify the target operation gesture corresponding to the hand motion trajectory generated by the user, and control the associated intelligent device based on the control instruction corresponding to the target operation gesture. When users need to control smart devices, they can trigger interactive devices for identification and control through non-contact hand movements. Users do not need to perform tedious control operations through mobile terminals such as mobile phones. This can simplify the process of users controlling smart devices, save users' time and energy, and improve control efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0062] Figure 1 A method flow chart of a method for controlling an intelligent device provided by an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of the structure of an interactive device provided by an embodiment of the present invention;

[0064] Figure 3 A flowchart of another method for controlling a smart device provided by an embodiment of the present invention;

[0065] Figure 4 Another method flow chart of a method for controlling a smart device provided by an embodiment of the present invention;

[0066] Figure 5 A schematic diagram of a gesture movement direction provided by an embodiment of the present invention;

[0067] Figure 6 A schematic diagram of the structure of a control device for an intelligent device provided by an embodiment of the present invention;

[0068] Figure 7 Another structural schematic diagram of an interactive device provided by an embodiment of the present invention;

[0069] Figure 8 A schematic diagram of an interactive device in use provided by an embodiment of the present invention;

[0070] Fig. 9 Another device schematic diagram of an interactive device in use provided by an embodiment of the present invention;

[0071] Fig.10 Another device schematic diagram of an interactive device provided by an embodiment of the present invention in use. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0074] The embodiment of the present invention provides a control method for an intelligent device. The method can be applied to a device system of an interactive device. The execution subject can be a controller of the interactive device. The method flow chart of the method is as follows: Figure 1 As shown, including:

[0075] S101: When it is detected that a user generates a hand motion trajectory relative to a preset interactive device, determining whether the interactive device is currently in an awake state;

[0076] The method provided by the embodiment of the present invention can be applied to an interactive device, which is a polyhedral structure, such as a cylindrical structure, a polyhedral prism or a polyhedral pyramid structure, etc., and a plurality of, that is, at least two, sensing areas are arranged on the surface of the interactive device, and a plurality of distance sensors are arranged in each sensing area. The setting position of each distance sensor in each sensing area on the interactive device is set according to the device shape of the sensing area in which it is located.

[0077] Optionally, the interactive device provided in the embodiment of the present invention may be a cylindrical structure, and its structural diagram may be as shown in FIG. Figure 2 As shown in the figure, the front of the interactive device is a circular surface, which is a sensing area. The solid circles marked with C0~C11 in the figure represent the distance sensors installed on the front, that is, there are twelve distance sensors C0~C11 installed in the sensing area. These twelve distance sensors are evenly distributed in a circle about the center point of the front of the device, that is, they are distributed according to the positions of the twelve points of the clock. The side of the device is also a sensing area. The solid circles marked with D0~D3 in the figure represent the distance sensors installed on the side, and Figure 2 This is only a brief schematic diagram, and not all distance sensors are shown. In fact, twelve distance sensors D0 to D11 are set in this area. The twelve sensors on the side of the device are also evenly distributed. Figure 2 The structure shown is only a specific embodiment when the interactive device is a cylindrical structure, and does not limit the number of distance sensors provided on the interactive device with a cylindrical structure and the distribution of the sensors.

[0078] Optionally, the interactive device provided in the embodiment of the present invention may also be a rectangular parallelepiped structure, the front of which may be a sensing area, and the upper side, lower side, left side, and right side of which may be a sensing area. The distance sensors on the front of the device may be evenly distributed in a circle about the center point of the front of the device, and the distance sensors on each side may be evenly distributed along a straight line parallel to the intersection line of the side and the front.

[0079] It should be noted that the specific structure of the interactive device, the specific number of sensing areas, the device form corresponding to the sensing areas and the setting position of each distance sensor can all be set according to actual needs, as long as they can complete the corresponding motion signal collection task, without affecting the function of the method provided in the embodiment of the present invention.

[0080] The placement of the interactive device in the method provided in the embodiment of the present invention can be hung on a wall, suspended in space, or placed on a table, as long as it does not affect the detection of the user's hand movement trajectory. When the user needs to control the smart device, the user can move his hand relative to the interactive device within the detection range of the interactive device, that is, use the hand as a corresponding gesture in the spatial range. When the interactive device detects the hand movement trajectory through the detection of various distance sensors, it can be determined whether the interactive device is currently in an awake state, that is, whether the current hand movement trajectory is identified to determine the corresponding control instruction. The interactive device being in an awake state indicates that the user can control the smart device through the interactive device.

[0081] S102: If the interactive device is currently in an awake state, determining, among the multiple sensing areas, a target sensing area corresponding to the area where the hand motion trajectory is generated, and obtaining multiple output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory;

[0082] In the method provided by the embodiment of the present invention, if the interactive device is currently in the awake state, the target sensing area can be determined based on the output signals of each distance sensor in each sensing area, and the target sensing area is the area facing the user's hand motion trajectory. The interactive device can continuously record the output signal values ​​generated by each distance sensor, and can obtain the output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory from the pre-recorded information. The output signal value generated by the distance sensor represents the distance between it and the object being measured.

[0083] S103: Determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, each of the distance sensor sets including a plurality of distance sensors for identifying the corresponding operation gesture;

[0084] In the method provided in the embodiment of the present invention, a plurality of operation gestures are pre-set in the interactive device, such as swiping up, swiping down, swiping left, swiping right, rotating clockwise, rotating counterclockwise, and the like. Each operation gesture may correspond one-to-one to each control instruction for controlling the smart device. In each distance sensor in the target sensing area, each operation gesture corresponds to a plurality of distance sensors for identifying the operation gesture, that is, the output signal of the corresponding distance sensor is required to identify the operation gesture.

[0085] It should be noted that the distance sensor sets corresponding to different operation gestures may be the same. The distance sensor set corresponding to the operation gestures may include only some distance sensors set in the target sensing area, or may include all distance sensors set in the target sensing area. It can be set according to actual needs without affecting the function of the method provided in the embodiment of the present invention.

[0086] On the other hand, it should be noted that, in a specific application process, the operation gestures that can be recognized by each sensing area in the interactive device may be different, that is, the operation gestures that can be recognized by the target sensing area may be only a part of all the operation gestures preset in the interactive device. The multiple operation gestures preset in the interactive device mentioned in the method provided in an embodiment of the present invention are the operation gestures that can be recognized by the target sensing area among all the operation gestures preset in the interactive device.

[0087] S104: determining a target operation gesture corresponding to the hand motion trajectory based on time series changes of a plurality of output signal values ​​generated by each distance sensor in each of the distance sensor sets;

[0088] In the method provided by the embodiment of the present invention, the matching of the hand motion trajectory with each operation gesture can be determined according to the change of the output signal value of each distance sensor in each distance sensor set, so as to determine the target operation gesture corresponding to the hand motion trajectory. Specifically, a gesture recognition algorithm corresponding to each operation gesture can be pre-set, and the input of the algorithm can be constructed based on the output signal value generated by each distance sensor in the distance sensor set corresponding to each operation gesture, and the gesture recognition algorithm corresponding to each operation gesture can be used to determine whether the operation gesture matches the hand motion trajectory.

[0089] S105: Determine a control instruction corresponding to the target operation gesture, and control a smart device associated with the interactive device according to the control instruction.

[0090] In the method provided in the embodiment of the present invention, the interactive device can establish a communication connection with the smart device associated with it in advance, and after determining the control instruction corresponding to the target operation gesture, send the control instruction to the corresponding smart device to control the smart device. In a specific application process, the interactive device can be associated with multiple smart devices, and the control instruction corresponding to each operation gesture can correspond to a specific smart device. Based on the control instruction corresponding to the target operation gesture, the corresponding smart device can be controlled.

[0091] Based on the method provided by the embodiment of the present invention, if the interactive device detects the hand motion trajectory generated by the user when in the awake state, the target operation gesture corresponding to the hand motion trajectory can be determined in the preset multiple operation gestures based on the time sequence changes of the multiple output signal values ​​generated by each distance sensor in the target sensing area on the interactive device, and the control instruction corresponding to the target operation gesture can be determined to control the smart device associated with the interactive device. Applying the method provided by the embodiment of the present invention, the user can trigger the interactive device to perform gesture recognition and determine the corresponding control instruction through non-contact hand movement to control the associated smart device. The user does not need to install the corresponding control software on a mobile terminal such as a mobile phone, nor does he need to perform cumbersome control operations through the mobile terminal, which can simplify the operation process of the user controlling the smart device, save the user's time and energy, and improve the control efficiency. For the user, the smart device can be controlled in a silent interactive mode without too much human and device participation, which is conducive to improving the user's experience of using the smart device.

[0092] Furthermore, the embodiment of the present invention provides another method for controlling an intelligent device. Figure 1 On the basis of the method shown, the method provided in the embodiment of the present invention further includes:

[0093] If the interactive device is not currently in a wake-up state, determining whether the hand motion trajectory matches a preset wake-up gesture;

[0094] In the method provided in the embodiment of the present invention, in order to better illustrate the method provided in the embodiment of the present invention, Figure 1 The target sensing area in the method shown is regarded as the first target sensing area. If, during the judgment process of step S101, it is determined that the interactive device is not currently in a wake-up state, then based on a preset gesture recognition algorithm, it is determined whether the hand motion trajectory matches the preset wake-up gesture. Specifically, a second target sensing area corresponding to the hand motion trajectory may be determined in multiple sensing areas of the interactive device. A distance sensor set corresponding to a preset wake-up gesture is determined, and the distance sensor set corresponding to the wake-up gesture includes multiple distance sensors for identifying the wake-up gesture in the second target sensing area. In the process of generating the hand motion trajectory, multiple output signal values ​​generated by each distance sensor in the distance sensor set corresponding to the wake-up gesture are obtained, and based on the timing changes of the multiple output signal values ​​generated by each distance sensor in the distance sensor set, it is determined whether the hand motion trajectory matches the preset wake-up gesture.

[0095] It should be noted that, in a specific application process, the second target sensing area and the first target sensing area may be the same sensing area or different sensing areas.

[0096] If the hand motion trajectory matches the wake-up gesture, the interactive device is switched from a current non-awakening state to a wake-up state.

[0097] In the method provided by the embodiment of the present invention, if the hand motion trajectory matches the preset wake-up gesture, the interactive device is switched from the current non-awakening state to the awakening state. If the hand motion trajectory does not match the wake-up gesture, the interactive device is kept in the non-awakening state without any response.

[0098] In a specific application process, a gesture with a low false detection rate can be used as a wake-up gesture, but the specific type of the wake-up gesture can also be set according to needs without affecting the function of the method provided in the embodiment of the present invention.

[0099] In order to better illustrate the state switching of interactive devices, such as Figure 3 As shown in the flowchart, in the method provided by the embodiment of the present invention, the general process of state switching is:

[0100] S201: When the interactive device is in a non-awakening state, a hand movement trajectory is detected;

[0101] S202: Determine whether to wake up the interactive device;

[0102] In the method provided by the embodiment of the present invention, it is determined whether to wake up the interactive device by determining whether the hand motion trajectory matches the preset wake-up gesture. If the current hand motion trajectory matches the wake-up gesture, it is determined to wake up the interactive device and enter step S203. If the current hand motion trajectory does not match the wake-up gesture, it is determined not to wake up the interactive device.

[0103] S203: Switching the interactive device from a current non-awakening state to an awakening state;

[0104] In the method provided in the embodiment of the present invention, the interactive device can be switched from the current non-awakening state to the awakening state by sending an enable signal for state switching, etc., and wait for the next hand motion trajectory generated by the user relative to the interactive device.

[0105] S204: No hand movement trajectory generated by the user is detected within a preset time period;

[0106] In the method provided in the embodiment of the present invention, the waiting time can be preset in the interactive device, for example, 30 seconds. If the interactive device is switched to the awake state, if within the preset time, the interactive device does not detect the hand motion trajectory of the user relative to the interactive device, then step S205 is entered.

[0107] S205: Switch the interactive device from the current awake state to a non-awakened state.

[0108] Based on the method provided by the embodiment of the present invention, the interactive device needs to wake up the interactive device only after the user makes a hand movement that matches the wake-up gesture, so that the interactive device is in the awake state when detecting the subsequent hand movement trajectory of the user, and then realizes the control of the smart device. When the interactive device is in the non-awakening state, it will not further determine the target operation gesture corresponding to the hand movement trajectory, nor will it control the smart device, which is conducive to reducing the misrecognition of the operation gesture and reducing the error rate of the operation.

[0109] Furthermore, in the method provided in the embodiment of the present invention, Figure 1 The process of determining whether the interactive device is currently in the awake state mentioned in step S101 of the method includes:

[0110] Determine an enable signal corresponding to the current interactive device;

[0111] In the method provided by the embodiment of the present invention, when the interactive device switches between the awake state and the non-awakened state, an enable signal for switching to a specified state may be triggered, such as triggering a certain pulse signal. The enable signal last triggered by the current interactive device may be obtained as its current corresponding enable signal.

[0112] Determining whether the enable signal is an enable signal that triggers the interactive device to be in a wake-up state;

[0113] If the enable signal is an enable signal that triggers the interactive device to be in the awake state, it is determined that the interactive device is currently in the awake state.

[0114] In the method provided by the embodiment of the present invention, whether the interactive device is currently in the awake state can be determined by judging whether the enable signal corresponding to the current interactive device is an enable signal indicating that the interactive device is switched to the awake state.

[0115] In order to better illustrate the method provided by the embodiment of the present invention, Figure 1 On the basis of the method shown, in the method provided by the embodiment of the present invention, the process of determining the target sensing area corresponding to the area where the hand motion trajectory is generated in the multiple sensing areas mentioned in step S102 includes:

[0116] Determine the detection signal value corresponding to each of the sensing areas;

[0117] In the method provided by the embodiment of the present invention, the detection signal value corresponding to the sensing area can be determined by the output signal values ​​of each distance sensor set in the sensing area. Specifically, the average of the output signal values ​​generated by each distance sensor in the sensing area during the hand motion trajectory generation process can be used as the detection signal value of the sensing area, the maximum value of the output signal values ​​generated by each distance sensor can be used as the detection signal value, and the maximum value of the average value of the output signal values ​​generated by each distance sensor can be used as the detection signal value. The larger the output signal value generated by the distance sensor, the smaller the distance between the measured object and the distance sensor.

[0118] A target detection signal value is determined among the detection signal values, wherein the target detection signal value is the detection signal value with the largest value among the detection signal values; and a sensing area corresponding to the target detection signal value is used as the target sensing area.

[0119] In the method provided by the embodiment of the present invention, each detection signal value is compared, and the detection signal value with the largest value is used as the target detection signal value, and the sensing area corresponding to the target detection signal value is used as the target sensing area. The corresponding detection signal value is the largest, indicating that the sensing area is the closest to the object being detected, that is, the user's hand movement trajectory should be generated relative to the sensing area.

[0120] It should be noted that the embodiment of the present invention provides only a specific embodiment for determining the target sensing area. In the specific implementation process, other methods can be used to determine the target sensing area without affecting the function of the method provided by the embodiment of the present invention. For example, the number of distance sensors whose output signal values ​​are greater than a threshold in each sensing area can also be determined, and the sensing area with the largest number is used as the target sensing area.

[0121] In order to better illustrate the method provided by the embodiment of the present invention, Figure 1 Based on the method shown, Figure 4 As shown in the flowchart, in the method provided by the embodiment of the present invention, the process of determining the target operation gesture corresponding to the hand motion trajectory based on the time series changes of the multiple output signal values ​​generated by each distance sensor in each distance sensor set mentioned in step S104 includes:

[0122] S301: for each of the operation gestures, construct a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture;

[0123] In the method provided by the embodiment of the present invention, the interactive device is pre-set with a preset construction rule corresponding to each operation gesture, that is, the construction method of the input matrix in the gesture recognition algorithm corresponding to the operation gesture, which can be set according to the requirements of the specific recognition algorithm. According to the preset construction rule corresponding to each operation gesture, based on the output signal values ​​generated by the corresponding distance sensors, the signal value matrix corresponding to each operation gesture can be constructed.

[0124] S302: Determine a convolution kernel set corresponding to each of the operation gestures, wherein the convolution kernel set corresponding to each of the operation gestures includes at least one preset convolution kernel;

[0125] In the method provided by the embodiment of the present invention, a convolution kernel corresponding to each operation gesture is pre-set for identifying the operation gesture, and each operation gesture corresponds to at least one convolution kernel, and the number of corresponding convolution kernels is determined by its recognition requirements. The convolution kernel corresponding to each operation gesture can be obtained by training based on the signal sample corresponding to the operation gesture. The convolution kernels included in each convolution kernel set are the convolution kernels corresponding to the corresponding operation gesture.

[0126] S303: performing a product operation on the signal value matrix corresponding to each operation gesture and each convolution kernel in the corresponding convolution kernel set to obtain an output matrix corresponding to each operation gesture;

[0127] In the method provided by the embodiment of the present invention, the signal value matrix corresponding to each operation gesture can be multiplied with each convolution kernel corresponding to it to obtain the output matrix corresponding to each operation gesture. For example, if the signal value matrix corresponding to the operation gesture is A, and its corresponding convolution kernel set contains only one convolution kernel B, then its corresponding output matrix C is A*B. If its corresponding convolution kernel set contains convolution kernel B and convolution kernel D, then the output matrix C is A*B*D.

[0128] S304: for each of the operation gestures, comparing the output matrix corresponding to the operation gesture with the preset matching matrix corresponding to the operation gesture, and if the output matrix corresponding to the operation gesture is the same as the preset matching matrix corresponding to the operation gesture, determining that the operation gesture matches the hand motion trajectory;

[0129] In the method provided by the embodiment of the present invention, a preset matching matrix corresponding to each operation gesture is pre-set in the interactive device, and the preset matching matrix corresponding to each operation gesture is set based on the output matrix that should be obtained after multiplying the signal value matrix obtained by each output signal value matching the operation gesture with the corresponding convolution kernel, that is, if the hand motion trajectory matches the operation gesture, then the product of the signal value matrix corresponding to each output signal value triggered by the hand motion trajectory and the corresponding convolution kernel is the same as the preset matching matrix. The output matrix corresponding to each operation gesture is compared with the corresponding preset matching matrix. If the two are the same, it is determined that the operation gesture matches the hand motion trajectory, otherwise it is not matched.

[0130] S305: If there is only one operation gesture among the operation gestures that matches the hand motion trajectory, then the operation gesture that matches the hand motion trajectory is used as the target operation gesture;

[0131] In the method provided by the embodiment of the present invention, the final gesture recognition result can be determined according to the matching results of each operation gesture and the hand motion trajectory. If there is only one operation gesture that matches the hand motion trajectory among the matching results of each operation gesture, the operation gesture that matches the hand motion trajectory is used as the target operation gesture. If no operation gesture matches the hand motion trajectory, or at least two operation gestures match the hand motion trajectory, it is considered that the matching fails, the hand motion trajectory does not have a corresponding target operation gesture, and the control process ends.

[0132] In the specific algorithm implementation process, the following formula principle can be used to output the target operation gesture:

[0133]

[0134] In this formula, o represents the output matching result, a1, a2 and a3 represent the matching results of three operation gestures respectively, when the operation gesture matches the hand motion trajectory, the matching result is a value of 1, when there is only one matching result representing a match, the operation gesture corresponding to the output matching result is the target operation gesture. When the output matching result is a value of 0, it represents that there is no matching target operation gesture. It should be noted that the above formula only includes the matching results of three operation gestures, which is only for better illustrating an embodiment provided by the method provided in an embodiment of the present invention, and does not limit the number of operation gestures in actual application.

[0135] Based on the method provided in the embodiment of the present invention, by setting the convolution kernel corresponding to each operation gesture, it is possible to identify whether the user's hand motion trajectory matches the operation gesture. Only when there is only one operation gesture that matches the hand motion trajectory can the target operation gesture be obtained, which is beneficial to reducing the recognition error rate.

[0136] In order to better illustrate the method provided by the embodiment of the present invention, the process of constructing the signal value matrix corresponding to the operation gesture mentioned in step S301 in the embodiment of the present invention is further illustrated below.

[0137] Optionally, in the method provided in the embodiment of the present invention, the process of constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture includes:

[0138] For each distance sensor in the distance sensor set corresponding to the operation gesture, based on the timing of the distance sensor generating multiple output signal values, among the multiple output signal values ​​generated by the distance sensor, determine multiple output signal values ​​corresponding to each preset time interval, and determine an interval signal value set corresponding to the distance sensor, the interval signal value set including the interval signal value corresponding to each preset time interval, each of the interval signal values ​​being an average value of the multiple output signal values ​​corresponding to the corresponding preset time interval;

[0139] Each interval signal value in the interval signal value set corresponding to each distance sensor in the distance sensor set corresponding to the operation gesture is used as a matrix element to construct a first matrix, and the first matrix is ​​used as the signal value matrix corresponding to the operation gesture.

[0140] In the method provided in the embodiment of the present invention, Figure 2 In the device structure shown, the interactive device is a cylindrical structure, and the target sensing area is the front of the interactive device. In the target sensing area, twelve distance sensors C0 to C11 are evenly distributed in a circle about the center point of the front of the device. Each distance sensor outputs a signal value at its own frequency, for example, outputs a signal value every 20ms. A 200ms time window can be used to intercept multiple output signal values, that is, ten output signal values ​​corresponding to the distance sensor are used to identify whether the corresponding motion trajectory matches a certain operation gesture, that is, a signal value matrix is ​​constructed based on the ten output signal values ​​corresponding to the distance sensor.

[0141] In the method provided by the embodiment of the present invention, for the operation gesture of sliding up and down, the signal value matrix is ​​constructed based on the principle of the above construction process. Specifically, Figure 2In the structure shown, for the upward or downward sliding gesture, the distance sensor C3 and the distance sensor C9 are ignored. That is, the distance sensor set corresponding to the operation gesture includes C0, C1, C2, C4, C5, C6, C7, C8, C10 and C11, which are the ten distance sensors.

[0142] In the method provided by the embodiment of the present invention, the time period of signal generation is divided into two time windows, that is, two preset time intervals are obtained. For the multiple output signal values ​​generated by each distance sensor, the output signal values ​​generated in each time window are determined according to the time sequence of generation, and the average value of each output signal value corresponding to each time window is calculated. That is, the average value of each output signal value corresponding to each time window of each distance sensor in C0, C1, C2, C4, C5, C6, C7, C8, C10 and C11 is calculated as the interval signal value corresponding to the distance sensor. Each interval signal value corresponding to each distance sensor is used as a matrix element, and the interval signal values ​​corresponding to the same time window are used as matrix elements in the same row. In the same row, the interval signal values ​​corresponding to each distance sensor are placed in the order of the positions of the distance sensors. In the same column, the interval signal values ​​corresponding to the same distance sensor are placed. For example, the matrix elements in the first row of the matrix are C10, C11, C0, C1, C2, C4, C5, C6, C7 and C8, which correspond to the interval signal values ​​of the first time window. The matrix elements in the second row are C10, C11, C0, C1, C2, C4, C5, C6, C7 and C8, which correspond to the interval signal values ​​of the second time window. Finally, a 2*10 signal value matrix is ​​obtained.

[0143] Correspondingly, in the method provided by the embodiment of the present invention, the convolution kernel set corresponding to the swipe up or swipe down operation gesture includes a pre-set convolution kernel K1, and the convolution kernel K1 is a 10*1 matrix. After the signal value matrix is ​​multiplied by the convolution kernel K1, a 2*1 output matrix can be obtained. In the method provided by the embodiment of the present invention, for the swipe up and swipe down operation gestures, the matrix [0,1] is pre-set as the preset matching matrix corresponding to the swipe down gesture, and the matrix [1,0] is pre-set as the preset matching matrix corresponding to the swipe up gesture. By comparing the output matrix with the matrix [0,1] or the matrix [1,0], it can be determined whether the swipe up gesture or the swipe down gesture matches the user's hand movement trajectory.

[0144] Furthermore, in the method provided in the embodiment of the present invention, the construction principle of the signal value matrix corresponding to the left-slide or right-slide operation gesture is the same as the construction principle of the signal value matrix corresponding to the above-mentioned upward and downward swipe gestures. Please refer to the above description and will not be repeated here. In the method provided in the embodiment of the present invention, for the recognition of the left-slide gesture and the right-slide gesture, the distance sensors at the upper and lower endpoints can be ignored, such as Figure 2 The structure shown, that is, ignoring the distance sensors C0 and C6, and using the output signal values ​​of C1, C2, C3, C4, C5, C7, C8, C9, C10 and C11 to construct the signal value matrix. Correspondingly, the setting of the convolution kernel corresponding to the left swipe gesture and the right swipe gesture, and the matching recognition process with the hand motion trajectory, the principle is the same as that of the upward swipe gesture and the downward swipe gesture, which can be referred to the above description and will not be repeated here.

[0145] Furthermore, an embodiment of the present invention provides another process for constructing a signal value matrix. In the method provided in the embodiment of the present invention, the process for constructing a signal value matrix corresponding to the operation gesture according to the timing of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and the preset construction rule corresponding to the operation gesture includes:

[0146] Using each output signal value generated by each distance sensor in the distance sensor set corresponding to the operation gesture as a matrix element, constructing a second matrix;

[0147] The preset matrix dimension corresponding to the operation gesture is determined, and the second matrix is ​​converted into a third matrix corresponding to the preset matrix dimension, and the third matrix is ​​used as the signal value matrix corresponding to the operation gesture.

[0148] In the method provided by an embodiment of the present invention, each output signal value of each distance sensor in the distance sensor set corresponding to the operation gesture can be used as a matrix element to construct a matrix, and the matrix dimension of the matrix is ​​converted, and the converted matrix is ​​used as the signal value matrix corresponding to the operation gesture.

[0149] In the method provided in the embodiment of the present invention, Figure 2 The device structure shown is taken as an example, and its specific structure description can refer to the description of the above embodiment. For operation gestures such as rotation gestures, a signal value matrix is ​​constructed by taking each output signal value as a matrix element. The rotation gesture includes a clockwise rotation gesture and a counterclockwise rotation gesture.

[0150] Specifically, each output signal value generated by C0 to C11 is used as a matrix element to obtain a matrix. In the matrix, the output signal values ​​of the same distance sensor are placed in the same row, and the distribution of column elements is determined according to the position order of the distance sensors to obtain a 12*10 matrix. For example, the matrix elements of the first row of the matrix are the output signal values ​​generated by C9 in sequence, and the matrix elements of the second row are the output signal values ​​generated by C10 in sequence. By analogy, the columns of the matrix are the output signal values ​​of C9, C10, C11, C0, C1, C2, C3, C4, C5, C6, C7 and C8 in sequence. Then, the matrix is ​​converted into a 2*20 matrix through the reshape function, and the converted matrix is ​​used as the signal value matrix corresponding to the rotation gesture. The reshape function is a function that transforms a specified matrix into a matrix of a specific dimension. It is an existing function and will not be described in detail here.

[0151] Correspondingly, in the method provided in the embodiment of the present invention, the convolution kernel set corresponding to the rotation gesture includes a pre-set convolution kernel K2 and a convolution kernel K3, where the convolution kernel K2 is a 20*10 matrix and the convolution kernel K3 is a 10*2 matrix. After multiplying the signal value matrix corresponding to the rotation gesture with the two convolution kernels at the same time, a 1*2 output matrix can be obtained. In the method provided in the embodiment of the present invention, for the rotation gesture, the matrix [0,1] is pre-set as the preset matching matrix corresponding to the clockwise rotation gesture, and the matrix [1,0] is pre-set as the preset matching matrix corresponding to the counterclockwise rotation gesture. The output matrix can be compared with the two preset matching matrices to determine whether the clockwise rotation gesture or the counterclockwise rotation gesture matches the hand motion trajectory.

[0152] It should be noted that the description of the specific operation gestures, the specific construction process of the signal value matrix and the specific settings of the convolution kernel in the above embodiments is only to better illustrate the specific embodiments provided by the method provided by the present invention. The matrix form of the signal value matrix is ​​actually determined according to the matrix form of the convolution kernel and the matrix form of the output matrix, so the corresponding construction method can be determined based on this. In the specific implementation process, the specific operation gestures, the corresponding signal value matrix construction method and the number of convolution kernels can all be set according to actual needs without affecting the function of the method provided in the embodiment of the present invention.

[0153] On the other hand, it should be noted that the above two descriptions of the construction process of the signal value matrix are respectively just descriptions of the construction principle of a preset construction rule, and the construction of the signal value matrix corresponding to each operation gesture is constructed according to its corresponding preset construction rule, that is, the construction principle of the signal value matrix of each operation gesture preset in the interactive device may be the same or different, and the corresponding matrix forms may be the same or different.

[0154] In order to better illustrate the method provided by the embodiment of the present invention, the setting process of the convolution kernel in the convolution kernel set corresponding to the operation gesture mentioned in the above embodiment is further described below, and the setting process includes:

[0155] Determine a plurality of signal samples corresponding to each of the operation gestures;

[0156] In the method provided in the embodiment of the present invention, multiple signal samples corresponding to each operation gesture can be obtained in advance through test experiments, that is, a hand movement matching the corresponding operation gesture is artificially made to trigger the corresponding distance sensor to generate an output signal value, so as to obtain the output signal values ​​generated by the corresponding distance sensors under the triggering of the hand movement, so as to obtain signal samples.

[0157] Determine each preset convolution matrix corresponding to each of the operation gestures;

[0158] In the method provided by the embodiment of the present invention, for each operation gesture, a convolution matrix can be pre-constructed, and the preset convolution matrix can be obtained by randomly generating. The number of preset convolution matrices corresponding to each operation gesture is the same as the number of convolution kernels that need to be set corresponding to the operation gesture, which can be one or more.

[0159] For each of the operation gestures, based on the preset matching matrix corresponding to the operation gesture, the signal samples corresponding to the operation gesture and the preset gradient descent algorithm, the elements in each preset convolution matrix corresponding to the operation gesture are updated, and each preset convolution matrix that has been updated is used as the convolution kernel corresponding to the operation gesture.

[0160] In the method provided in an embodiment of the present invention, the matrix elements in the preset convolution matrix can be updated through a preset matching matrix (that is, the ideal output), various signal samples and a gradient descent algorithm. This can be understood as training it through signal samples and using the trained matrix as the convolution kernel.

[0161] For example, in the matching calculation process of analogous operation gestures, in the process of updating only one convolution matrix, the calculation formula of the output matrix corresponding to the operation gesture is: O=I*K4, I is the input signal value matrix, and K4 is the preset convolution matrix. For this operation gesture, it has an ideal output, that is, the output that matches the operation gesture, that is, the preset matching matrix corresponding to the operation gesture. According to the gradient descent algorithm, when the input signal value matrix I and the output matrix O are given, a gradient will be generated, which can be used to update K4. Therefore, through multiple signal samples, K4 can be updated multiple times, and then a better value is obtained, and K4 that completes the update process is used as the convolution kernel. If multiple convolution kernels need to be set for an operation gesture, multiple preset convolution matrices need to be updated at the same time. At this time, the gradient feedback mechanism can be used.

[0162] Further, based on the method provided in the above embodiment, the embodiment of the present invention provides another control method of an intelligent device. In the method provided in the embodiment of the present invention, the shape of the interactive device is a cylindrical structure. When the target sensing area is the side surface of the interactive device, the multiple distance sensors in the target sensing area are evenly distributed in a circle in the target sensing area with respect to the center position point of the interactive device. The center position point is any point on the center line of the interactive device.

[0163] In the process of determining the target operation gesture corresponding to the hand motion trajectory, if the hand motion trajectory contains a first sub-trajectory, a second sub-trajectory and a third sub-trajectory generated in sequence, and the first sub-trajectory matches the first operation gesture, the second sub-trajectory matches the second operation gesture, and the third sub-trajectory matches the third operation gesture, then it is determined whether the movement direction corresponding to the second operation gesture and the movement direction corresponding to the first operation gesture are opposite directions to each other; the first operation gesture, the second operation gesture and the third operation gesture are respectively operation gestures among the multiple operation gestures;

[0164] If the movement direction corresponding to the second operation gesture and the movement direction corresponding to the first operation gesture are opposite to each other, determining whether the movement direction corresponding to the third operation gesture is the same as the movement direction corresponding to the first operation gesture;

[0165] In the method provided by the embodiment of the present invention, the gesture recognition process mentioned in the above-mentioned embodiments is mainly an explanation of the single matching and recognition of the operation gesture for the motion trajectory generated within a certain time interval. In the specific implementation process, a hand motion trajectory can trigger the matching and recognition process of the operation gestures in multiple time windows, that is, the motion trajectories in different time periods in the hand motion trajectory are matched and recognized with each operation gesture respectively. In the method provided by the embodiment of the present invention, if the hand motion trajectory contains three sub-trajectories, and these three sub-trajectories are respectively matched with an operation gesture corresponding to them, the operation gestures corresponding to each sub-trajectory are further recognized to filter out the return gestures in the hand motion trajectory. The return gesture refers to the operation gesture matched by some motion trajectories that are not intended to control the smart device due to motion restrictions during the user's hand movement.

[0166] like Figure 5 As shown in the figure, when the target sensing area is the side surface of the interactive device, the return gesture mainly refers to when the user uses the sensor on the side surface of the device to trigger control, such as rotating clockwise from the side. Due to the limited range of hand rotation, if the range of a single rotation cannot achieve the final control effect, the user's hand may need to rotate in the opposite direction first, that is, to produce a counterclockwise rotation movement, and then continue the previous clockwise rotation action. At this time, a counterclockwise rotation gesture that is not intended for control is generated, and this counterclockwise rotation gesture is considered a return gesture. Figure 5 As shown, the gesture return direction is the opposite direction of the gesture sliding direction.

[0167] In the method provided by the embodiment of the present invention, if the hand motion trajectory contains three sub-trajectories that match the first operation gesture, the second operation gesture and the third operation gesture respectively, then it is determined whether the motion direction corresponding to the second operation gesture is opposite to the motion direction of the first operation gesture, for example, clockwise rotation and counterclockwise rotation are opposite operation gestures. If opposite, it is further determined whether the motion directions corresponding to the third operation gesture and the first operation gesture are the same. The first operation gesture, the second operation gesture and the third operation gesture are all operation gestures among the multiple operation gestures pre-set in the interactive device, which can be any one of the multiple operation gestures. These three operation gestures can be the same or different, which is determined by the specific matching process.

[0168] If the movement direction corresponding to the third operation gesture is the same as the movement direction corresponding to the first operation gesture, determining the operation duration corresponding to the second operation gesture, and judging whether the operation duration is less than a preset threshold;

[0169] If the operation duration is less than the preset threshold, the first operation gesture and the third operation gesture are superimposed, and the superimposed operation gesture is used as the target operation gesture corresponding to the hand motion trajectory.

[0170] In the method provided by the embodiment of the present invention, if the third operation gesture and the first operation gesture correspond to the same direction of motion, it means that the second sub-trajectory in the hand motion trajectory may be a motion trajectory generated by the gesture return. At this time, the operation duration of the second operation gesture is determined, that is, the movement duration corresponding to the second sub-trajectory. If the operation duration is less than the preset threshold, the second operation gesture is considered to be a return gesture and is filtered out. The first operation gesture and the third operation gesture are superimposed to obtain the target operation gesture. It should be noted that the superposition of operation gestures is not the superposition of actual gestures, and the superposition does not generate a new operation gesture. For example, the first operation gesture and the third operation gesture are both clockwise rotation gestures, and the target operation gesture is still a clockwise rotation gesture. The so-called superposition of operation gestures refers to the superposition of the control effects of the corresponding control instructions, that is, the control effect generated by the control instruction corresponding to the final target operation gesture, which is the control effect obtained by superimposing the effects of the first operation gesture and the third operation gesture.

[0171] For example, when the light intensity of a smart lamp is adjusted, the user increases the light brightness by rotating the light clockwise. After the user rotates the light clockwise for the first time, the user is restricted in movement and rotates counterclockwise, and then continues to rotate the light clockwise for the second time. The control effect of the user's first clockwise rotation is to increase the light intensity by two levels, while the control effect of the second clockwise rotation is to increase the light intensity by one level. During the processing of the interactive device, the operation gesture corresponding to the user's counterclockwise rotation is filtered out, and the counterclockwise rotation gesture is not responded to. The clockwise rotation gestures corresponding to the two clockwise rotation movements are superimposed, and the matching target operation gesture is still a clockwise rotation gesture, but the control effect of the control instruction corresponding to the target operation gesture is the superposition effect of the two clockwise rotation gestures, that is, the control effect generated by the control instruction is to increase the light intensity by three levels.

[0172] In addition, in interactive devices Figure 5 In the case of the structure shown, the rotation gesture within a certain interval can also be regarded as a return gesture, and it can be set as an operation gesture. When the motion trajectory is identified to match the operation gesture representing the return gesture, it is filtered out and no response is given. In the actual application process, certain instructions can be provided to the user to remind the user to avoid starting the movement intended for control in the area used for return recognition to avoid mis-filtering. For example, the side area on the upper right part of the interactive device can be used as a filter for counterclockwise return motion, that is, Figure 5The sensing area D0 to D3 shown is used as a filter for counterclockwise return motion. According to the output signal values ​​of the distance sensors D0 to D3 in this area, the counterclockwise return gesture is recognized. The recognition principle is the same as that of the counterclockwise rotation gesture, except that not all distance sensors are used. Instead, the side area of ​​the lower right part of the interactive device is used as a filter for clockwise return motion, and the clockwise return gesture is recognized by each distance sensor in this area.

[0173] It should be noted that the description of the return gesture provided in the embodiment of the present invention is only for better illustrating the specific embodiment provided by the method provided by the present invention. In the specific application process, the operation gestures that can be filtered without response are not limited to rotation gestures, and can also be other sliding operation gestures. In the method provided in the embodiment of the present invention, the opposite direction of movement is used as a judgment condition for return filtering. In the actual application scenario, when the user controls through hand movement, the return motion trajectory generated by the user can be directly opposite to the original motion trajectory, such as a horizontal straight line to the left and a horizontal straight line to the right, or it can be opposite in direction, such as a horizontal straight line to the left and a direction with a certain angle with the horizontal direction to the right.

[0174] and Figure 1 Corresponding to the control method of the smart device shown in FIG. 1 , an embodiment of the present invention further provides a control device for a smart device, Figure 1 The specific implementation of the method shown in is shown in the structural diagram Figure 6 As shown, including:

[0175] The judging unit 401 is used to judge whether the interactive device is currently in an awake state when detecting that the user generates a hand motion trajectory relative to a preset interactive device; the interactive device is provided with a plurality of sensing areas, and each of the sensing areas is provided with a plurality of distance sensors according to the device form corresponding to the sensing area in the interactive device;

[0176] A first determining unit 402 is configured to determine, if the interactive device is currently in an awake state, a target sensing area corresponding to the area where the hand motion trajectory is generated among the multiple sensing areas, and obtain a plurality of output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory;

[0177] A second determining unit 403 is used to determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, each of the distance sensor sets including a plurality of distance sensors for identifying the corresponding operation gesture;

[0178] A third determining unit 404 is used to determine the target operation gesture corresponding to the hand motion trajectory based on the time series changes of multiple output signal values ​​generated by each distance sensor in each of the distance sensor sets;

[0179] The control unit 405 is used to determine the control instruction corresponding to the target operation gesture, and control the smart device associated with the interactive device according to the control instruction.

[0180] By using the device provided in the embodiment of the present invention, the user can trigger the interactive device to perform gesture recognition and determine the corresponding control instructions through non-contact hand movements, and control the associated smart device. The user does not need to install the corresponding control software on a mobile terminal such as a mobile phone, nor does he need to perform cumbersome control operations through the mobile terminal, which can simplify the operation process of the user controlling the smart device, save the user's time and energy, and improve the control efficiency. For the user, the smart device can be controlled in a silent interactive manner without too much human and device participation, which is conducive to improving the user's experience of using the smart device.

[0181] To better illustrate Figure 1 The present invention also provides an interactive device, the structure diagram of which is shown in FIG. Figure 7 As shown, the interactive device includes:

[0182] An interactive device body 501, wherein the interactive device body 501 is a cylindrical structure, and a front view surface of the interactive device body 501 is a circular surface;

[0183] A circular display screen 502 is provided on the front surface of the interactive device body 501, and the center point of the circular display screen 502 coincides with the center point of the front surface of the interactive device body 501;

[0184] A plurality of distance sensors 503 are arranged on the front surface of the interactive device body 501, and the distance sensors 503 arranged on the front surface of the interactive device body 501 are evenly distributed in a circle about the center point of the front surface of the interactive device body 501;

[0185] A plurality of indicator lights 504 are arranged on the front surface of the interactive device body 501, and the plurality of indicator lights 504 are evenly distributed in a circle about the center point of the front surface of the interactive device body 501;

[0186] A plurality of distance sensors 505 are arranged on the side of the interactive device body 501. The distance sensors arranged on the side of the interactive device body 501 are evenly distributed in a circle about a target position point. The target position point is any point on the center line of the interactive device body 501.

[0187] The interactive device body 501 has a built-in control chip, and each distance sensor provided on the front and side surfaces of the interactive device body 501 is connected to the control chip via a serial interface;

[0188] The interactive device body 501 has a built-in communication device.

[0189] It should be noted that Figure 7 The structural diagram of the interactive device shown is only a brief diagram. The solid circles evenly distributed in the circular surface in the figure represent the distance sensors set in the front view surface, the small circles represent the indicator lights, and the solid circles on the side represent the distance sensors set in the side surface. Figure 7 Only one distance sensor 503, indicator light 504 and distance sensor 505 shown in the figure are labeled, and not all device structures are labeled. Figure 7 The structure shown in the figure is only a schematic structure provided by the interactive device provided by the embodiment of the present invention to better illustrate. The number and specific setting positions of the components shown in the figure are only for schematic illustration and do not limit the number and position of components in actual application.

[0190] In the interactive device provided by the embodiment of the present invention, the specific type of the distance sensor may be an optical distance sensor, an infrared distance sensor or an ultrasonic distance sensor, etc. The circular display screen arranged on the front surface of the device is a borderless screen.

[0191] Furthermore, in the interactive device provided in the embodiment of the present invention, the communication device built into the interactive device body may be Bluetooth.

[0192] Furthermore, in the interactive device provided in the embodiment of the present invention, a speaker is built into the interactive device body.

[0193] Combination Figure 7 The interactive device shown in the figure will further explain the actual working status of the device during the use of the interactive device.

[0194] In the interactive device provided by the embodiment of the present invention, each distance sensor disposed on the front face thereof can be used to identify a rotation gesture. For example, if the smart device associated with the interactive device is a smart lamp, the rotation gesture can be used to adjust the light brightness of the smart lamp. Figure 8As shown in the schematic diagram of the device, when the interactive device recognizes a rotation gesture, each indicator light can be activated one by one along with the rotation gesture, that is, light up one by one, to provide a prompt. Furthermore, the interactive device can play corresponding sound effects, such as a sound effect similar to a rotary switch. Specifically, if the user's hand motion trajectory corresponds to a clockwise rotation gesture, then after the interactive device recognizes the operation gesture corresponding to the hand motion trajectory, it will trigger each indicator light on the interactive device, starting from the indicator light at the origin (the indicator light on the far left can be defined as the indicator light at the origin), and light up one by one clockwise along the user's hand motion trajectory. If the user's hand motion trajectory corresponds to a counterclockwise rotation gesture, then each indicator light is controlled to light up one by one counterclockwise along the hand motion trajectory, starting from the indicator light at the origin.

[0195] The distance sensors provided on the front face of the interactive device can be used to identify the up and down sliding gestures. If the smart device associated with the interactive device is a smart lamp, the upward sliding gesture can be used to turn on the light, and the downward sliding gesture can be used to turn off the light. When the interactive device determines that the user's hand movement trajectory corresponds to the upward sliding gesture, the indicator lights on the upper part of the interactive device can be controlled to turn on for prompting. If the downward sliding gesture is recognized, the indicator lights on the lower part are controlled to turn on.

[0196] The distance sensors set on the front face of the interactive device can also be used to identify left and right sliding gestures. If the smart device associated with the interactive device is a smart lamp, the left sliding and right sliding gestures can be used to turn on lights of different colors respectively. When the interactive device recognizes a left sliding gesture, the indicator lights on the left half of the interactive device can be controlled to light up, and when a right sliding gesture is recognized, the indicator lights on the right half can be controlled to light up.

[0197] The distance sensors set on the side of the interactive device can also be used to identify various operation gestures. When the user needs to view the display content of the display screen, he can make corresponding hand movements relative to the side to trigger the gesture recognition of the interactive device. In actual applications, the side recognition mainly recognizes rotation gestures. Using the distance sensors set on the side, the corresponding return gesture in the user's hand movement trajectory can be identified.

[0198] In actual use, users can set passwords for interactive devices. Fig. 9 As shown in the schematic diagram of the device, the password can be set using the distance sensors set on the front face of the interactive device. The user can click around the distance sensors and set the password by clicking on different sensors in order. In the specific setting process, when the user clicks around a distance sensor, the indicator light at the corresponding position will light up as a prompt.

[0199] Furthermore, the user can input corresponding characters to the interactive device through gestures. Fig.10 As shown in the schematic diagram of the device, the interactive device can display characters available for input to the user through the display screen, such as Fig.10 As shown, the display screen displays characters such as A, B, C, D, E, F, G, H, I, J and K, and the display position of each character can correspond to the position of each distance sensor and each indicator light. The user can select the input letter through the rotation gesture. When the user's hand movement trajectory triggers the selection of a certain letter, the display screen can highlight the letter, such as making the character larger, such as Fig.10 In the state shown, the character E is currently selected. At the same time, the indicator light at the corresponding position on the device will light up as a prompt. The user can confirm the selected character by pausing the action, etc., to achieve the purpose of inputting the character. At the same time, when the user selects the character through the rotation gesture, the interactive device can also play a sound effect similar to the rotation of the switch.

[0200] It should be noted that Figure 8 to Figure 10 The device interface shown in the figure is only a schematic structure provided to better illustrate the display effect of the interactive device in the embodiment of the present invention, and does not limit the surface color, light color and screen display color of the device.

[0201] An embodiment of the present invention further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the control method of the smart device as described above.

[0202] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0203] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0204] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A control method for an intelligent device, characterized in that: include: When detecting that the user generates a hand motion trajectory relative to a preset interactive device, determining whether the interactive device is currently in an awake state; The interactive device is provided with a plurality of sensing areas, and each of the sensing areas is provided with a plurality of distance sensors according to the device form corresponding to the sensing area in the interactive device; If the interactive device is currently in an awake state, then, among the multiple sensing areas, determining a target sensing area corresponding to the area where the hand motion trajectory is generated, and acquiring multiple output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory; Determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, each of the distance sensor sets including a plurality of distance sensors for identifying the corresponding operation gesture; For each of the operation gestures, constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture; Determine a convolution kernel set corresponding to each of the operation gestures, wherein the convolution kernel set corresponding to each of the operation gestures includes at least one preset convolution kernel; Performing a product operation on the signal value matrix corresponding to each operation gesture and each convolution kernel in the corresponding convolution kernel set to obtain an output matrix corresponding to each operation gesture; For each of the operation gestures, compare the output matrix corresponding to the operation gesture with the preset matching matrix corresponding to the operation gesture, and if the output matrix corresponding to the operation gesture is the same as the preset matching matrix corresponding to the operation gesture, determine that the operation gesture matches the hand motion trajectory; If there is only one operation gesture among the operation gestures that matches the hand motion trajectory, then the operation gesture that matches the hand motion trajectory is used as the target operation gesture; Determine a control instruction corresponding to the target operation gesture, and control the smart device associated with the interactive device according to the control instruction.

2. The method according to claim 1, characterized in that Also includes: If the interactive device is not currently in a wake-up state, determining whether the hand motion trajectory matches a preset wake-up gesture; If the hand motion trajectory matches the wake-up gesture, the interactive device is switched from a current non-awakening state to a wake-up state.

3. The method according to claim 1, characterized in that The determining whether the interactive device is currently in an awake state includes: Determine an enable signal corresponding to the current interactive device; Determining whether the enable signal is an enable signal that triggers the interactive device to be in a wake-up state; If the enable signal is an enable signal that triggers the interactive device to be in the awake state, it is determined that the interactive device is currently in the awake state.

4. The method according to claim 1, characterized in that Determining, among the multiple sensing areas, a target sensing area corresponding to the area where the hand motion trajectory is generated includes: Determine the detection signal value corresponding to each of the sensing areas; Determine a target detection signal value among the detection signal values, wherein the target detection signal value is the detection signal value with the largest value among the detection signal values; The sensing area corresponding to the target detection signal value is used as the target sensing area.

5. The method according to claim 1, characterized in that The constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and the preset construction rule corresponding to the operation gesture includes: For each distance sensor in the distance sensor set corresponding to the operation gesture, based on the timing of the distance sensor generating multiple output signal values, among the multiple output signal values ​​generated by the distance sensor, determine multiple output signal values ​​corresponding to each preset time interval, and determine an interval signal value set corresponding to the distance sensor, the interval signal value set including the interval signal value corresponding to each preset time interval, each of the interval signal values ​​being an average value of the multiple output signal values ​​corresponding to the corresponding preset time interval; Each interval signal value in the interval signal value set corresponding to each distance sensor in the distance sensor set corresponding to the operation gesture is used as a matrix element to construct a first matrix, and the first matrix is ​​used as the signal value matrix corresponding to the operation gesture.

6. The method according to claim 1, characterized in that The constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and the preset construction rule corresponding to the operation gesture includes: Using each output signal value generated by each distance sensor in the distance sensor set corresponding to the operation gesture as a matrix element, constructing a second matrix; The preset matrix dimension corresponding to the operation gesture is determined, and the second matrix is ​​converted into a third matrix corresponding to the preset matrix dimension, and the third matrix is ​​used as the signal value matrix corresponding to the operation gesture.

7. The method according to claim 1, characterized in that The process of setting each convolution kernel in the convolution kernel set corresponding to each operation gesture includes: Determine a plurality of signal samples corresponding to each of the operation gestures; Determine each preset convolution matrix corresponding to each of the operation gestures; For each of the operation gestures, based on the preset matching matrix corresponding to the operation gesture, the signal samples corresponding to the operation gesture and the preset gradient descent algorithm, the elements in each preset convolution matrix corresponding to the operation gesture are updated, and each preset convolution matrix that has been updated is used as the convolution kernel corresponding to the operation gesture.

8. The method according to claim 1, characterized in that The interactive device has a cylindrical structure. When the target sensing area is a side surface of the interactive device, a plurality of distance sensors in the target sensing area are evenly distributed in a circle in the target sensing area with respect to a center point of the interactive device. The center point is any point on the center line of the interactive device. In the process of determining the target operation gesture corresponding to the hand motion trajectory, if the hand motion trajectory contains a first sub-trajectory, a second sub-trajectory and a third sub-trajectory generated in sequence, and the first sub-trajectory matches the first operation gesture, the second sub-trajectory matches the second operation gesture, and the third sub-trajectory matches the third operation gesture, then it is determined whether the movement direction corresponding to the second operation gesture and the movement direction corresponding to the first operation gesture are opposite directions to each other; the first operation gesture, the second operation gesture and the third operation gesture are respectively operation gestures among the multiple operation gestures; If the movement direction corresponding to the second operation gesture and the movement direction corresponding to the first operation gesture are opposite to each other, determining whether the movement direction corresponding to the third operation gesture is the same as the movement direction corresponding to the first operation gesture; If the movement direction corresponding to the third operation gesture is the same as the movement direction corresponding to the first operation gesture, determining the operation duration corresponding to the second operation gesture, and judging whether the operation duration is less than a preset threshold; If the operation duration is less than the preset threshold, the first operation gesture and the third operation gesture are superimposed, and the superimposed operation gesture is used as the target operation gesture corresponding to the hand motion trajectory.

9. A control device for an intelligent device, characterized in that: include: A judging unit, configured to judge whether the interactive device is currently in a wake-up state when detecting that a user generates a hand motion trajectory relative to a preset interactive device; The interactive device is provided with a plurality of sensing areas, and each of the sensing areas is provided with a plurality of distance sensors according to the device form corresponding to the sensing area in the interactive device; A first determining unit is configured to determine, if the interactive device is currently in an awake state, a target sensing area corresponding to the area where the hand motion trajectory is generated among the multiple sensing areas, and obtain a plurality of output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory; A second determining unit, configured to determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, wherein each of the distance sensor sets includes a plurality of distance sensors for identifying the corresponding operation gesture; The third determining unit is used to construct, for each of the operation gestures, a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and the preset construction rule corresponding to the operation gesture; determine a convolution kernel set corresponding to each of the operation gestures, wherein the convolution kernel set corresponding to each of the operation gestures includes at least one preset convolution kernel; perform a product operation on the signal value matrix corresponding to each of the operation gestures and each of the convolution kernels in the corresponding convolution kernel set to obtain an output matrix corresponding to each of the operation gestures; for each of the operation gestures, compare the output matrix corresponding to the operation gesture with the preset matching matrix corresponding to the operation gesture; if the output matrix corresponding to the operation gesture is the same as the preset matching matrix corresponding to the operation gesture, then determine that the operation gesture matches the hand motion trajectory; If there is only one operation gesture among the operation gestures that matches the hand motion trajectory, then the operation gesture that matches the hand motion trajectory is used as the target operation gesture; The control unit is used to determine the control instruction corresponding to the target operation gesture, and control the smart device associated with the interactive device according to the control instruction.

10. An interactive device, characterized in that: include: An interactive device body, wherein the interactive device body is a cylindrical structure, and the front view surface of the interactive device body is a circular surface; A circular display screen is provided on the front surface of the interactive device body, and the center point of the circular display screen coincides with the center point of the front surface of the interactive device body; A plurality of distance sensors are arranged on the front surface of the interactive device body, and the distance sensors arranged on the front surface of the interactive device body are evenly distributed in a circle about the center point of the front surface of the interactive device body; A plurality of indicator lights are arranged on the front surface of the interactive device body, and the plurality of indicator lights are evenly distributed in a circle about the center point of the front surface of the interactive device body; A plurality of distance sensors are arranged on the side of the interactive device body, and the distance sensors arranged on the side of the interactive device body are evenly distributed in a circle about a target position point, and the target position point is any point on the center line of the interactive device body; The interactive device body has a built-in control chip, and each distance sensor provided on the front surface and the side surface of the interactive device body is connected to the control chip via a serial interface; The interactive device body has a built-in communication device; The control chip is used to execute the steps: When detecting that the user generates a hand motion trajectory relative to the interactive device, determining whether the interactive device is currently in an awake state; If the interactive device is currently in an awake state, then determining a target sensing area corresponding to the area where the hand motion trajectory is generated in the front view surface and the side view of the interactive device body, and obtaining a plurality of output signal values ​​generated by each distance sensor in the target sensing area during the generation of the hand motion trajectory; Determine a plurality of operation gestures preset in the interactive device, and determine a distance sensor set corresponding to each of the operation gestures in the target sensing area, each of the distance sensor sets including a plurality of distance sensors for identifying the corresponding operation gesture; For each of the operation gestures, constructing a signal value matrix corresponding to the operation gesture according to the time sequence of multiple output signal values ​​generated by each distance sensor in the corresponding distance sensor set and a preset construction rule corresponding to the operation gesture; Determine a convolution kernel set corresponding to each of the operation gestures, wherein the convolution kernel set corresponding to each of the operation gestures includes at least one preset convolution kernel; Performing a product operation on the signal value matrix corresponding to each operation gesture and each convolution kernel in the corresponding convolution kernel set to obtain an output matrix corresponding to each operation gesture; For each of the operation gestures, compare the output matrix corresponding to the operation gesture with the preset matching matrix corresponding to the operation gesture, and if the output matrix corresponding to the operation gesture is the same as the preset matching matrix corresponding to the operation gesture, determine that the operation gesture matches the hand motion trajectory; If there is only one operation gesture among the operation gestures that matches the hand motion trajectory, then the operation gesture that matches the hand motion trajectory is used as the target operation gesture; Determine a control instruction corresponding to the target operation gesture, and control the smart device associated with the interactive device according to the control instruction.

11. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the control method for the smart device according to any one of claims 1 to 8.

Images