Interaction method and system of wearable device, wearable device and storage medium

Abstract

The invention provides an interaction method and an interaction system based on a wearable device, the wearable device and a storage medium. The method is applied to a first wearable device, the first wearable device is worn on the hand of a wearer and comprises at least one sensor, the first wearable device is connected with a second wearable device, and the method comprises the steps that hand motion information collected by the at least one sensor is obtained; determining coordinate change information of the hand of the wearer in a target space coordinate system based on the hand motion information; wherein the coordinate change information comprises a starting coordinate value and an ending coordinate value of the hand; determining hand action information of the wearer based on the starting coordinate value and the ending coordinate value; and sending the hand action information to the second wearable device, so that the second wearable device executes corresponding operation based on the hand action information. The invention aims to provide an interaction method based on a wearable device, and aims to identify the hand action information so as to execute the corresponding operation based on the hand action information.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an interaction method based on a wearable device, an interaction system based on a wearable device, a wearable device, and a computer-readable storage medium. Background Art

[0002] With the development of computer vision technology, the way of human-computer interaction has gradually changed from contact interaction to non-contact interaction. The current mainstream non-contact interaction includes voice input and hand motion recognition. Among them, hand motion recognition is the most natural and intuitive way.

[0003] In hand motion recognition, most of them use infrared LED, capacitive sensor or camera vision sensor to collect and recognize hand motion data. The above sensors are relatively simple and have low recognition rate, limited recognized motions, and are greatly affected by the environment. If the hand is blocked by other objects, it is basically impossible to recognize. Summary of the invention

[0004] The present application provides an interaction method based on a wearable device, an interaction system based on a wearable device, a wearable device and a computer-readable storage medium, aiming to realize the recognition of hand motion information and then perform corresponding operations based on the hand motion information.

[0005] To achieve the above object, the present application provides an interaction method based on a wearable device, which is applied to a first wearable device, wherein the first wearable device is worn on a wearer's hand and includes at least one sensor, and the first wearable device is connected to a second wearable device, and the method includes:

[0006] Acquiring hand movement information collected by the at least one sensor;

[0007] Based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand;

[0008] Determining hand motion information of the wearer based on the starting coordinate value and the ending coordinate value;

[0009] The hand motion information is sent to the second wearable device so that the second wearable device performs a corresponding operation based on the hand motion information.

[0010] To achieve the above-mentioned object, the present application also provides an interaction method based on a wearable device, which is applied to a second wearable device, the second wearable device is communicatively connected with a first wearable device, the first wearable device is worn on a wearer's hand, the first wearable device includes at least one sensor, and the method includes:

[0011] receiving hand movement information collected by the at least one sensor and sent by the first wearable device;

[0012] Based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand;

[0013] Determining hand motion information of the wearer based on the starting coordinate value and the ending coordinate value;

[0014] The hand motion information is sent to the second wearable device so that the second wearable device performs a corresponding operation based on the hand motion information.

[0015] To achieve the above objectives, the present application also provides an interactive system based on a wearable device, the interactive system based on a wearable device comprising:

[0016] A first wearable device, which is worn on a hand of a wearer and includes at least one sensor;

[0017] A second wearable device, the second wearable device is communicatively connected with the first wearable device, and is used for receiving hand motion information collected by the at least one sensor and sent by the first wearable device; based on the hand motion information, determining the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; based on the starting coordinate value and the ending coordinate value, determining the hand movement information of the wearer; and performing corresponding operations based on the hand movement information.

[0018] In addition, to achieve the above-mentioned purpose, the present application also provides a wearable device, which includes a memory and a processor; wherein the memory is connected to the processor for storing programs, and the processor is used to implement the steps of any one of the wearable device-based interaction methods provided in the embodiments of the present application by running the programs stored in the memory.

[0019] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor enables the processor to implement the steps of any one of the wearable device-based interaction methods provided in the embodiments of the present application.

[0020] The embodiment of the present application discloses an interactive method based on a wearable device, an interactive system based on a wearable device, a wearable device and a computer-readable storage medium, which can be applied to a first wearable device, which is worn on the hand of the wearer, includes a sensor, and the first wearable device is connected to the second wearable device. Specifically, the hand motion information collected by at least one sensor can be obtained, and then the coordinate change information of the wearer's hand in the target space coordinate system can be determined based on the hand motion information. Among them, the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand. Further, the hand motion information of the wearer can be determined based on the starting coordinate value and the ending coordinate value, and the hand motion information is sent to the second wearable device, so that the second wearable device performs the corresponding operation based on the hand motion information. Since the present application directly collects the hand motion information of the wearer through the sensor, it can avoid the influence of the external environment, thereby obtaining a more accurate recognition result, and then the corresponding operation can be performed based on the recognition result. In addition, the method proposed in the present application can identify the recognition result of the hand motion without complex computing power, and improves the recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 is a schematic diagram of steps of an interaction method based on a wearable device provided in an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a step of determining coordinate change information provided by an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of the steps of another interactive method based on a wearable device provided in an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of steps of another interaction method based on a wearable device provided in an embodiment of the present application;

[0026] Figure 5is a schematic block diagram of an interactive system based on a wearable device provided in an embodiment of the present application;

[0027] Figure 6 It is a schematic block diagram of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions. In addition, although the functional modules are divided in the device schematic, in some cases, the module division can be different from that in the device schematic.

[0030] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0031] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0032] See also Figure 1 , Figure 1 1 is a schematic diagram of the steps of an interactive method based on a wearable device provided in an embodiment of the present application. The interactive method based on a wearable device can be applied to a first wearable device, which is worn on the hand of a wearer and includes at least one sensor, and the first wearable device is connected to a second wearable device.

[0033] like Figure 1 As shown, the wearable device-based interaction method includes steps S11 to S14.

[0034] Step S11: Acquire hand movement information collected by at least one sensor.

[0035] Among them, the first wearable device can be a smart nail plate, a smart fingertip (such as a ring, a finger clip, etc.), a smart glove, or other device that can be worn on the wearer's hand. This application does not limit this, and takes the first wearable device as a smart nail plate as an example for explanation.

[0036] It should be noted that the present application does not limit the way in which the above-mentioned first wearable device is worn on the wearer's hand. For example, when the first wearable device is a smart nail plate, it can be attached to the wearer's bracket; when the first wearable device is a smart finger sleeve (such as a ring, a finger clip, etc.), it can be put on or clamped on the wearer's finger; when the first wearable device is a smart glove, it can be put on the wearer's hand.

[0037] Optionally, the number of smart nail pieces can be one or more. When the number of smart nail pieces is more than one, each smart nail piece can be connected to each other via wired or wireless means to collect the hand motion information of the wearer.

[0038] Furthermore, the second wearable device may be smart glasses, such as AR glasses, VR glasses, etc. This application does not limit this.

[0039] It should be noted that the wired connection between the above-mentioned smart nail pieces may include Ethernet connection, Universal Serial Bus (USB) connection, etc.; the wireless connection between the above-mentioned smart nail pieces may include Bluetooth connection, wireless LAN connection, long-distance radio connection, etc. The first wearable device and the second wearable device may be connected to each other by the above-mentioned wired or wireless method, and this application does not limit this.

[0040] This application is described with the number of smart nail pieces being one, that is, a smart nail piece is worn on one fingernail of the wearer. Furthermore, the smart nail piece includes a sensor, which may be an IMU (Inertial Measurement Unit) sensor, a position sensor, a GPS (Global Positioning System) sensor, etc., which is not limited in this application. Thus, the wearer's hand motion information can be collected based on at least one sensor on the smart nail piece.

[0041] In the embodiment of the present application, the wearer's hand movement information can be collected by the sensor on the smart nail piece, and then used to identify the wearer's hand movements.

[0042] Step S12: Based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand.

[0043] Among them, the target space coordinate system can be established based on the coordinate origin, and the coordinate origin can be the position of any finger wearing the smart nail piece when the wearer turns on the hand motion recognition instruction, or it can be the relative offset position of any finger wearing the smart nail piece. This application does not limit this.

[0044] Furthermore, when the wearer's hand moves, the coordinate change information of the wearer's hand in the target space coordinate system, that is, the starting coordinate value and the ending coordinate value of the hand, can be determined based on the hand's movement information. It can be understood that the starting coordinate value of the hand is obtained by mapping the position of the wearer's hand at the beginning of the movement to the target space coordinate system; the ending coordinate value of the hand is obtained by mapping the position of the wearer's hand at the end of the movement to the target space coordinate system.

[0045] In an embodiment of the present application, the coordinate change information of the wearer's hand in the target space coordinate system can be determined based on the wearer's hand motion information, so that the hand movement can be identified after the hand coordinate change information is determined.

[0046] Step S13: Determine the wearer's hand motion information based on the starting coordinate value and the ending coordinate value.

[0047] Specifically, after obtaining the starting coordinate value and the ending coordinate value of the wearer's hand, a corresponding algorithm may be used for recognition processing to thereby obtain the wearer's hand motion information.

[0048] It should be noted that the recognition of hand movements can be obtained based on computer vision and machine learning technology. This application is explained by taking the machine learning technology as an example, wherein the machine learning technology includes algorithms such as convolutional neural networks and recurrent neural networks. This application is explained by taking the algorithm for recognizing hand movements as a convolutional neural network as an example.

[0049] Convolutional Neural Network (CNN) is a feedforward neural network that uses convolution instead of general matrix multiplication in at least one layer. The neurons of the convolutional neural network can respond to other neurons within the receptive field, and perform well in tasks such as target detection, semantic segmentation, and action recognition. Therefore, in this application, the hand action recognition can be performed through the convolutional neural network.

[0050] In an embodiment of the present application, a convolutional neural network can be used to identify the wearer's hand movements based on the starting coordinate value and the ending coordinate value, thereby determining the wearer's hand movement information.

[0051] Step S14: Send the hand motion information to the second wearable device so that the second wearable device performs corresponding operations based on the hand motion information.

[0052] After obtaining the hand motion information, the hand motion information can be sent to the second wearable device, so that the second wearable device can perform corresponding operations based on the hand motion information by means of a gesture operation mapping table or conditional statements. For example, when the second wearable device determines that the user's hand motion is an action of the user tapping a virtual keyboard based on the hand motion information, the second wearable device can determine the key on the virtual keyboard tapped by the user based on the hand motion information, so as to display the text information, picture information, etc. corresponding to the key in its display interface.

[0053] In an embodiment of the present application, hand motion information can be sent to a second wearable device so that the second wearable device performs corresponding operations based on the hand motion information, thereby realizing intelligent interaction.

[0054] An interactive method based on a wearable device disclosed in an embodiment of the present application can be applied to a first wearable device, which is worn on the hand of the wearer and includes a sensor, and the first wearable device is connected to a second wearable device. Specifically, hand motion information collected by at least one sensor can be obtained, and then based on the hand motion information, the coordinate change information of the wearer's hand in the target space coordinate system can be determined. Among them, the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand. Further, the hand motion information of the wearer can be determined based on the starting coordinate value and the ending coordinate value, and the hand motion information is sent to the second wearable device, so that the second wearable device performs a corresponding operation based on the hand motion information. Since the present application directly collects the hand motion information of the wearer through the sensor, it can avoid the influence of the external environment, thereby obtaining a more accurate recognition result, and then the corresponding operation can be performed based on the recognition result. In addition, the method proposed in the present application can identify the recognition result of the hand motion without complex computing power, and improves the recognition efficiency.

[0055] Please continue reading Figure 2 , Figure 2 Schematic diagram of a step of determining coordinate change information provided by an embodiment of the present application. Figure 2 As shown, the hand motion information includes hand movement time information and hand movement speed information, and the coordinate change information can be determined through steps S121 to S122.

[0056] Step S121: Determine hand displacement information based on the hand movement time information and the hand movement speed information.

[0057] Step S122: Based on the hand displacement information, determine the coordinate change information of the wearer's hand in the target space coordinate system.

[0058] The hand displacement information includes the displacement of the hand on the x-axis, y-axis and z-axis of the target space coordinate system.

[0059] Specifically, after obtaining the wearer's hand movement time information and hand movement speed information through the sensor, it can be integrated to determine the hand displacement information. It can be understood that the hand displacement information includes the displacement of the wearer's hand in the target space coordinate system.

[0060] Furthermore, based on the hand displacement information, the coordinate change information of the wearer's hand in the target space coordinate system can be determined.

[0061] Optionally, the hand movement time information includes the time before movement and the time after movement; the hand movement speed information includes movement acceleration; based on the hand displacement information, the coordinate change information of the wearer's hand in the target space coordinate system is determined, including: integrating the movement acceleration from the time before movement to the time after movement, and obtaining the displacement of the hand in the target space coordinate system on the x-axis, y-axis and z-axis.

[0062] Specifically, the movement acceleration from the moment before the movement to the moment after the movement can be integrated respectively, and then the coordinate change information of the wearer's hand in the target space coordinate system can be obtained, that is, the displacement of the hand on the x-axis, y-axis and z-axis of the target space coordinate system.

[0063] Optionally, the movement acceleration includes x-axis acceleration, y-axis acceleration and z-axis acceleration, and the movement acceleration from the moment before the movement to the moment after the movement is integrated to obtain the displacement of the hand in the target space coordinate system on the x-axis, y-axis and z-axis, including: integrating the x-axis acceleration from the moment before the movement to the moment after the movement to obtain the x-axis displacement of the hand in the target space coordinate system; integrating the y-axis acceleration from the moment before the movement to the moment after the movement to obtain the y-axis displacement of the hand in the target space coordinate system; integrating the z-axis acceleration from the moment before the movement to the moment after the movement to obtain the z-axis displacement of the hand in the target space coordinate system.

[0064] Specifically, the following formula can be used to express the integral operation of the movement acceleration from the time before the movement to the time after the movement, and the displacement of the hand on the x-axis, y-axis and z-axis in the target space coordinate system can be obtained:

[0065]

[0066] Among them, s is the x-axis displacement, y-axis displacement or z-axis displacement of the hand in the target space coordinate system; t1 is the time before movement; t2 is the time after movement; t is the movement time; a is the x-axis acceleration, or the y-axis acceleration, or the z-axis acceleration.

[0067] In the embodiment of the present application, the wearer's hand movement time information and hand movement speed information can be integrated to obtain the wearer's hand displacement information. Based on the hand displacement information, the coordinate change information of the wearer's hand in the target space coordinate system can be determined, that is, the displacement of the hand in the x-axis, y-axis and z-axis of the target space coordinate system.

[0068] Optionally, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of another interactive method based on a wearable device provided in an embodiment of the present application. Figure 3 As shown, the first wearable device includes at least two sensors, each sensor is set corresponding to a finger, and the recognition of hand movements can be achieved through steps S21 to S23.

[0069] Step S21: Acquire hand movement information collected by at least two sensors.

[0070] Step S22: Based on at least two pieces of hand motion information, determine the coordinate change information of at least two fingers of the wearer in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of each finger.

[0071] Step S23: Determine the hand motion information of the wearer based on the starting coordinate values ​​and the ending coordinate values ​​of at least two fingers.

[0072] Step S23: Send the hand motion information to the second wearable device so that the second wearable device performs corresponding operations based on the hand motion information.

[0073] It is understandable that the number of the first wearable device can be multiple, that is, the number of corresponding sensors is multiple. The embodiment of the present application is described by taking the number of sensors as 2 as an example, wherein each sensor is set corresponding to one finger.

[0074] Specifically, the hand motion information of the wearer can be collected through two sensors, and then the coordinate change information of the two fingers of the wearer in the target space coordinate system can be determined, and the hand motion information of the wearer can be determined based on the starting coordinate value and the ending coordinate value of the two fingers, and then the hand motion information can be sent to the second wearable device, so that the second wearable device can perform corresponding operations based on the hand motion information through gesture operation mapping table or conditional statements.

[0075] It should be noted that, for the above-mentioned specific implementation method, please refer to step S11 to step S14. To avoid repetition, this application will not elaborate on it.

[0076] Furthermore, in the process of identifying the above two fingers, the coordination relationship between the two fingers can be taken into consideration, and then the hand movements of the wearer, such as snapping fingers, can be identified.

[0077] In the embodiment of the present application, at least two sensors can be used to collect the wearer's hand movement information, and then obtain the coordinate change information of at least two fingers in the target space coordinate system to determine the wearer's hand movement. Since the coordination relationship between the fingers is taken into account, the wearer's hand movement information can be determined more accurately.

[0078] Optionally, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of another step of an interaction method based on a wearable device provided in an embodiment of the present application. Figure 4 As shown, the method can be applied to a second wearable device, the second wearable device is communicatively connected to a first wearable device, the first wearable device is worn on a hand of a wearer, and the first wearable device includes at least one sensor. The recognition of hand movements can be achieved through steps S31 to S33.

[0079] Step S31: receiving hand movement information collected by at least one sensor from a first wearable device.

[0080] Step S32: Based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand.

[0081] Step S33: Determine the hand movement of the wearer based on the starting coordinate value and the ending coordinate value.

[0082] Step S34: Execute corresponding operations based on the hand motion information.

[0083] The second wearable device may be smart glasses, such as AR glasses, VR glasses, etc. Furthermore, the first wearable device and the second wearable device may be connected to each other by wired or wireless communication, which is not limited in the present application.

[0084] Specifically, the smart glasses can receive the hand motion information collected by the smart nail piece based on at least one sensor, and then determine the coordinate change information of the wearer's hand in the target space coordinate system, that is, the starting coordinate value and the ending coordinate value of the hand based on the hand motion information. Based on the starting coordinate value and the ending coordinate value, the wearer's hand motion information is obtained through corresponding algorithm recognition, and then the corresponding operation based on the hand motion information can be performed through gesture operation mapping tables or conditional statements.

[0085] It should be noted that, in the embodiment of the present application, please refer to step S12 for establishing the target space coordinate system and determining the coordinate change information. To avoid repetition, they are not described here.

[0086] Optionally, based on the starting coordinate value and the ending coordinate value, the wearer's hand movement is determined, including: mapping the ending coordinate value to a virtual keyboard to obtain a target key corresponding to the ending coordinate value; wherein the virtual keyboard is constructed based on the target space coordinate system and the initial hand position; based on the z-axis coordinate values ​​in the starting coordinate value and the ending coordinate value, the wearer's displacement change in the z-axis direction is determined; when the displacement change in the z-axis direction is less than a preset threshold, the wearer's touch action on the target key is determined; wherein the coordinate direction of the z-axis is perpendicular to the virtual keyboard and downward.

[0087] Among them, the virtual keyboard can be 9-key Pinyin, 9-key English, full English key, full Pinyin key, etc. The specific mode can be switched through smart glasses, and this application does not limit this.

[0088] Specifically, the initial hand position can be used as the coordinate origin to establish a two-dimensional coordinate system. In this way, the smart glasses can use the display page to build a two-dimensional virtual keyboard based on the two-dimensional coordinate system, wherein the virtual keyboard corresponds to the x-axis and y-axis of the target space coordinate system in the horizontal direction.

[0089] Furthermore, the ending coordinate value can be mapped to the virtual keyboard to determine the target key corresponding to the ending coordinate value. The z-axis coordinate value in the starting coordinate value and the z-axis coordinate value in the ending coordinate value are subtracted to obtain the displacement change of the wearer in the z-axis direction; when the displacement change in the z-axis direction is less than a preset threshold, the touch action of the wearer on the target key can be determined.

[0090] It should be noted that the preset threshold is not limited in the present application, for example, it can be 2mm, 3mm, 4mm, etc. It can be understood that when the displacement change is less than the preset threshold, it means that the key is pressed, and the touch action of the wearer on the target key can be determined.

[0091] In the embodiment of the present application, after receiving the hand movement information of the wearer sent by the smart nail piece, the smart glasses can map the hand movement information to the keyboard, and then determine the key touch action of the wearer based on the coordinate direction of the z-axis of the wearer's hand in the end coordinate value. Since the mapping of the virtual keyboard is introduced in the recognition process, the recognition result is more accurate and the recognition efficiency is improved.

[0092] Figure 5 is a schematic block diagram of an interactive system based on a wearable device provided in an embodiment of the present application. Figure 5As shown, the interactive system includes: a first wearable device and a second wearable device.

[0093] A first wearable device, the first wearable device is worn on the hand of a wearer, and includes at least one sensor; a second wearable device, the second wearable device is communicatively connected to the first wearable device, and is used to receive hand motion information sent by the first wearable device and collected by at least one sensor; based on the hand motion information, the coordinate change information of the wearer's hand in the target space coordinate system is determined; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; based on the starting coordinate value and the ending coordinate value, the hand motion information of the wearer is determined; and corresponding operations are performed based on the hand motion information.

[0094] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0095] The method and apparatus of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0096] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in Figure 6 The wearable device shown is running on the

[0097] See also Figure 6 , Figure 6 1 is a schematic block diagram of a wearable device provided in an embodiment of the present application. The wearable device may be a server.

[0098] like Figure 6 As shown, the wearable device 400 includes a processor 401, a memory 402, a sensor 403 and a communication module 404 connected through a system bus, wherein the memory 402 may include a volatile storage medium, a non-volatile storage medium and an internal memory, and the communication module 404 includes a wireless communication method or a wired communication method, which is not limited in the present application.

[0099] The processor 401 is used to provide computing and control capabilities to support the operation of the entire wearable device 400.

[0100] The memory 402 provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor 401 can execute any interaction method based on the wearable device.

[0101] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that the structure of the wearable device 400 is only a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation on the wearable device 400 to which the present application scheme is applied. The specific wearable device 400 may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0102] It should be understood that the processor 401 may be a central processing unit (CPU), and the processor 401 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0103] In some embodiments, the processor 401 is used to run a computer program stored in the memory 402 to implement the following steps: obtaining hand motion information collected by the at least one sensor; determining the coordinate change information of the wearer's hand in the target space coordinate system based on the hand motion information; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; determining the hand motion information of the wearer based on the starting coordinate value and the ending coordinate value; and sending the hand motion information to the second wearable device so that the second wearable device performs a corresponding operation based on the hand motion information.

[0104] In some embodiments, the processor 401 is also used to determine hand displacement information based on the hand movement time information and the hand movement speed information; based on the hand displacement information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the hand displacement information includes the displacement of the hand in the target space coordinate system on the x-axis, y-axis and z-axis.

[0105] In some embodiments, the processor 401 is further configured to integrate the movement acceleration from the time before the movement to the time after the movement to obtain the displacement of the hand on the x-axis, y-axis, and z-axis in the target space coordinate system.

[0106] In some embodiments, the processor 401 is also used to determine the hand position information of the person wearing the first wearable device in response to a start-up instruction of the first wearable device; wherein the hand position information includes finger position information; the position corresponding to the finger position information or the position corresponding to a preset offset of the finger position information is determined as the coordinate origin; and based on the coordinate origin, the target space coordinate system is constructed.

[0107] In some embodiments, the processor 401 is also used to obtain hand motion information collected by the at least two sensors; based on at least two of the hand motion information, determine the coordinate change information of at least two fingers of the wearer in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of each of the fingers; based on the starting coordinate value and the ending coordinate value of at least two fingers, determine the hand motion information of the wearer; and send the hand motion information to the second wearable device so that the second wearable device performs a corresponding operation based on the hand motion information.

[0108] In some embodiments, the processor 401 is also used to receive hand motion information collected by the at least one sensor and sent by the first wearable device; based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; based on the starting coordinate value and the ending coordinate value, determine the hand movement information of the wearer; and perform corresponding operations based on the hand movement information.

[0109] In some embodiments, the processor 401 is further used to map the end coordinate value to a virtual keyboard to obtain a target key corresponding to the end coordinate value; wherein the virtual keyboard is constructed based on the target space coordinate system and the initial hand position; based on the z-axis coordinate values ​​in the starting coordinate value and the end coordinate value, the displacement change of the wearer in the z-axis direction is determined; when the displacement change in the z-axis direction is less than a preset threshold, the touch action of the wearer on the target key is determined; wherein the coordinate direction of the z-axis is perpendicular to the virtual keyboard and downward.

[0110] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any one of the wearable device-based interaction methods provided in the embodiments of the present application is implemented.

[0111] The computer-readable storage medium may be an internal storage unit of the smart glasses described in the foregoing embodiments, such as a hard disk or memory of the smart glasses. The computer-readable storage medium may also be an external storage device of the smart glasses, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the smart glasses.

[0112] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like.

[0113] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An interactive method based on a wearable device, characterized in that: Applied to a first wearable device, the first wearable device is worn on a wearer's hand and includes at least one sensor, the first wearable device is connected to a second wearable device, and the method includes: Acquiring hand movement information collected by the at least one sensor; Based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; Determining hand motion information of the wearer based on the starting coordinate value and the ending coordinate value; The hand motion information is sent to the second wearable device so that the second wearable device performs a corresponding operation based on the hand motion information.

2. The method according to claim 1, characterized in that The hand motion information includes hand movement time information and hand movement speed information; and determining the coordinate change information of the wearer's hand in the target space coordinate system based on the hand motion information includes: Determining hand displacement information based on the hand movement time information and the hand movement speed information; Based on the hand displacement information, the coordinate change information of the wearer's hand in the target space coordinate system is determined; wherein the hand displacement information includes the displacement of the hand in the x-axis, y-axis and z-axis of the target space coordinate system.

3. The method according to claim 2, characterized in that The hand movement time information includes a time before the movement and a time after the movement; the hand movement speed information includes movement acceleration; and determining the hand displacement information based on the hand movement time information and the hand movement speed information includes: The movement acceleration from the time before the movement to the time after the movement is integrated to obtain the displacement of the hand on the x-axis, y-axis and z-axis in the target space coordinate system.

4. The method according to claim 1, characterized in that The method further comprises: In response to a start instruction of the first wearable device, determining the position information of a hand wearing the first wearable device; wherein the hand position information includes finger position information; Determine a position corresponding to the finger position information or a position corresponding to a preset offset of the finger position information as a coordinate origin; Based on the coordinate origin, the target space coordinate system is constructed.

5. The method according to claim 1, characterized in that The first wearable device includes at least two sensors, each of which is arranged corresponding to a finger, and the method further includes: Acquiring hand movement information collected by the at least two sensors; Based on at least two of the hand motion information, determine the coordinate change information of at least two fingers of the wearer in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of each of the fingers; Determining hand motion information of the wearer based on the starting coordinate values ​​and the ending coordinate values ​​of at least two fingers; The hand motion information is sent to the second wearable device so that the second wearable device performs a corresponding operation based on the hand motion information.

6. An interactive method based on a wearable device, characterized in that: Applied to a second wearable device, the second wearable device is communicatively connected to a first wearable device, the first wearable device is worn on a hand of a wearer, the first wearable device includes at least one sensor, and the method includes: receiving hand movement information collected by the at least one sensor and sent by the first wearable device; Based on the hand motion information, determine the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; Determining a hand motion of the wearer based on the starting coordinate value and the ending coordinate value; A corresponding operation is performed based on the hand motion information.

7. The method according to claim 6, characterized in that The determining the hand motion of the wearer based on the starting coordinate value and the ending coordinate value includes: Mapping the end coordinate value to a virtual keyboard to obtain a target key corresponding to the end coordinate value; wherein the virtual keyboard is constructed based on the target space coordinate system and the initial hand position; Determine the displacement change of the wearer in the z-axis direction based on the z-axis coordinate values ​​in the starting coordinate value and the ending coordinate value; When the displacement change in the z-axis direction is less than a preset threshold, the touch action of the wearer on the target key is determined; wherein the coordinate direction of the z-axis is perpendicular to the virtual keyboard and downward.

8. An interactive system based on a wearable device, characterized in that: The interactive system comprises: A first wearable device, which is worn on a hand of a wearer and includes at least one sensor; A second wearable device, the second wearable device is communicatively connected with the first wearable device, and is used for receiving hand motion information collected by the at least one sensor and sent by the first wearable device; based on the hand motion information, determining the coordinate change information of the wearer's hand in the target space coordinate system; wherein the coordinate change information includes the starting coordinate value and the ending coordinate value of the hand; based on the starting coordinate value and the ending coordinate value, determining the hand movement information of the wearer; and performing corresponding operations based on the hand movement information.

9. A wearable device, characterized in that: include: A memory and a processor; wherein the memory is connected to the processor for storing a program; the processor is used to implement the steps of the wearable device-based interaction method as described in any one of claims 1 to 7 by running the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the wearable device-based interaction method as described in any one of claims 1-7.

Images