Input method and system based on intelligent ring, intelligent ring and mobile equipment

[0022] The purpose of the present invention is to overcome the problems of inconsistent input methods, high learning cost, simple input interface, low interaction efficiency, and lack of intuitive input interface in the prior art, and propose an input method based on smart ring.

[0023] In the prior art, due to different input interfaces of different mobile devices, users need to learn different input methods, which increases the learning cost of users and also increases the complexity of user interaction. With the increase of mobile devices and the enhancement of functions, users will need to learn more and more input methods.

[0024] At the same time, due to constraints such as size and cost, most of the input interfaces on mobile devices are buttons, and the interaction channel is narrow, and complex interaction tasks cannot be completed with high efficiency.

[0025] For example, the existing VR and AR control handles need to use external positioning devices for 3D input, which has high requirements on the surrounding environment and cannot provide an instant interactive experience; 3D input through a two-dimensional interface such as a mobile phone touch screen is not intuitive and inefficient. .

[0026] In view of the above problems, the present invention proposes an input method based on a smart ring. figure 1 It is a flowchart of the smart ring-based input method of the present invention. Such as figure 1 As shown, the input method of the present invention includes:

[0027] Step S1, using the hand wearing the smart ring as the auxiliary hand and the other hand as the interactive hand, wear the smart ring on the index finger of the auxiliary hand; distinguish between the auxiliary hand and the interactive hand according to user habits, for example, left-handed users can hold the left hand It is set as an interactive hand, and the right hand is set as an auxiliary hand, while right-handed users are just the opposite; the present invention does not limit the use of two-handed operation, the smart ring can be fixed, and the left hand and/or the left hand can be moved by gestures of the left and/or right hands. Or the position of each part of the right hand, or any part of the body, such as the sole, elbow, knee, etc., for input operations;

[0028] Step S2, extend the auxiliary hand flat, and use the area between the thumb of the auxiliary hand and the index finger of the auxiliary hand as the interactive operation area; the auxiliary hand can be laid flat on a substantial plane so that the interactive operation area is a substantial two-dimensional plane, or A certain range of three-dimensional area above the actual plane; the auxiliary hand can also be suspended to make the interactive operation area a virtual two-dimensional plane in the auxiliary palm surface, or a certain range of three-dimensional area extended from the virtual two-dimensional plane, the range of the interactive operation area The interactive movement of the interactive hand can be recognized as the standard; in actual operation, the interactive operation surface can be positioned on the surface of the table and chair, wall, door and window and other physical objects, or the body surface such as thigh, abdomen, etc. to define a two-dimensional plane The substantial interactive operation area for open or secret interactive operations;

[0029] Step S3, the smart ring recognizes the interactive movement of the interactive hand in the interactive operation area; continuously collects the moving image of the interactive hand through the sensor (such as a camera) of the smart ring, and reconstructs the movement trajectory of the interactive hand based on the moving image to Obtain motion information; continuously collect motion images of one or more fingertips of the interactive hand through the camera, or continuously collect gesture motion images of the interactive hand, to obtain motion information of the interactive hand.

[0030] Step S4, extract the motion information of the interactive motion, and generate input information; the smart ring can transmit the motion image to the mobile terminal communicatively connected with the smart ring, and the mobile terminal extracts the motion information according to the motion image of the interactive hand, and generates the input information ; The smart ring can also extract motion information based on the motion image of the interactive hand, and generate input information and transmit it to the mobile terminal.

[0031] The sensor of the smart ring is a camera, and the types of the camera include: RGB camera, RGBD camera, infrared camera, thermal imaging camera; smart ring can also use sensors such as ultrasonic, radar and other sensors to obtain the movement information of the interactive hand. This is limited.

[0032] The present invention also provides an input system based on a smart ring, including: an input reference module for determining the range of motion of interactive actions; wherein the hand wearing the smart ring is used as the auxiliary hand, and the other hand is used as the interactive hand. The smart ring is worn on the index finger of the assisting hand; the assisting hand is stretched out, and the flat area between the thumb of the assisting hand and the index finger of the assisting hand is used as the interactive operation area; the interactive recognition module is used to identify the movement information of the interactive hand to generate input Information; among them, the smart ring recognizes the interactive motion of the interactive hand in the interactive operation area; extracts the motion information of the interactive motion and generates input information; the transmission module is used to transfer the motion information obtained by the smart ring or the input information generated by the smart ring, It is sent to the mobile terminal communicating with the smart ring.

[0033] In the input system of the present invention, the smart ring includes a camera, and the interactive recognition module includes: a collection module for continuously collecting moving images of the interactive hand through the camera; and a recognition module for moving the interactive hand based on the moving images Trajectory reconstruction to obtain motion information; the types of the camera include: RGB camera (camera for obtaining color information), RGBD camera (camera for obtaining depth information and color information), infrared camera, thermal imaging camera, etc.

[0034] The present invention also provides a smart ring, which is provided with a camera, and the smart ring is used for gesture input using the input method of the present invention.

[0035] The present invention also provides a mobile smart device, including: a smart ring; a mobile terminal, which is communicatively connected with the smart ring, and uses the smart ring as an input device.

[0036] In the smart ring-based input method of the present invention, the smart ring is worn on one hand (auxiliary hand), and a camera (including but not limited to RGB camera, RGBD camera, infrared, thermal imaging camera) is integrated in the smart ring to collect another Image information of a hand (interactive hand). The plane of the auxiliary hand is used as the reference plane, and the interaction area is specified by extending the thumb. The user can interact by tapping, moving his finger or other gestures in the interaction area. This method uses image information to control the current gesture, fingertip position, etc. This method transfers the input interface from the mobile device to the wearable device, regardless of the size of the mobile device , Cost and other conditions; support a variety of interactive methods, greatly improving input efficiency; it can also achieve instant and intuitive three-dimensional control to meet the input requirements of VR and AR glasses.

[0037] The input method of the present invention can pair with different devices and control these devices. In a complex intelligent environment, a harmonious and unified input method is used to reduce the cost of user interactive learning and improve user experience. In this method, a smart ring is worn on one hand (auxiliary hand), and a camera is integrated in the smart ring to collect image information of the other hand (interactive hand) to obtain movement information of the interactive hand. The mobile device can be a smart mobile device. In this way, the smart ring can only collect the motion information of the interactive hand and transmit the motion information to the smart mobile device. The smart mobile device analyzes the motion information of the interactive hand and obtains the input information. The advantage of this is that you can not do a lot of processing work on the smart ring, which reduces the performance requirements of the smart ring. The smart ring can also collect the movement information of the interactive hand and generate the input information, and the input information can be directly sent to Smart mobile devices, which reduce the performance requirements of smart mobile devices and improve the adaptability of smart rings; mobile devices can also be non-smart mobile devices, which requires the smart ring to independently collect interactive hand movement information and generate The operation of inputting information is not limited to this in the present invention.

[0038] In the input method of the present invention, the plane of the auxiliary hand is used as a reference plane, and the interaction area is designated by extending the thumb. Users can interact by tapping, moving their fingers or other gestures in the interactive area. This method transfers the input interface from the smart device end to the wearable device end through image information to the current gesture, fingertip position, etc. This method is not limited by the size and cost of the smart device.

[0039] An input method proposed by this method supports multiple interactive modes, greatly improves input efficiency, and can also realize instant and intuitive three-dimensional control, and meet the input requirements of VR and AR glasses. The input method of the present invention includes the following key points:

[0040] Key point 1, based on the input method of the smart ring, use the wearable ring for interaction.

[0041] Key point 2, the interactive mode using the assisting hand as the reference plane is not limited by the size and size of the device.

[0042] Key point 3, support a variety of interaction methods, through clicks, finger movements, gestures and other forms to carry out rich interaction.

[0043] Key point 4: Provide instant and intuitive three-dimensional control, which is conducive to interaction in the AR and VR environment.

[0044] The following examples illustrate the specific operation of the input method of the present invention.

[0045] Such as figure 2 As shown, the user can place the assistant hand on a flat surface for two-dimensional input, such as drawing, clicking, single-finger dragging, multi-finger dragging, etc., to simulate the input function of the touchpad.

[0046] Such as image 3 As shown, the user can hang the assisting hand to perform three-dimensional input. According to the position of the fingertip of the interactive hand in the image, the method judges its relative position in space with the reference plane, so as to complete the three-dimensional motion track recognition.

[0047] Such as Figure 4 As shown, this method can recognize the fingertip movement trajectory of the interactive hand finger (such as the index finger) through this process, and complete the corresponding command input

[0048] Such as Figure 5 As shown, this method can recognize the current gesture of the interactive hand through this process, and complete the corresponding command input

[0049] Image 6 It is a schematic diagram of the mobile device of the present invention. Such as Image 6 As shown, the embodiment of the present invention provides a mobile device, including a mobile terminal and a smart ring. The smart ring of the present invention is used as an input device of the mobile terminal. By recognizing the user's hand movement to input the mobile terminal, the smart ring passes Bluetooth, wifi, infrared, etc. communicate with the mobile terminal. Those of ordinary skill in the art can understand that all or part of the steps in the above method can be completed by a program instructing relevant hardware (such as a processor, FPGA, ASIC, etc.), and the program can be stored in a readable storage medium, such as a read-only memory. , Disk or CD, etc. All or part of the steps of the above-mentioned embodiments can also be implemented using one or more integrated circuits. Correspondingly, each module in the above-mentioned embodiment can be implemented in the form of hardware, for example, an integrated circuit to realize its corresponding function, or it can be implemented in the form of a software function module, for example, the program/instruction stored in the memory is executed by the processor. To achieve its corresponding functions. The embodiments of the present invention are not limited to the combination of hardware and software in any specific form.

[0050] Compared with the prior art, the smart ring-based input method provided by the present invention utilizes wearable devices, and interaction can be performed without the user holding peripherals or wearing complex settings, and the interaction is no longer limited by the volume and size of the device. . Using the camera, you can achieve a rich interactive form and expand the interactive channel. This method can provide users with intuitive interactive experience in a three-dimensional environment, which is conducive to its application in AR and VR.

[0051] The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all The equivalent technical solutions also belong to the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.