Method for controlling operation of smart glasses by finger gestures and system therefor
By connecting a camera to a smartwatch and setting tattoo sticker features on fingers and palms, the YOLOv5 algorithm is used to recognize gestures, solving the problem of inconvenient operation of handheld devices in existing technologies. This enables portable smart glasses control and improves the convenience and accuracy of operation.
Patent Information
- Application Number
- CN202211406286.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-10
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2042-11-10
AI Technical Summary
Existing smart glasses operation methods require handheld devices for gesture recognition, which is inconvenient for the hands and requires high camera resolution, making portable operation impossible.
A camera is connected to a smartwatch, fixed to a ring, and tattoo sticker features are set on the knuckles and palms. Gestures are recognized using the YOLOv5 algorithm, and the recognized features are converted into keyboard or mouse button events, which are then sent to smart glasses via Bluetooth for operation.
It enables portable control of smart glasses via finger gestures, freeing up hands and improving the convenience and precision of operation.
Smart Images

Figure CN115629669B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of intelligent device control, in particular to a method for controlling operation of intelligent glasses by finger gestures and a system thereof. BACKGROUND
[0002] Intelligent glasses, also known as smart glasses, refer to a device having an independent operating system like a smart phone, and the user can install software and programs provided by software service providers on the intelligent glasses. The intelligent glasses project an interface of an app in front of the glasses. Currently, the intelligent glasses recognize gestures through a camera on the glasses, and the user needs to put his hand in front of the camera to operate. The hand will be sore after a long time of operation. In addition, the distance between the glasses and the hand is relatively far, and the resolution of the camera is required to be high.
[0003] The prior art discloses a "direction gesture input method and device for intelligent television or intelligent glasses", referring to Chinese patent with publication number "CN105700806A", which discloses a direction gesture input method and device for intelligent television or intelligent glasses. The method uses a direction gesture made by a user's finger on a touch screen of a handheld intelligent terminal to operate a nine-square keyboard arranged on a screen of the intelligent television or the intelligent glasses. The user can conveniently and quickly input on the touch screen like using a cross-screen input method, and the user's line of sight does not leave the screen of the intelligent television or the intelligent glasses, so that the touch blind operation of character input and function calling is realized conveniently, quickly and accurately. The device includes two parts of hardware and software. The hardware includes a touch-type handheld intelligent terminal and an intelligent television or an intelligent glasses, and the two parts exchange data in real time through a transmission unit. The software includes a keyboard display module, a data acquisition module, a gesture analysis module, a function acquisition module, a character acquisition module, a function calling module and a character input module. However, the patent uses a touch handheld device, and the user needs to hold a touch panel similar to a mobile phone to touch on the touch panel to realize nine-square input. In the present application, the user does not need to hold anything, and only needs to wear a ring on the thumb. The ring can be recognized by a camera on the ring, and the user can do other things without putting down the handheld device. The technical solution of the present application is different from that of the present application. SUMMARY
[0004] To overcome the above problems, the present application aims to provide a method for controlling operation of intelligent glasses by finger gestures, which can free the hands and control the intelligent glasses in a portable manner.
[0005] The present application adopts the following solution: a method for controlling operation of intelligent glasses by finger gestures, the method comprising the following steps:
[0006] Step S1: Connect a camera to a smartwatch and fix the camera to the ring. Set the corresponding tattoo sticker features on the finger joints and palms.
[0007] Step S2: Run an image processing program on the smartwatch and use the camera and YOLOv5 algorithm to identify the features of the tattoo stickers on the finger joints or palms.
[0008] Step S3: The identified features of the tattoo stickers on the finger joints and palms are distinguished into corresponding keyboard or mouse button events and transmitted to the smartwatch. The smartwatch then sends the data to the smart glasses via Bluetooth.
[0009] Step S4: The smart glasses receive the event, convert it into the corresponding button in the smart glasses app, and execute the button operation.
[0010] Furthermore, in step S3, the identification of the fingerprint joints and hand tattoo sticker features as corresponding mouse button events and their transmission to the smartwatch specifically involves: using mouse mode mapping, i.e., setting a numbered table on the tattoo sticker on the palm, identifying the table content to determine the palm coordinates, obtaining the movement coordinates, and sending the movement coordinates to the smart glasses. The smart glasses then control the mouse pointer within the glasses to move the corresponding pixels according to the movement coordinates. The palm table area is mapped to the screen in the smart glasses; that is, the screen pixels are equally divided according to the table, and the divided pixels are mapped to the numbers in the table. The mouse then moves according to the midpoint of the number captured by the camera, thus obtaining the corresponding coordinates.
[0011] Furthermore, in step S3, the identified tattoo sticker features on the finger joints and palm are distinguished into corresponding keyboard key events and transmitted to the smartwatch. Specifically, tattoo sticker feature marks are applied to the three joints and metacarpophalangeal joints of the index, middle, and ring fingers, respectively. The tattoo sticker feature marks of the three joints and metacarpophalangeal joints of the index, middle, and ring fingers are shown in the table below:
[0012]
[0013]
[0014] This simulates a 9-key Pinyin input method, where A1 represents 1, A2 represents 2abc, A3 represents 3def, A4 is empty, A5 represents 4ghi, A6 represents 5jkl, A7 represents 6mno, A8 is empty, A9 represents pqrs, AA represents 8tuv, AB represents 9wxyz, AC is empty, AD (little finger knuckle) represents switching between number and Pinyin input methods, AE (ring finger knuckle) represents 0, AF (middle finger knuckle) represents switching between candidate characters, and AG (index finger knuckle) represents confirming candidate characters. Users select the corresponding marked letter in the table to choose either Pinyin or a word.
[0015] Further, when inputting, the smart glasses project the interface of the app in front of the glasses, and the screen interface displays several pinyin candidate combinations inputted at the bottom, and the candidate pinyin or candidate words are selected by clicking the corresponding knuckles, and then inputted into the smart watch, and the smart watch sends to the smart glasses through Bluetooth.
[0016] The application further provides a system for controlling the operation of smart glasses through finger gestures, which comprises an installation setting module, an identification module, a data transmission module, and an execution module.
[0017] The installation setting module is connected with a camera on the smart watch, and the camera is fixed on a ring, and corresponding tattoo sticker features are set on the knuckles of fingers and the palm.
[0018] The identification module runs an image program on the smart watch, and identifies the tattoo sticker features of the knuckles of fingers or the palm through the camera and the yolov5 algorithm.
[0019] The data transmission module divides the identified tattoo sticker features of the knuckles of fingers and the palm into corresponding keyboard or mouse key events and transmits them to the smart watch, and the smart watch sends them to the smart glasses through Bluetooth.
[0020] The execution module converts the events received by the smart glasses into corresponding keys of the smart glasses app, and executes the key operation.
[0021] Further, the data transmission module divides the identified tattoo sticker features of the knuckles of fingers and the palm into corresponding mouse key events and transmits them to the smart watch, specifically through mouse mode mapping, that is, a table with serial numbers is set on the palm on the tattoo sticker, the table content is identified to find out the palm coordinates, and the moving coordinates are obtained and sent to the smart glasses, and the smart glasses control the mouse pointer in the glasses to move according to the moving coordinates; wherein, the palm table area and the screen in the smart glasses are mapped, that is, the screen pixels are equally divided according to the table, the divided pixels are corresponded to the serial numbers in the table, and then the mouse moves according to the midpoint position of the serial numbers captured by the camera, that is, the corresponding coordinates can be obtained.
[0022] Further, the data transmission module divides the identified tattoo sticker features of the knuckles of fingers and the palm into corresponding keyboard key events and transmits them to the smart watch, specifically by pasting tattoo sticker feature marks on the knuckles of the index finger, middle finger and ring finger and the palm knuckles, and the tattoo sticker feature marks on the knuckles of the index finger, middle finger and ring finger and the palm knuckles are as follows:
[0023]
[0024] This simulates a 9-key Pinyin input method, where A1 represents 1, A2 represents 2abc, A3 represents 3def, A4 is empty, A5 represents 4ghi, A6 represents 5jkl, A7 represents 6mno, A8 is empty, A9 represents pqrs, AA represents 8tuv, AB represents 9wxyz, AC is empty, AD (little finger knuckle) represents switching between number and Pinyin input methods, AE (ring finger knuckle) represents 0, AF (middle finger knuckle) represents switching between candidate characters, and AG (index finger knuckle) represents confirming candidate characters. Users select the corresponding marked letter in the table to choose either Pinyin or a word.
[0025] Furthermore, during input, the smart glasses will project the app interface in front of the glasses. At the bottom of the screen, several pinyin candidate combinations are displayed. After clicking the knuckle corresponding to the number, the candidate pinyin or candidate word is selected by the number. After selection, it is input into the smartwatch, and then the smartwatch sends it to the smart glasses via Bluetooth.
[0026] The beneficial effects of this invention are as follows: This invention connects a camera to a smartwatch and fixes the camera to a ring, setting corresponding tattoo sticker features on finger joints and palms; an image processing program runs on the smartwatch, and the tattoo sticker features on the finger joints or palms are identified through the camera and YOLOv5 algorithm; the identified tattoo sticker features on the finger joints and palms are distinguished as corresponding keyboard or mouse button events and transmitted to the smartwatch, which then sends them to smart glasses via Bluetooth, thereby controlling the smart glasses to operate; this frees up the hands and allows for portable control of the smart glasses. Attached Figure Description
[0027] Figure 1 This is a schematic diagram of the method flow of the present invention.
[0028] Figure 2 This is a schematic diagram of the system framework of the present invention. Detailed Implementation
[0029] The invention will now be further described with reference to the accompanying drawings.
[0030] Please see Figure 1 As shown, the present invention provides a method for controlling the operation of smart glasses via finger gestures, the method comprising the following steps:
[0031] Step S1: Connect a camera to a smartwatch and fix the camera to the ring. Set the corresponding tattoo sticker features on the finger joints and palms.
[0032] Step S2: Run an image processing program on the smartwatch and use the camera and YOLOv5 algorithm to identify the features of the tattoo stickers on the finger joints or palms.
[0033] Step S3, the identified finger knuckle and palm tattoo sticker feature is divided into corresponding keyboard or mouse key event transmission to the smart watch, and the smart watch sends to the smart glasses through Bluetooth;
[0034] Step S4, the smart glasses receive the event and convert it into a corresponding key of the smart glasses app, and perform the key operation.
[0035] The application will be further described below in combination with a specific embodiment:
[0036] A method for controlling the operation of smart glasses by finger gestures, the method comprising:
[0037] 1. A camera is connected to the smart watch, and the camera is fixed on the ring, and corresponding tattoo sticker features are set on the finger knuckles and palms.
[0038] 2. This scheme runs an image recognition module on the watch, and identifies the finger knuckles or palm features through the yolov5 algorithm (this technology is already a prior art and will not be described in detail here). In order to improve the recognition accuracy, a pen can be used to draw characters on the hand in advance, or a night light tattoo sticker can be used to paste patterns. The feature of the knuckle is the texture pattern feature of the knuckle, and the feature recognition rate can be enhanced by using a fingerprint sticker pattern. The palm feature is also similar, and the original texture shape feature on the palm can be enhanced by pasting a tattoo sticker.
[0039] 3. The identified finger knuckles and palm tattoo sticker features are divided into corresponding keyboard or mouse key events and transmitted to the smart watch, and the smart watch sends to the smart glasses through Bluetooth.
[0040] The scheme for dividing the identified finger knuckles and palm tattoo sticker features into corresponding mouse key events and transmitting them to the smart watch is as follows:
[0041] The smart glasses device receives the event and converts it into a corresponding key
[0042] 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030
[0043] 3.1 Mouse mode mapping, through mouse mode mapping, that is, setting a table with serial numbers on the palm by tattoo stickers, recognizing the table content to view the palm coordinates, and obtaining the moving coordinates, sending the moving coordinates to the smart glasses, the smart glasses controlling the mouse pointer in the glasses to move according to the moving coordinates; wherein, the palm table area and the screen in the smart glasses are mapped, that is, according to the table, the screen pixels are equally divided and cut, the cut pixels are corresponded to the serial numbers in the table, then the mouse moves according to the midpoint position of the serial numbers shot by the camera, that is, the corresponding coordinates can be obtained. The mouse pointer position can be specified by Bluetooth signal, and the mobile phone also has this function. When the mobile phone uses a Bluetooth mouse, the corresponding mouse pointer will appear on the screen.
[0044] For example: according to the table, the screen pixels are equally divided and cut, for example, 500X600 resolution, then the 500X600 resolution is cut, the (0, 0) to (100, 100) coordinates are controlled by the camera midpoint in the 001 area, similarly, 002 corresponds to (101, 0) to (100, 200), and so on, the palm table area and the screen in the smart glasses are equally divided and mapped;
[0045] The table with serial numbers is as follows:
[0046] 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030
[0047] Table 1
[0048] That is, 5 rows and 6 columns, corresponding to 500X600 resolution; the resolution is equally divided to correspond to the mapping.
[0049] The scheme for distinguishing the recognized finger knuckles and palm tattoo sticker feature area into corresponding keyboard key events and transmitting to the smart watch is as follows:
[0050] 3.2 Keyboard mode, by pasting tattoo sticker feature marks on the middle finger, ring finger and palm knuckles of the index finger, the tattoo sticker feature marks of the middle finger, ring finger and palm knuckles of the index finger are as follows:
[0051]
[0052]
[0053] Table 2
[0054] That is, it simulates the 9-grid pinyin. Among them, A1 represents 1, A2 represents 2abc, A3 represents 3def, A4 is empty, A5 represents 4ghi, A6 represents 5jkl, A7 represents 6mno, A8 is empty, A9 represents pqrs, AA represents 8tuv, AB represents 9wxyz, AC is empty, AD, that is, the metacarpophalangeal joint of the little finger, represents switching between the numeric and pinyin input methods, AE, that is, the metacarpophalangeal joint of the ring finger, represents 0, AF, that is, the metacarpophalangeal joint of the middle finger, represents switching candidate characters, AG, that is, the metacarpophalangeal joint of the index finger, represents confirming candidate characters, and the pinyin or word selection is completed by selecting the marked letters in the corresponding table.
[0055] That is, Table 3 below is the content corresponding to the serial numbers in Table 2:
[0056]
[0057] Table 3
[0058] 3.2.1 During input, the smart glasses will project the interface of the app in front of the glasses, and the most likely pinyin candidate combinations of the input pinyins will be displayed at the bottom of the screen. After clicking the corresponding finger joint of the number, the candidate pinyin or candidate word is selected through the number.
[0059] For example: When the center point of the current shot by the camera of the thumb is A2, it means it is on 2ABC. When pressed, it is detected that the camera image is enlarged, indicating that the current button is pressed and the event of abc is triggered. For example, pressing A2 twice can display candidate characters 1. Ba; 2. Ba..., the default selection is Ba, pressing AF switches to the second candidate character Ba, and pressing AG confirms the input;
[0060] 3.2.2 After selection, it is input into the smart glasses app, and the smart watch then sends it to the smart glasses via Bluetooth.
[0061] For example: In the WeChat chat window, pressing A2 twice can display candidate characters 1. Ba; 2. Ba..., the default selection is Ba, pressing AF switches to the second candidate character Ba, and pressing AG confirms the input. At this time, there will be a "Ba" character in the chat window.
[0062] 4. The smart glasses receive the event and convert it into the corresponding button of the smart glasses app, and execute the button operation.
[0063] Please refer to Figure 2 As shown, the present invention also provides a system for controlling the operation of smart glasses through finger gestures. The system includes: an installation and setting module, an identification module, a data transmission module, and an execution module;
[0064] The installation setting module connects a camera on the smart watch, and fixes the camera on the ring, and sets corresponding tattoo sticker features on the finger joints and the palm;
[0065] The identification module runs an image program on the smart watch, and identifies the tattoo sticker features of the finger joints or the palm through the camera and the yolov5 algorithm;
[0066] The data transmission module divides the identified tattoo sticker features of the finger joints and the palm into corresponding keyboard or mouse key events and transmits them to the smart watch, and the smart watch sends them to the smart glasses through Bluetooth;
[0067] In the data transmission module, the identified tattoo sticker features of the finger joints and the palm are divided into corresponding mouse key events and transmitted to the smart watch, which is specifically: through mouse mode mapping, that is, setting a table with serial numbers on the tattoo sticker on the palm, identifying the table content to find out that it is the palm coordinates, and obtaining the moving coordinates, and sending the moving coordinates to the smart glasses, and the smart glasses control the mouse pointer in the glasses to move according to the moving coordinates; wherein, the palm table area and the screen in the smart glasses are mapped, that is, according to the table, the screen pixels are equally divided and cut, the cut pixels are corresponded to the serial numbers in the table, then the mouse moves according to the midpoint position of the serial numbers shot by the camera, that is, the corresponding coordinates can be obtained.
[0068] For example: according to the table, the screen pixels are equally divided and cut, for example, the resolution of 500X600 is cut, then the (0, 0) to (100, 100) coordinates are controlled by the midpoint in the 001 area of the camera, similarly, 002 corresponds to (101, 0) to (100, 200), and so on, the palm table area and the screen in the smart glasses are mapped;
[0069] The table with serial numbers is as follows:
[0070]
[0071]
[0072] In the data transmission module, the identified tattoo sticker features of the finger joints and the palm are divided into corresponding keyboard key events and transmitted to the smart watch, which is specifically: by pasting tattoo sticker feature marks on the joints of the index finger, middle finger and ring finger and the palm joints, the tattoo sticker feature marks on the joints of the index finger, middle finger and ring finger and the palm joints are as follows:
[0073]
[0074] This simulates a 9-key Pinyin input method, where A1 represents 1, A2 represents 2abc, A3 represents 3def, A4 is empty, A5 represents 4ghi, A6 represents 5jkl, A7 represents 6mno, A8 is empty, A9 represents pqrs, AA represents 8tuv, AB represents 9wxyz, AC is empty, AD (little finger knuckle) represents switching between number and Pinyin input methods, AE (ring finger knuckle) represents 0, AF (middle finger knuckle) represents switching between candidate characters, and AG (index finger knuckle) represents confirming candidate characters. Users select the corresponding marked letter in the table to choose either Pinyin or a word.
[0075] The execution module converts the events received by the smart glasses into corresponding button presses in the smart glasses app and executes the button operations.
[0076] When inputting, the smart glasses will project the app interface in front of the glasses. At the bottom of the screen, several pinyin candidate combinations are displayed. After clicking the knuckle corresponding to the number, the candidate pinyin or candidate word is selected by the number. After selection, it is input into the smartwatch, and then the smartwatch sends it to the smart glasses via Bluetooth.
[0077] In summary, this invention connects a camera to a smartwatch and fixes the camera to a ring, setting corresponding tattoo sticker features on the finger joints and palms; an image program runs on the smartwatch, and the tattoo sticker features on the finger joints or palms are identified through the camera and YOLOv5 algorithm; the identified tattoo sticker features on the finger joints and palms are distinguished as corresponding keyboard or mouse button events and transmitted to the smartwatch, which then sends them to smart glasses via Bluetooth, thereby controlling the smart glasses to operate; this frees up the hands and allows for portable control of smart glasses.
[0078] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be included in the scope of the present invention.
Claims
1. A method for controlling the operation of smart glasses by finger gestures, characterized in that: The method includes the following steps: Step S1: Connect a camera to a smartwatch and fix the camera to the ring. Set the corresponding tattoo sticker features on the finger joints and palms. Step S2: Run an image processing program on the smartwatch and use the camera and YOLOv5 algorithm to identify the features of the tattoo stickers on the finger joints or palms. Step S3: The identified features of the tattoo stickers on the finger joints and palms are distinguished into corresponding keyboard or mouse button events and transmitted to the smartwatch. The smartwatch then sends the data to the smart glasses via Bluetooth. Step S4: The smart glasses receive the event, convert it into the corresponding button in the smart glasses app, and execute the button operation; In step S3, the identified features of the finger joints and palm tattoo stickers are distinguished into corresponding mouse button events and transmitted to the smartwatch. Specifically, this involves mapping the tattoo stickers to a numbered table on the palm, identifying the table content to determine the palm coordinates, obtaining the movement coordinates, and sending the movement coordinates to the smart glasses. The smart glasses then control the mouse pointer within the glasses to move the corresponding pixels according to the movement coordinates. Furthermore, the palm table area is mapped to the screen in the smart glasses. This involves dividing the screen pixels into equal parts based on the table, and then mapping the divided pixels to the numbers in the table. The mouse then moves according to the midpoint of the number captured by the camera, thus obtaining the corresponding coordinates. In step S3, the identified tattoo sticker features of the finger joints and palms are distinguished into corresponding keyboard key events and transmitted to the smartwatch. Specifically, tattoo sticker feature marks are affixed to the three joints of the index, middle, ring, and little fingers and the metacarpophalangeal joints. The tattoo sticker feature marks of the three joints of the index, middle, ring, and little fingers and the metacarpophalangeal joints are shown in the table below: This simulates a 9-key Pinyin input method, where A1 represents 1, A2 represents 2abc, A3 represents 3def, A4 is empty, A5 represents 4ghi, A6 represents 5jkl, A7 represents 6mno, A8 is empty, A9 represents pqrs, AA represents 8tuv, AB represents 9wxyz, AC is empty, AD (little finger knuckle) represents switching between number and Pinyin input methods, AE (ring finger knuckle) represents 0, AF (middle finger knuckle) represents switching between candidate characters, and AG (index finger knuckle) represents confirming candidate characters. Users select the corresponding marked letter in the table to choose either Pinyin or a word.
2. The method for controlling the operation of the smart glasses by finger gestures according to claim 1, characterized in that: When inputting, the smart glasses will project the app interface in front of the glasses. At the bottom of the screen, several pinyin candidate combinations are displayed. After clicking the knuckle corresponding to the number, the candidate pinyin or candidate word can be selected by the number. After selection, the information is entered into the smartwatch, which then sends it to the smart glasses via Bluetooth.
3. A system for controlling the operation of smart glasses by finger gestures, characterized in that it comprises: The system includes: an installation and setting module, an identification module, a data transmission module, and an execution module; The installation and setting module connects a camera to the smartwatch and fixes the camera to the ring, setting corresponding tattoo sticker features on the finger joints and palms; The recognition module runs an image program on the smartwatch and uses the camera and YOLOv5 algorithm to recognize the features of tattoo stickers on the finger joints or palms. The data transmission module identifies the features of the tattoo stickers on the finger joints and palms and distinguishes them as corresponding keyboard or mouse button events, which are then transmitted to the smartwatch. The smartwatch then sends the data to the smart glasses via Bluetooth. The execution module converts the events received by the smart glasses into corresponding button presses in the smart glasses app and executes the button operations. The data transmission module identifies the features of the finger joints and palm tattoo stickers, distinguishes them as corresponding mouse button events, and transmits them to the smartwatch. Specifically, it uses mouse mode mapping: a numbered table is set on the tattoo sticker on the palm, the table content is identified to determine the palm coordinates, and the movement coordinates are obtained and sent to the smart glasses. The smart glasses control the mouse pointer in the glasses to move the corresponding pixels according to the movement coordinates. The palm table area is mapped to the screen in the smart glasses. That is, according to the table, the screen pixels are divided equally, and the divided pixels are mapped to the numbers in the table. Then, the mouse moves according to the midpoint position of the number captured by the camera, and the corresponding coordinates can be obtained. The data transmission module identifies the tattoo sticker features on the finger joints and palms, classifies them into corresponding keyboard key events, and transmits them to the smartwatch. Specifically, tattoo sticker feature marks are applied to the three joints of the index, middle, ring, and little fingers, as well as the metacarpophalangeal joints. The tattoo sticker feature marks for the three joints of the index, middle, ring, and little fingers, and the metacarpophalangeal joints are shown in the table below: This simulates a 9-key Pinyin input method, where A1 represents 1, A2 represents 2abc, A3 represents 3def, A4 is empty, A5 represents 4ghi, A6 represents 5jkl, A7 represents 6mno, A8 is empty, A9 represents pqrs, AA represents 8tuv, AB represents 9wxyz, AC is empty, AD (little finger knuckle) represents switching between number and Pinyin input methods, AE (ring finger knuckle) represents 0, AF (middle finger knuckle) represents switching between candidate characters, and AG (index finger knuckle) represents confirming candidate characters. Users select the corresponding marked letter in the table to choose either Pinyin or a word.
4. A system for controlling smart glasses operation via finger gestures according to claim 3, characterized in that: When inputting, the smart glasses will project the app interface in front of the glasses. At the bottom of the screen, several pinyin candidate combinations are displayed. After clicking the knuckle corresponding to the number, the candidate pinyin or candidate word can be selected by the number. After selection, the information is entered into the smartwatch, which then sends it to the smart glasses via Bluetooth.
Citation Information
Patent Citations
Directional gesture input method and device for intelligent televisions or intelligent glasses
CN105700806A
Wrist gesture control system and method
CN102915111A
Augmented reality virtual keyboard input method and apparatus using same
CN105224069A