A system and a method for enabling gesture-based typing for smart devices

Abstract

Disclosed is a system and method for enabling gesture-based typing for smart device that allows a user to enter characters such as texts, numbers and / or symbols without physically touching the device. The system comprises a gesture detection and recognition module, a character prediction module and a display interface that outputs or displays the predicted character. This includes a wearable device incorporated with a motion sensing unit forms the gesture detection and recognition module in communication with the smart device. The character prediction module is configured to receive and extract the features from gesture data and produce command signals corresponding to the respective character in accordance with a set of predefined features in the customizable library file containing gesture features which forms a unique "language / script" containing finite set of gestures, similar to the scripts of a language.

Description

A SYSTEM AND A METHOD FOR ENABLING GESTURE-BASED TYPING FOR SMART DEVICESFIELD OF THE INVENTION

[0001] The present invention generally relates to information technology and more particularly relates to a system and a method designed to enable gesture-based typing on smart devices.BACKGROUND OF THE INVENTION

[0002] Traditional way of typing on smart devices involves using tiny on-screen keyboards, which can be cumbersome and error-prone, especially on smaller devices like smartphones and smartwatches. The keyboards of such devices, also have limitations in terms of portability, tactile feedback, and hygiene. Even more pertinent is the fact that they implicitly require the specific letter to be placed in a certain location. This means that the user must possess knowledge of the specific location of the desired letter also needs sufficient dexterity to tap the exact letter. Further, voice typing has emerged as an alternative, but it lacks precision and privacy. It is also languagedependent, requiring complex processing and extensive data handling. Additionally, pronunciation and accents can further reduce its accuracy. Hence, there is a need for a novel solution for typing on a screen or executing commands that combines mobility, inclusivity, and hygiene. This solution should enable users to type on smart devices directly, without relying on a prescribed layout. It must also function without direct contact with the output surface and maintain accuracy in both line-of-sight and out-of line-of-sight conditions.

[0003] Accordingly, there exists a need to provide a system and a method for enabling gesture-based based typing on smart devices by replacing traditional on-screen keyboards with gesture recognition, thereby providing a more intuitive user experience.OBJECTIVES OF THE INVENTION

[0004] An objective of the present invention is to facilitate accurate and faster gesture based communication with a smart device via hand gestures.

[0005] Another objective of the present invention is to empower users with different abilities, and help them to express themselves effortlessly while typing on a smart device such as smartphones, tablets, smartwatches, and other similar devices.

[0006] Yet, another objective of the present invention is to interpret a finite set of hand gestures in the air and to generate signals corresponding to different letters, numbers and symbols for typing on a smart device.

[0007] Yet, another objective of the present invention is to allow the typist to reassign the gesture to letter mapping to his / her desired preference.

[0008] Yet, another objective of the present invention is to allow the typist to type symbols and phrases instead of single letters.

[0009] Yet, another objective of the present invention is to expand the number of letters and symbols that can be typed to an unlimited quantity simply by adding additional modes.SUMMARY OF THE INVENTION

[0010] The present invention presents a system and a method for enabling gesturebased typing for smart devices such as smartphones, tablets, smartwatches, and other similar devices. This eliminates the need for an on-screen keyboard as well as a mouse that gives a visible typing surface, in close proximity or, within a touchable distance from the user. The gesture-based typing in accordance with the present invention aims to augment the user experience by enabling efficient text input via gestures, as opposed to conventional typing methods. This approach obviates the necessity for a keyboard or the requirement to adhere to a specific predefined layout.

[0011] According to the present invention a gesture detection and recognition module comprise of an electronic device that captures the gestures of the user particularly the hand gestures in the air. The electronic device is designed as a wearable device incorporating a motion sensing unit that employs various sensors such as accelerometer, magnetometers and gyroscope alone or in combination coupled to a processor. The sensors are configured to capture the hand gestures of a user and the processor with an embedded application that processes the sensor data. The motion sensing unit is configured to measure and deliver the observed data related to acceleration, orientation, angular rates, and other values associated with the hand gesture.

[0012] According to the present invention a method for enabling gesture-character typing for the smart devices is presented in accordance with an implementation of an embodiment of the present invention. The method starts with setting up the connectivity between the wearable device and the smart device. The wearable device triggers the motion sensing unit and reads the motion till the sensor stopped obtaining user's gesture data. An intelligent terminal of the character prediction module receives a feature corresponding to the realtime captured gesture and gesture detection. The character prediction module maps the realtime gesture data code with the look up table stored in the library file. This includes one to one mapping of the gesture feature withthe described gesture feature data in library file. The intelligent terminal judges that whether described gesture feature data is consistent with the gesture feature data in look up table in the library file and obtains the gesture title corresponding with described gesture feature data. This gesture data is then converted into command signals for the character entry. If the described gesture feature data is inconsistent with the gesture feature data in library file, the module sends an incorrect prompting of gesture and returns to the earlier step.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The objectives and advantages of the present invention will become apparent when the disclosure is read in conjunction with the following figures, whereinFigure 1 shows an exploded view of a wearable device of a system for enabling gesturebased typing for smart devices in ring configuration in accordance with an embodiment of the present invention;Figure 2 shows an exploded view of a wearable device of a system for enabling gesturebased typing for smart devices in smart watch configuration in accordance with an embodiment of the present invention; andFigure 3 shows a pictorial view of the wearable device of the system enabling gesturebased typing for smart devices for smart devices in accordance with an embodiment of the present invention.Figure 4 illustrates overall functional flow of the system and method thereof of for enabling gesture-based typing for smart devices in accordance with one of the embodiments of the present invention.Figure 5 A-C illustrates flow diagram of activating keyboard mode according to one of the embodiments of the present invention.Figure 5 D-G illustrates exemplary gestures lists for an exemplary language selected for communication according to one of the embodiments of the present invention.Figure 6 illustrates flow diagram of executing basic gesture training process according to one of the embodiments of the present invention.Figure 7 illustrates flow diagram of executing complex gesture training and operation process according to one of the embodiments of the present invention.Figure 8 illustrates flow diagram of executing process of complex camera gesture logic implementation according to one of the embodiments of the present invention.Figure 9 illustrates flow diagram of executing process for changing letters that is being typed according to one of the embodiments of the present invention.Figure 10, illustrates flow diagram of executing process for adding a new keyboard layout according to one of the embodiments of the present invention.DETAILED DESCRIPTION OF THE INVENTION

[0014] The foregoing objects of the present invention are accomplished and the problems and shortcomings associated with the prior art, techniques, and approaches are overcome by the present invention as described below in the preferred embodiments.

[0015] The present disclosure presents a system and a method for enabling gesturebased typing for smart devices such as smartphones, tablets, smartwatches, and other similar devices. This eliminates the need for an on-screen keyboard as well as a mouse that gives a visible typing surface, in close proximity or, within a touchable distance from the user. The gesture-based typing in accordance with the present invention aims to augment the user experience by enabling efficient text input via gestures, as opposedto conventional typing methods. This approach obviates the necessity for a keyboard or the requirement to adhere to a specific predefined layout.

[0016] In the following description, for the purpose of explanation, specific details are set forth in order to provide an understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these details. One skilled in the art will recognize that embodiments of the present invention, some of which are described below, may be incorporated into a number of systems.

[0017] Throughout this application, with respect to all reasonable derivatives of such terms, and unless otherwise specified (and / or unless the particular context clearly dictates otherwise), each usage of:“a” or “an” is meant to read as “at least one.”“the” is meant to be read as “the at least one.”

[0018] References in the present invention to “one embodiment” or “an embodiment” mean that a particular feature, structure, characteristic, or function described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

[0019] Embodiments of the present invention include various steps, which will be described below. The steps may be performed by hardware components and may be embodied in machine-executable instructions, which may be used to cause a general- purpose or special purpose processor programmed with the instructions to perform the steps. Alternatively, steps may be performed by a combination of hardware, software, firmware and / or by human operators.

[0020] Various methods described herein may be practiced by combining one or more machine-readable storage media containing the code according to the present invention with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various embodiments of the present invention may involve one or more processors and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of the invention could be accomplished by modules, routines, subroutines, or subparts of a computer program product.

[0021] If the specification states a component or feature "may”, “can", "could", or "might" be included or have a characteristic, that particular component or feature is not required to be included or have the characteristic.

[0022] As used in the description herein and throughout the claims that follow, the meaning of "a”, “an," and "the" includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.

[0023] Exemplary embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this invention will be thorough and complete and will fully convey the scope of the invention to those of ordinary skilled in the art. Moreover, all statements herein reciting embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure).

[0024] Hereinafter, embodiments will be described in detail. For clarity of the description, known constructions and functions will be omitted. Parts of the description may be presented in terms of operations performed by a plurality of electrical or electronic components with the manner commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. As is well understood by those skilled in the art, these quantities take the form of data stored / transferred in the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, and otherwise manipulated through mechanical and electrical components of the electronic / electrical systems.

[0025] While embodiments of the present invention have been illustrated and described, it will be clear that the invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the invention, as described in the claim.

[0026] The present invention is illustrated with reference to the accompanying drawing, throughout which reference numbers indicate corresponding parts in the various figures. These reference numbers are shown in brackets in the following description.

[0027] In an aspect of the present invention, a system and a method for enabling gesture-based typing for smart devices is disclosed. The gesture-based typing in accordance with the present invention provides an improved experience for the users by enabling them to input text through simple straight-line and limited curved gestures, thereby increasing the typing speed and accuracy. The system comprises a gesture detection and recognition module, a character prediction module and a display interface that outputs or displays the predicted character.

[0028] In an embodiment of the present invention, the gesture detection and recognition module comprise of an electronic device that captures the gestures of the user particularly the hand gestures in the air. The electronic device is designed as a wearable device incorporating a motion sensing unit that employs various sensors such as accelerometer, magnetometers and gyroscope alone or in combination coupled to a processor. The sensors are configured to capture the hand gestures of a user and the processor with an embedded application that processes the sensor data. The motion sensing unit is configured to measure and deliver the observed data related to acceleration, orientation, angular rates, and other values associated with the hand gesture.

[0029] The electronic device is incorporated with a communication interface unit that establishes a wireless or a wired link to allow the communication with the smart device through radio signals.

[0030] In an embodiment of the present invention, the communication module includes a transceiver that is designed to prioritize the connectivity to automatically connect and disconnect from the devices transmitting the stronger signals.

[0031] In an embodiment of the present invention, the communication module includes a wireless module capable of enabling communication between the wearable device and the smart device via near field communication. The wireless module is capable of transmitting data by way of Bluetooth communication protocols but not limited thereto.

[0032] In an embodiment of the present invention, the processor is configured to receive the observed data from the motion sensors, process them and calculate maximum and minimum angles for pitch, roll, and yaw using the sensor particularly the gyroscope readings and produce a composite number derived from the observed values.

[0033] The gesture detection and recognition module are configured to be in communication with the character prediction module and is configured to transfer the processed realtime captured data thereto.

[0034] The character prediction module comprising an application module in communication with a database and a processing unit within the smart device. The database contains a look up table with a predetermined set of features, and each feature in the predetermined set corresponding to a specific command for the inputting characters.

[0035] The character prediction module thus produces command signals and are communicated through the application module that interprets the associated action commands for the smart device. The look up table comprises a predetermined set of features corresponding to limited number of gestures, where each of the gesture is corresponding to multiple characters. The character interpretation is context sensitive and can operate in a multitude of usage modes. This allows usage of the same set of gestures for different characters based on the operation mode selected. For example, in one embodiment, a gesture is used to enter a lower-case letter, in a second mode the same gesture is used to enter numerical value.

[0036] In an embodiment of the present invention, the character prediction module includes an intelligent terminal module that receives the realtime gesture information from the motion sensor module and predicts the character by applying machine intelligence. This also includes extracting the feature data from the sensor readings and predict the character such as texts, numerical values and / or symbols from the extracted features based on a learned data set.

[0037] In an implementation of an embodiment, the look up table containing the finite number of gestures are customisable by the user. This improves the accuracy of communication and ease of use. The application module is configured to interact withthe user via the display interface of the smart device and provide a list of modes that the user can choose from. This can also replace the previously assigned action for the gestures.

[0038] The accuracy and consistency of gesture detection is maintained for varying users, by allowing modification of the angle and duration of each gesture using an image capturing device-based application within the character prediction module. Thus, the wearable device can read the best default angle and duration of each gesture.

[0039] In an implementation of an embodiment, the character prediction module is configured to modify the letters that are assigned to a gesture with letters from a different language script of the users choosing. This capability allows the users to support any standard writing system in existence for example the Devnagari, Cyrillic Traditional Chinese etc.

[0040] Referring to the figure 1 to 3(A-C), a wearable device is illustrated in accordance with an embodiment of the present invention. In various embodiments, the wearable device (50) can be made in the form of a ring, a wristwatch, or any other wearable device. The wearable device (50) includes a motion sensing unit, a communication module, a touch-control module and a rigid enclosure. The enclosure houses the electronic circuit therein, and can be worn or held by a user, allowing for unencumbered movement. The circuit includes a microcontroller (5) and a set of sensors (7) together forms motion sensing unit, and a power control unit embedded on a printed circuit board (PCB) (1). The power control unit is configured to manage the battery (12) and charging thereof. The communication module transmits the realtime data captured by the motion sensing unit to the smart device. The wearable device collects data through the motion sensing unit, and communicates the movement in the form of a gesture code with the proprietary software application installed in the target device such as the smart device. In the implementation, the hand gesture acts as a bridgebetween the user and the smart device. The gesture performed by the user converts into a character by the character prediction module.

[0041] In an implementation of the present invention, a Bluetooth (BLE) API is incorporated with the communication module that facilitates the data transfer from the wearable device to the BLE enabled smart device application in the form of digital data stream of Bytes.

[0042] In an embodiment of the present invention, the wearable device includes a button for setting multiple operation modes and also supports the hand gesture-based mode selection for the keyboard operations. This includes lowercase, uppercase, numeric lowercase, uppercase numeric and emojis / shortcuts. The user can also assign new modes for specific functionality that are active only when the application module is active and in-focus.

[0043] In another aspect of the present invention, a method for enabling gesturecharacter typing for the smart devices is presented in accordance with an implementation of an embodiment of the present invention. The method starts with setting up the connectivity between the wearable device and the smart device. The wearable device triggers the motion sensing unit and reads the motion till the sensor stopped obtaining user's gesture data. An intelligent terminal of the character prediction module receives a feature corresponding to the realtime captured gesture and gesture detection. The character prediction module maps the realtime gesture data code with the look up table stored in the library file. This includes one to one mapping of the gesture feature with the described gesture feature data in library file. The intelligent terminal judges that whether described gesture feature data is consistent with the gesture feature data in look up table in the library file and obtains the gesture title corresponding with described gesture feature data. This gesture data is then converted into command signals for the character entry. If the described gesture feature data isinconsistent with the gesture feature data in library file, the module sends an incorrect prompting of gesture and returns to the earlier step.

[0044] The connectivity between the wearable device and the smart devices happens through the steps: initiating smart device by intelligent terminal module and scans to obtain the wearable device in vicinity; sending a request by the intelligent terminal module to connect with a target device application module receives response therefrom; establishes the connectivity with the wearable device if the request match with connectivity requests.

[0045] In an embodiment of the present invention, the wearable device includes a switch button (10) for setting multiple operation modes and also supports the hand gesture based mode selection. In a preferred embodiment, the system provides 3 basic operational modes for a smartphone, such as gesture, phone and mouse. The system also allows a user to assign new modes for specific functionality that are active only when the other applications of the smart device is active and in-focus.

[0046] The figure 3B illustrates block diagram of the wearable gesture sensor device of the system facilitating gesture-based control for devices in accordance with an embodiment of the present invention. As illustrated in the figure the wearable gesture sensor device of the system facilitating gesture-based control according to one of the embodiments of the present invention comprises a battery, a plurality of switches, at least one 6D accelerometer sensor, at least one LOD module, at least one PMIC Battery charging IC, at least one Fuel guage, at least one Microcontroller. The wearable gesture sensor device upon power on activates the at least one 6D accelerometer sensor to detect the movement the wearable device undergoes to perform a gesture. The 6D accelerometer sensor sense the movement and generate and send the movement data in digital form to the microcontroller. The microcontroller is configured to control the overall functioning of the wearable gesture sensor device and embedded with a data encryption / decryption logic. The microcontroller receives the data of the movementrespective to the gesture in digital form from 6D accelerometer sensor, encrypt it to convert the data in digital format into gesture code in Hex format and communicates the generated gesture code data to the system application embedded at smart device according to one of the embodiments of the present invention

[0047] The figure 3C illustrates process flow diagram of the wearable gesture sensor device of the system facilitating gesture-based control for devices in accordance with an embodiment of the present invention. As illustrated in the figure the wearable gesture sensor device of the system facilitating gesture-based control according to one of the embodiments of the present invention is initialized by powering it on. The wearable gesture sensor device upon power on initiates a connection through Bluetooth connecting protocol and connects to the system application embedded at smart device according to one of the embodiments of the present invention. The wearable gesture sensor device detects the gesture movement performed and shares the detected gesture movement to the system application embedded at smart device according to one of the embodiments of the present invention. The wearable gesture sensor device when not in use switches to power off mode according to one of the embodiments of the present invention.

[0048] Referring to the Figure 4 overall functional flow of the system and method thereof of for enabling gesture-based typing for smart devices is illustrated in accordance with one of the embodiments of the present invention.

[0049] In an implementation according to one of the embodiments of the present invention a method for enabling gesture-based typing for smart devices is comprising step of connecting / reconnecting a wearable gesture sensor device and a system application embedded at an application module of the character prediction module embedded at a processing unit of a smart device. The method for enabling gesturebased typing for smart devices is comprising step of locking the wearable gesture sensor device to a user’s smart device such as phone. The method for enabling gesture-based typing for smart devices is comprising step of personalizing the wearable gesture sensor device by performing gesture configuration for selected device and modes including phone mode, gesture mode, mouse mode, and keyboard mode. The method for enabling gesture-based typing for smart devices is comprising step of activating a keyboard mode at the smart device. The method for enabling gesture-based typing for smart devices is comprising step of executing basic gesture training process. The method for enabling gesture-based typing for smart devices is comprising step of executing complex gesture training and operation process. The method for enabling gesture-based typing for smart devices is comprising step of executing process of complex camera gesture logic implementation. The method for enabling gesture-based typing for smart devices is comprising step of executing process for changing letters that is being typed. The method for enabling gesture-based typing for smart devices is comprising step of executing process for adding a new keyboard layout.

[0050] Referring to the Figure 5 A-C, flow diagram of activating keyboard mode is illustrated according to one of the embodiments of the present invention.

[0051] Referring to the Figure 5 D-G, exemplary gestures lists for an exemplary language selected for communication is illustrated according to one of the embodiments of the present invention.

[0052] In an implementation of a method for enabling gesture-based typing for smart devices according to one of the embodiments of the present invention the method step of activating a keyboard mode at the smart device is comprising step of initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device. The method step includes step of selecting phone mode at the system application initialized at the smart device. The method step includes step of activating wearable gesture sensor device for wearable gesture sensor device personalization by taping the option of wearable gesture sensor device personalization at the system application initialized at the smart device. The methodstep includes step of displaying default gestures assigned to various characters for the language selected for communication.

[0053] Referring to the Figure 6, flow diagram of executing basic gesture training process is illustrated according to one of the embodiments of the present invention.

[0054] In an implementation of a method for enabling gesture-based typing for smart devices according to one of the embodiments of the present invention the method step of executing basic gesture training process is comprising step of initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device. The method step includes step of selecting and activating gesture tuning mode at the system application initialized at the smart device. The method step includes step of selecting a gesture to be trained from the displayed list of gesture by taping on the desired gesture to be trained. The method step includes step of opening and activating the camera of the smart device by tapping the image of the camera displayed at the system application initialized at the smart device. The method step includes step of displaying the gesture arrows for the selected gesture to be trained on the camera view at the screen of the smart device by the system application at the smart device. The method step includes step of waving hand in air tracing the arrows as shown on the screen by the user wearing the wearable gesture sensor device communicatively connected to the smart device. The method step includes step of measuring, recording and saving at the database for each sub-gesture a direction of movement, duration and angle.

[0055] Referring to the Figure 7, flow diagram of executing complex gesture training and operation process is illustrated according to one of the embodiments of the present invention.

[0056] In an implementation of a method for enabling gesture-based typing for smart devices according to one of the embodiments of the present invention the method stepof executing complex gesture training and operation process is comprising step of initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device. The method step includes step of selecting and activating gesture tuning (Complex Gestures) mode at the system application initialized at the smart device. The method step includes step of selecting a plurality of characters to be trained for gestures from the displayed list of characters by taping on the desired characters to be trained. The method step includes step of opening and activating the camera with hand-tracking by tapping the image of the camera displayed at the system application initialized at the smart device. The method step includes step of displaying the selected character to be trained for gesture on the camera view at the screen of the smart device by the system application at the smart device. The method step includes step of waving hand in air tracing the character as shown on the screen by the user wearing the wearable gesture sensor device communicatively connected to the smart device. The method step includes step of measuring, recording and saving at the database for each character selected, a direction of movement, duration and angle for each gesture and each sub-gesture made which includes steps of opening the wearable gesture sensor device firmware, recording for each character the firmware records, accelerometer and gyroscope movements of the gesture made, mapping the screen and the wearable gesture sensor device movements to the character and store the relational data at the database.

[0057] In an implementation of a method for enabling gesture-based typing for smart devices according to one of the embodiments of the present invention the method step of executing complex gesture training and operation process is comprising step of initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device. The method step includes step of opening the wearable gesture sensor device firmware. The method step includes step of loading the stored accelerometer and gyroscope measurements for each character. The method step includes step of capturing wearable gesture sensor device movementreadings. The method step includes step of finding closest match to database values. The method step includes step of returning the closest match character.

[0058] Referring to the Figure 8, flow diagram of executing process of complex camera gesture logic implementation is illustrated according to one of the embodiments of the present invention.

[0059] In an implementation of a method for enabling gesture-based typing for smart devices according to one of the embodiments of the present invention the method step of executing process of complex camera gesture logic implementation is comprising step of initializing camera screen. The method step includes step of changing the focus to “Hand Movement”. The method step includes step of loading saved “character” models. The method step includes step of showing character model on camera screen. The method step includes step of assigning traced gesture model to “character” and saving the gesture model in a database. The method step includes calling / executing “processGesture with predictions” and process the gesture by iterating through states. The method step includes step of calling / executing recognition for each state, recognizing the gesture, checking if state matches and return recognized gesture.

[0060] Referring to the Figure 9, flow diagram of executing process for changing letters that is being typed is illustrated according to one of the embodiments of the present invention.

[0061] In an implementation of a method for enabling gesture-based typing for smart devices according to one of the embodiments of the present invention the method step of executing process for changing letters that is being typed is comprising step of initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device. The method step includes step of activating and opening gesture personalization by taping gesture personalization tab. The method step includes step of displaying assigned letters. The method step includesstep of tapping on desired assigned letter which user wants to change and selecting assigned as desired by the user. The method step includes step of updating the change of the selected assigned letter at the database. The method step includes step of updating the assigned letter and updating gesture personalization screen. The method step includes step of assigning gesture to the updated letter and make the updated letter ready for gesture.

[0062] Referring to the Figure 10, flow diagram of executing process for adding a new keyboard layout is illustrated according to one of the embodiments of the present invention.

[0063] In an implementation of a method for enabling gesture- based typing for smart devices according to one of the embodiments of the present invention the method step of executing process for adding a new keyboard layout is comprising step of initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device. The method step includes step of choosing a layout type and selecting a keyboard. The method step includes sending to the system application embedded at the smart device the keyboard value of the selected keyboard. The method step includes step of assigning the gesture to the keyboard value of the selected keyboard, update the database with assigned gesture to the selected keyboard and provide the keyboard ready to use to the user.

[0064] In an implementation according to preferred embodiments of the present invention a system and a method for enabling gesture-based typing for smart devices provides a predictive typing feature. Once a user has typed the first letter a short autocurated list of auto complete words is displayed. If desired the typist can select any one of the words displayed by performing a gesture to select the ordinal number of the word to be used for auto completion. This process continues as the user types the next letter and so on. The list of words is obtained from a database consisting of words from third-party databases, the words previously typed by the current typist and words submitted by other users of the invisible keyboard.

[0065] In an implementation according to preferred embodiments of the present invention a system and a method for enabling gesture-based typing for smart devices provides a predictive typing feature operated in two levels, Novice level and Expert level. In the novice level context dependent virtual keyboards are displayed to guide the user in learning and practising the gesture movement that types a particular letter. An audio announcement is provided that speaks the gesture made and the action selected (i.e. letter typed) and when the user can make the next gesture. In the expert level once the user has practised and mastered the knowledge of which gesture, types which letter, the user can switch to expert level. The user can toggle the levels back and forth at any time. In expert level, only the letter typed is announced. The user can immediately perform the next gesture, after the announcement is made. So in expert level, the user can type much faster, without the need to look at or touch the keyboard or adhere to a keyboard layout. All of this can be done from a distance, even when out of line-of-sight of the smart device.

[0066] ADVANTAGES OF THE INVENTION1. The system provides different modes of operation for the same set of gesture, this separation of command signals for the smart devices provides several tangible benefits, such as the user can perform activities without physically touching the device, for example, the user does not have to hold or touch the smart device within an arm’s length for typing or even looking at the display screen,2. The wearable device of the system helps to operate a smart device by a single commanding device that detects a finite number of gestures.3. The system employs a user customizable library file containing gestures which forms a unique “language / scripf ’ containing finite set of gestures, similar to the scriptsof a language. These gestures are taken for different meanings in different sequences and contexts. Therefore, the same set of gestures could possibly perform multiple set of characters.The foregoing descriptions of specific embodiments of the present invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the present invention and its practical application, to thereby enable others skilled in the art to best utilize the present invention and various embodiments with various modifications as are suited to the particular use contemplated. It is understood that various omission and substitutions of equivalents are contemplated as circumstance may suggest or render expedient, but such are intended to cover the application or implementation without departing from the scope of the present invention.List of components

Claims

WE CLAIM:

1. A method for enabling gesture-based typing for smart devices, the method comprising: connecting / reconnecting a wearable gesture sensor device and a system application embedded at an application module of a character prediction module embedded at a processing unit of a smart device; locking the wearable gesture sensor device to a user’s smart device; personalizing the wearable gesture sensor device by performing gesture configuration for selected device and modes including phone mode, gesture mode, mouse mode, and keyboard mode; activating a keyboard mode at the smart device; executing basic gesture training process; executing complex gesture training and operation process; executing process of complex camera gesture logic implementation; executing process for changing letters that is being typed; and executing process for adding a new keyboard layout.

2. The method as claimed in claim 1 wherein the method step of activating a keyboard mode at the smart device comprising: initializing a wearable gesture sensor device connectivity to the smart device and initialize system application embedded at the smart device; selecting phone mode at the system application initialized at the smart device; activating wearable gesture sensor device for wearable gesture sensor device personalization by taping the option of wearable gesture sensor device personalization at the system application initialized at the smart device; displaying default gestures assigned to various characters for the language selected for communication.

3. The method as claimed in claim 1 wherein the method step of executing basic gesture training process comprising: initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device; selecting and activating gesture tuning mode at the system application initialized at the smart device; selecting a gesture to be trained from the displayed list of gesture by taping on the desired gesture to be trained; opening and activating the camera of the smart device by tapping the image of the camera displayed at the system application initialized at the smart device; displaying the gesture arrows for the selected gesture to be trained on the camera view at the screen of the smart device by the system application at the smart device; waving hand in air tracing the arrows as shown on the screen by the user wearing the wearable gesture sensor device communicatively connected to the smart device; and measuring, recording and saving at the database for each sub-gesture a direction of movement, duration and angle.

4. The method as claimed in claim 1 wherein the method step of executing complex gesture training and operation process comprising: initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device; selecting and activating gesture tuning (Complex Gestures) mode at the system application initialized at the smart device; selecting a plurality of characters to be trained for gestures from the displayed list of characters by taping on the desired characters to be trained;opening and activating the camera with hand-tracking by tapping the image of the camera displayed at the system application initialized at the smart device; displaying the selected character to be trained for gesture on the camera view at the screen of the smart device by the system application at the smart device; waving hand in air tracing the character as shown on the screen by the user wearing the wearable gesture sensor device communicatively connected to the smart device; measuring, recording and saving at the database for each character selected, a direction of movement, duration and angle for each gesture and each sub-gesture made, includes: opening the wearable gesture sensor device firmware; recording for each character the firmware records, accelerometer and gyroscope movements of the gesture made; mapping the screen and the wearable gesture sensor device movements to the character and storing the relational data at the database. opening the wearable gesture sensor device firmware; loading the stored accelerometer and gyroscope measurements for each character; capturing wearable gesture sensor device movement readings; finding closest match to database values; and returning the closest match character.

5. The method as claimed in claim 1 wherein the method step of executing process of complex camera gesture logic implementation comprising: initializing camera screen; changing the focus to “Hand Movement”; loading saved “character” models;showing character model on camera screen; assigning traced gesture model to “character” and saving the gesture model in a database; calling / executing “processGesture with predictions” and process the gesture by iterating through states; calling / executing recognition for each state, recognizing the gesture, checking if state matches and return recognized gesture.

6. The method as claimed in claim 1 wherein the method step of executing process for changing letters that is being typed comprising: initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device; activating and opening gesture personalization by taping gesture personalization tab; displaying assigned letters; tapping on desired assigned letter which user wants to change and selecting assigned as desired by the user; updating the change of the selected assigned letter at the database; updating the assigned letter and updating gesture personalization screen; and assigning gesture to the updated letter and make the updated letter ready for gesture.

7. The method as claimed in claim 1 wherein the method step of executing process for adding a new keyboard layout comprising: initializing a wearable gesture sensor device connectivity to smart device and initialize system application embedded at the smart device; choosing a layout type and selecting a keyboard;sending to the system application embedded at the smart device the keyboard value of the selected keyboard; and assigning the gesture to the keyboard value of the selected keyboard, update the database with assigned gesture to the selected keyboard and provide the keyboard ready to use to the user.

8. The method as claimed in claim 1 wherein the method for enabling gesture-based typing for smart devices provides a predictive typing feature operated in two levels, Novice level and Expert level.

9. A system for enabling gesture-based typing for smart devices, the system comprises: at least one wearable gesture sensor device, the at least one wearable gesture sensor device comprises a battery, a plurality of switches, at least one 6D accelerometer sensor, at least one LOD module, at least one PMIC Battery charging IC, at least one Fuel guage, and at least one microcontroller, wherein the wearable gesture sensor device configured to activate upon power on the at least one 6D accelerometer sensor to detect the movement the wearable device undergoes to perform a gesture; the 6D accelerometer sensor configured to sense the movement and generate and send the movement data in digital form to the microcontroller; the microcontroller embedded with a data encryption / decryption logic configured to control the overall functioning of the wearable gesture sensor device; the microcontroller configured to receive the data of the movement respective to the gesture in digital form from 6D accelerometer sensor, encryptit to convert the data in digital format into gesture code in Hex format and communicate the generated gesture code data; at least one smart device, the at least on smart device comprising at least one processor and at least one memory communicatively coupled to the processor and the memory configured to store the command instructions set, the at least one processor having embedded a system application, the processor through the embedded system application configured to: connect / reconnect a wearable gesture sensor device and a system application embedded at an application module embedded at a processing unit of a smart device; lock the wearable gesture sensor device to a user’s smart device such as phone; personalize the wearable gesture sensor device by performing gesture configuration for selected device and modes including phone mode, gesture mode, mouse mode, and keyboard mode; build App gesture action table for the manually added apps, and apps from inventory having wearable gesture sensor device support; operate a third-party app through wearable gesture sensor device; customize the gesture actions of third-party App by changing the action of gesture; and operate an IOT device through wearable gesture sensor device and a system application embedded at IOT device.

Images