A single-finger virtual keyboard system and its use and input method

By using a six-axis inertial sensor and a twelve-aliased circular keyboard layout module in the virtual keyboard system, the target displacement of finger movement is solved, and the existing virtual keyboard solution requires a large amount of user data training is achieved, and efficient and flexible text input is achieved.

CN115686214BActive Publication Date: 2025-04-18SOUTH CHINA UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211402652.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-10
Publication Date
2025-04-18
Estimated Expiration
2042-11-10

AI Technical Summary

Technical Problem

The existing virtual keyboard solution requires a large amount of user data for training, which is poor in scalability, and the complex sensors are easily affected by the environment, making it difficult to effectively perform performance.

Method used

A single-finger virtual keyboard system based on a six-axis inertial sensor is designed. Through a twelve-allocated circular virtual keyboard layout module and a six-axis inertial sensor, the target displacement of the user's finger movement is calculated, free from the dependence on user data, and only one six-axis inertial sensor is used.

Benefits of technology

It realizes effective input without user data training, improves the system's expansion and anti-environmental interference capabilities, simplifies system design, and improves the accuracy of key recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115686214B_ABST
    Figure CN115686214B_ABST
Patent Text Reader

Abstract

The present invention discloses a single-finger virtual keyboard system and its use and input method. The system is composed of a virtual keyboard layout module, a six-axis inertial sensor, a microprocessor, and a PC host computer; the virtual keyboard layout module is used to display key positions and key values ​​for users; the six-axis inertial sensor is installed at the tip of the index finger of one hand, a total of one, for measuring the acceleration and angular velocity data during finger movement; the microprocessor is used to receive the acceleration and angular velocity data of the six-axis inertial sensor, and directly splice the acceleration and angular velocity data and send them to the PC host computer; the PC host computer is installed with PC-side software for receiving the acceleration and angular velocity data sent by the microprocessor, and processing the acceleration and angular velocity data. The present invention does not require user data training, improves the expansibility of the system, and the system is easy to build, expanding the use scenarios of the system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of virtual keyboards, and in particular to a single-finger virtual keyboard system based on a six-axis inertial sensor and a use and input method thereof. Background Art

[0002] As a text input method in traditional human-computer interaction, physical keyboards are widely used in daily work and life. The traditional physical keyboard is large in size and not easy to carry, which limits its scope of use. In addition, new human-computer interaction solutions such as gesture recognition, voice input, and handwriting input are also gradually becoming popular. Although these solutions have their own advantages, they all require user data for training, lack scalability, and are easily affected by light and noise in the usage scenario, and cannot fully exert their performance.

[0003] Virtual keyboard technology can effectively solve the problems of physical keyboards and allow users to apply their experience of using physical keyboards to usage scenarios, reducing the user's learning cost. However, the current virtual keyboard solution has the disadvantages of requiring a large amount of user data for training, binding motion data features to key positions, poor scalability, requiring more and more complex sensors, being easily affected by the environment, and not being able to fully exert its performance. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings and deficiencies of the existing virtual keyboard solutions. A single-finger virtual keyboard system based on a six-axis inertial sensor and its use and input method are proposed. A twelve-equally divided circular virtual keyboard layout module is designed, and the target displacement of the user's finger movement is calculated according to the acceleration and angular velocity data measured by a six-axis inertial sensor installed on the user's fingertips, and is linked to the key position of the virtual keyboard layout module, thereby getting rid of the deficiency of needing user data for training, and the sensor only needs one six-axis inertial sensor, so the system is simple and not easily affected by the environment.

[0005] To achieve the above object, the technical solution provided by the present invention is: a one-finger virtual keyboard system, the one-finger virtual keyboard system is composed of a virtual keyboard layout module, a six-axis inertial sensor, a microprocessor, and a PC host computer;

[0006] The virtual keyboard layout module is used to display key positions and key values ​​to the user;

[0007] The six-axis inertial sensor has one and is installed at the tip of the index finger of one hand, and is used to measure the acceleration and angular velocity data when the finger moves;

[0008] The microprocessor is used to receive the acceleration and angular velocity data of the six-axis inertial sensor, and directly splice the acceleration and angular velocity data and send them to the PC host computer;

[0009] The PC host computer is installed with PC-side software for receiving the acceleration and angular velocity data sent by the microprocessor and processing the acceleration and angular velocity data.

[0010] Furthermore, the virtual keyboard layout module is composed of an inner circle and an outer circle with the same center position and different radii, and all key positions are evenly distributed on twelve equally divided points of the inner circle and the outer circle. In addition, the center position is also used as a key position.

[0011] Furthermore, the six-axis inertial sensor integrates an accelerometer and a gyroscope, and can output acceleration and angular velocity data of the finger in three directions, X, Y, and Z; the direction forward along the tip of the index finger is defined as the Y-axis direction, the direction parallel to the palm and perpendicular to the Y-axis is defined as the X-axis direction, and the direction perpendicular to the XY plane is defined as the Z-axis direction. The X-axis, Y-axis, and Z-axis constitute a local three-dimensional coordinate system of the six-axis inertial sensor.

[0012] The present invention also provides a method for using the above-mentioned single-finger virtual keyboard system. Before use, the fingertip of the user wearing the six-axis inertial sensor is used as the center of the circle of the virtual keyboard layout module; when in use, the user refers to the relative position of the center of the circle and the key position in the virtual keyboard layout module, and quickly moves the fingertip wearing the six-axis inertial sensor from the center of the circle to the target key position, and then moves back to the center of the circle to complete a keystroke operation of the target key, and the moving process does not require pause.

[0013] The present invention also provides an input method of the single-finger virtual keyboard system, comprising the following steps:

[0014] 1) The accelerometer and gyroscope in the six-axis inertial sensor collect the acceleration and angular velocity of the finger movement, and send them to the PC host computer through the microprocessor for processing;

[0015] 2) The PC software installed on the PC host computer processes the data. The steps are as follows:

[0016] 2.1) Data preprocessing: The acceleration and angular velocity data sent by the microprocessor are respectively subjected to fixed deviation removal and smoothing filtering to remove noise, thereby obtaining the processed acceleration and angular velocity data;

[0017] 2.2) Motion segmentation: The processed acceleration and angular velocity data are used with the acceleration standard deviation as motion features to obtain the start and end times of the keystroke motion;

[0018] 2.3) Calculate the target displacement: The acceleration and angular velocity data between the start time and the end time of the keystroke movement are integrated using the integral error control algorithm to obtain the target displacement;

[0019] 2.4) Obtain the target key: According to the relative position of the center of the virtual keyboard layout module and the key position, the target displacement is compared with the relative position to obtain the position of the key pressed by the user, and then the key value of the target key pressed by the user is obtained according to the relationship between the position and the key value in the virtual keyboard layout module.

[0020] Further, in step 2.1), in a stationary state, the acceleration data and angular velocity data under more than 4 different postures are sampled by a six-axis inertial sensor, and a spherical surface is fitted to the acceleration data and the angular velocity data using the least square method, and the center of the spherical surface is the fixed deviation of the acceleration and the angular velocity respectively;

[0021] The fixed deviation only needs to be estimated once before the system is put into operation, and does not need to be estimated again after the system is put into operation;

[0022] The filtering adopts a sliding window smoothing filtering method with a fixed length of N, and averages the current data point and N-1 data before the current data point, and the obtained average value is used as the filtering value of the current data point.

[0023] Further, in step 2.2), the keystroke motion start time is the time when the user's finger is located at the center of the circle and is ready to move to the target key; the keystroke motion end time is the time when the user's finger returns from the target key to the center of the circle and the user's finger is located at the center of the circle;

[0024] When the keystroke motion starts, the user's finger changes from a static state to a dynamic state; when the keystroke motion ends, the user's finger changes from a dynamic state to a static state;

[0025] Using the acceleration standard deviation as the motion feature, two thresholds are considered: the start motion threshold and the end motion threshold. In order to prevent the system from misjudging the start of a keystroke due to a slight hand shake when the user is still, and to prevent the system from misjudging the end of a keystroke due to a brief hand pause during the motion, the start motion threshold is set to be greater than the end motion threshold.

[0026] The initial state of the user's finger is static; the state transition process is as follows:

[0027] If the current state is static and the acceleration standard deviation does not exceed the threshold for starting motion, the current state is determined to be static;

[0028] If the current state is static and the acceleration standard deviation is greater than the threshold for starting motion, the static state is changed to a moving state, and the current moment is recorded as the start moment of the keystroke motion;

[0029] If the current state is a moving state and the acceleration standard deviation is less than the end-of-movement threshold, the moving state is changed to a stationary state, and the current moment is recorded as the end-of-keystroke movement moment.

[0030] Further, in step 2.3), the target displacement is calculated, including the following steps:

[0031] 2.3.1) The angular velocity between the start time and the end time of the keystroke motion is obtained by using the Runge-Kutta iteration method to obtain the attitude quaternion at each moment between the start time and the end time of the keystroke motion;

[0032] 2.3.2) Multiply the attitude quaternion at each moment between the start time and the end time of the keystroke movement by the acceleration at the same moment, and then multiply it by the inverse of the attitude quaternion at the same moment to obtain the converted acceleration at each moment;

[0033] 2.3.3) Integrate the converted acceleration between the start time of the keystroke movement and the end time of the keystroke movement using an integral error control algorithm to obtain the speed at each moment;

[0034] 2.3.4) Integrate the velocity between the start time and the end time of the keystroke motion using the integral error control algorithm to obtain the displacement at each moment;

[0035] 2.3.5) For each displacement between the start and end of the keystroke movement, find the displacement with the longest modulus, which is the target displacement of the user's finger from the center of the circle to the target key.

[0036] Furthermore, the integral error control algorithm is based on two facts: the motion state at the starting moment and the ending moment is known, and the data at each moment has errors, and the errors increase with the increase of the number of integration times. Therefore, the integral error control algorithm first iteratively calculates the forward integral of the data at each moment from the starting moment to the ending moment, and then iteratively calculates the backward integral of the data at each moment from the ending moment to the starting moment, and finally linearly weights the forward integral and backward integral of the data at each moment to obtain the final integral at each moment.

[0037] Further, in step 2.4), obtaining the target key includes the following steps:

[0038] 2.4.1) Taking the center of the circle in the virtual keyboard layout module as the origin, 12 direction vectors are obtained, and the virtual keyboard layout module is divided into three areas: center, inner circle and outer circle according to the radius length;

[0039] 2.4.2) Calculate the similarity between the target displacement and the 12 direction vectors using vector cosine to obtain the direction of the target key;

[0040] 2.4.3) Calculate the modulus of the target displacement and compare it with the three areas to get the area where the target button is located;

[0041] 2.4.4) By combining the direction of the target key and the area where the target key is located, the position of the target key in the virtual keyboard layout module can be determined, and then the key value of the target key pressed by the user can be obtained based on the relationship between the position and the key value in the virtual keyboard layout module.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] 1. The twelve-equally divided circular virtual keyboard designed by the present invention can be operated by the user with only a single finger, and the keystroke process has low restrictions on hand movements, which can meet the user's text input needs.

[0044] 2. The integral error control algorithm designed in the present invention reduces the error of displacement calculation and improves the recognition accuracy of system keystrokes.

[0045] 3. The present invention performs displacement calculation based on the acceleration and angular velocity collected by the six-axis inertial sensor to determine the target key pressed by the user. No user data is required for training, which improves the scalability of the system, makes the keyboard layout design more flexible and diverse, and broadens the application scenarios of the system.

[0046] 4. The sensor used in the present invention only requires a six-axis inertial sensor. The system is simple and easy to build and is not easily affected by the environment, which expands the scope of use scenarios of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The schematic diagram of the virtual keyboard key layout module of the present invention. The letters A to Z in the figure correspond to the key values ​​A to Z respectively; the numbers 0 to 9 in the figure correspond to the key values ​​0 to 9 respectively; the "," in the figure indicates that the key value is a comma; the "." in the figure indicates that the key value is a decimal point symbol; the "NUM" in the figure indicates the number lock key; the "..." in the figure indicates the more symbol key; the "Tab" in the figure indicates the tabulation key; the "Capslock" in the figure indicates the caps lock key; the "Shift" in the figure indicates the upshift key; the "Ctrl" in the figure indicates the control key; the "Alt" in the figure indicates the alternating shift key; the "Space" in the figure indicates the space bar; the "Enter" in the figure indicates the enter key; and the "Backspace" in the figure indicates the backspace key.

[0048] Figure 2 This is a front view of the installation of the six-axis inertial sensor of the present invention.

[0049] Figure 3 It is a flow chart of the virtual keyboard input method of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with specific embodiments.

[0051] This embodiment discloses a one-finger virtual keyboard system based on a six-axis inertial sensor, and designs a twelve-equally divided concentric circle virtual keyboard layout module, which enables users to operate with one hand and meets daily text input needs. Considering the error accumulation problem of displacement integration, an integral error control algorithm is designed according to the characteristics of the integral error, which reduces the error of displacement calculation, improves the accuracy of key recognition, and makes the system usable as a whole. The one-finger virtual keyboard system consists of a virtual keyboard layout module, a six-axis inertial sensor, a microprocessor, and a PC host computer;

[0052] The virtual keyboard layout module is used to display key positions and key values ​​to the user;

[0053] The six-axis inertial sensor has one and is installed at the tip of the index finger of one hand, and is used to measure the acceleration and angular velocity data when the finger moves;

[0054] The microprocessor is used to receive the acceleration and angular velocity data of the six-axis inertial sensor, and directly splice the acceleration and angular velocity data and send them to the PC host computer;

[0055] The PC host computer is installed with PC-side software for receiving the acceleration and angular velocity data sent by the microprocessor and processing the acceleration and angular velocity data.

[0056] like Figure 1 As shown, the virtual keyboard layout module consists of an inner circle and an outer circle, which divides the whole into twelve equal parts, divided into 24 key positions, and 1 key position where the center of the circle is located, a total of 25 key positions. In order to expand the number of key values, 3 key values ​​are placed in each of the 12 key positions in the inner circle, and the key value corresponding to the key position is switched by clicking the center position of the circle. In this way, the entire virtual keyboard can accommodate a total of 48 key values ​​​​except the center of the circle, which can meet daily text input needs.

[0057] like Figure 2 As shown, Figure 2 Sensor 1 is a six-axis inertial sensor that integrates an accelerometer and a gyroscope and can output acceleration and angular velocity data of the finger in three directions: X, Y, and Z. The direction along the tip of the index finger is defined as the Y-axis direction, the direction parallel to the palm and perpendicular to the Y-axis is defined as the X-axis direction, and the direction perpendicular to the XY plane is defined as the Z-axis direction. The X-axis, Y-axis, and Z-axis form a local three-dimensional coordinate system of the six-axis inertial sensor.

[0058] Before using the single-finger virtual keyboard system, the user's fingertip wearing the six-axis inertial sensor is used as the center of the circle of the virtual keyboard layout module; when using it, the user refers to the relative position of the center of the circle and the key position in the virtual keyboard layout module, and quickly moves the fingertip wearing the six-axis inertial sensor from the center of the circle to the target key position, and then moves back to the center of the circle to complete a keystroke operation of the target key, and the movement process does not require pause.

[0059] Figure 3 The virtual keyboard input method flow chart is shown in FIG. The single-finger virtual keyboard input method based on the six-axis inertial sensor includes the following steps:

[0060] 1) The accelerometer and gyroscope in the six-axis inertial sensor collect the acceleration and angular velocity of the finger movement, and send them to the PC host computer through the microprocessor for processing;

[0061] 2) The PC software installed on the PC host computer processes the data. The steps are as follows:

[0062] 2.1) Data preprocessing: The acceleration and angular velocity data sent by the microprocessor are respectively subjected to fixed deviation removal and smoothing filtering to remove noise, thereby obtaining the processed acceleration and angular velocity data;

[0063] 2.2) Motion segmentation: The processed acceleration and angular velocity data are used with the acceleration standard deviation as motion features to obtain the start and end times of the keystroke motion;

[0064] 2.3) Calculate the target displacement: The acceleration and angular velocity data between the start time and the end time of the keystroke movement are integrated using the integral error control algorithm to obtain the target displacement;

[0065] 2.4) Obtain the target key: According to the relative position of the center of the virtual keyboard layout module and the key position, the target displacement is compared with the relative position to obtain the position of the key pressed by the user, and then the key value of the target key pressed by the user is obtained according to the relationship between the position and the key value in the virtual keyboard layout module.

[0066] Specifically, in step 2.1), the fixed deviation estimation method is: in a stationary state, the acceleration data and angular velocity data under more than 4 different postures are sampled by a six-axis inertial sensor, and a spherical surface is fitted to the acceleration data and the angular velocity data using the least squares method, and the center of the spherical surface is the fixed deviation of the acceleration and angular velocity. The fixed deviation only needs to be estimated once before the system is run, and no estimation is required after the system is run. The filtering adopts a sliding window smoothing filtering method with a fixed length of N, and the current data point is averaged with the N-1 data before the current data point, and the average value is used as the filtering value of the current data point.

[0067] Specifically, in step 2.2), the start time of the keystroke movement is the moment when the user's finger is located at the center of the circle and is ready to move to the target key; the end time of the keystroke movement is the moment when the user's finger returns from the target key to the center of the circle and is located at the center of the circle. At the start time of the keystroke movement, the user's finger changes from a static state to a moving state; at the end time of the keystroke movement, the user's finger changes from a moving state to a static state. Using the acceleration standard deviation as the motion feature, two thresholds are considered: a start motion threshold and an end motion threshold; in order to prevent the system from misjudging the keystroke movement as the start time due to slight shaking of the user's hand when the user is static, and to prevent the system from misjudging the keystroke movement as the end time due to a brief pause of the user's hand during the movement, the start motion threshold is specified to be greater than the end motion threshold. The initial state of the user's finger is a static state; the state transition process is as follows:

[0068] If the current state is static and the acceleration standard deviation does not exceed the threshold for starting motion, the current state is determined to be static;

[0069] If the current state is static and the acceleration standard deviation is greater than the threshold for starting motion, the static state is changed to a moving state, and the current moment is recorded as the start moment of the keystroke motion;

[0070] If the current state is a moving state and the acceleration standard deviation is less than the end-of-movement threshold, the moving state is changed to a stationary state, and the current moment is recorded as the end-of-keystroke movement moment.

[0071] Specifically, in step 2.3), the target displacement is calculated, including the following steps:

[0072] 2.3.1) The angular velocity between the start time and the end time of the keystroke motion is obtained by using the Runge-Kutta iteration method to obtain the attitude quaternion at each moment between the start time and the end time of the keystroke motion;

[0073] 2.3.2) Multiply the attitude quaternion at each moment between the start time and the end time of the keystroke movement by the acceleration at the same moment, and then multiply it by the inverse of the attitude quaternion at the same moment to obtain the converted acceleration at each moment;

[0074] 2.3.3) Integrate the converted acceleration between the start time of the keystroke movement and the end time of the keystroke movement using an integral error control algorithm to obtain the speed at each moment;

[0075] 2.3.4) Integrate the velocity between the start time and the end time of the keystroke motion using the integral error control algorithm to obtain the displacement at each moment;

[0076] 2.3.5) For each displacement between the start and end of the keystroke movement, find the displacement with the longest modulus, which is the target displacement of the user's finger from the center of the circle to the target key.

[0077] Specifically, in step 2.3), the integral error control algorithm is based on two facts: the motion state at the starting moment and the ending moment is known, and the data at each moment has errors, and the errors increase with the increase of the number of integration times. Therefore, the integral error control algorithm first iteratively calculates the forward integral of the data at each moment from the starting moment to the ending moment, and then iteratively calculates the backward integral of the data at each moment from the ending moment to the starting moment, and finally linearly weights the forward integral and backward integral of the data at each moment to obtain the final integral at each moment.

[0078] Specifically, in step 2.4), obtaining the target key recognition includes the following steps:

[0079] 2.4.1) Taking the center of the circle in the virtual keyboard layout module as the origin, 12 direction vectors are obtained, and the virtual keyboard layout module is divided into three areas: center, inner circle and outer circle according to the radius length;

[0080] 2.4.2) Calculate the similarity between the target displacement and the 12 direction vectors using vector cosine to obtain the direction of the target key;

[0081] 2.4.3) Calculate the modulus of the target displacement and compare it with the three areas to get the area where the target button is located;

[0082] 2.4.4) By combining the direction of the target key and the area where the target key is located, the position of the target key in the virtual keyboard layout module can be determined, and then the key value of the target key pressed by the user can be obtained based on the relationship between the position and the key value in the virtual keyboard layout module.

[0083] In summary, the present invention mainly designs a single-finger virtual keyboard system composed of a virtual keyboard layout module, a six-axis inertial sensor, a microprocessor, and a PC host computer. The accelerometer and gyroscope in the six-axis inertial sensor collect the acceleration and angular velocity of the finger movement, and send them to the PC host computer through the microprocessor. The PC-side software installed on the PC host computer performs data preprocessing to obtain the processed acceleration and angular velocity data, and then uses the acceleration standard deviation as the motion feature to obtain the start time and the end time of the keystroke movement. Then, the acceleration and angular velocity data between the start time and the end time of the keystroke movement are integrated using the integral error control algorithm to obtain the target displacement. Finally, according to the relative position of the center of the circle of the virtual keyboard layout module and the key position, the target displacement is compared with the relative position to obtain the key position pressed by the user, and then according to the relationship between the position and the key value in the virtual keyboard layout module, the target key value pressed by the user is obtained.

[0084] The above-described embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A single-finger virtual keyboard system, characterized in that: The single-finger virtual keyboard system consists of a virtual keyboard layout module, a six-axis inertial sensor, a microprocessor, and a PC host computer; The virtual keyboard layout module is used to display key positions and key values ​​to the user; The six-axis inertial sensor has one and is installed at the tip of the index finger of one hand, and is used to measure the acceleration and angular velocity data when the finger moves; The microprocessor is used to receive the acceleration and angular velocity data of the six-axis inertial sensor, and directly splice the acceleration and angular velocity data and send them to the PC host computer; The PC host computer is installed with PC software for receiving the acceleration and angular velocity data sent by the microprocessor and processing the acceleration and angular velocity data; The virtual keyboard layout module is composed of an inner circle and an outer circle with the same center position and different radii, and all key positions are evenly distributed on twelve equally divided points of the inner circle and the outer circle. In addition, the center position of the circle is also used as a key position; The method for using the single-finger virtual keyboard system is as follows: before use, the fingertip of the user wearing the six-axis inertial sensor is used as the center of the circle of the virtual keyboard layout module; when using, the user refers to the relative position of the center of the circle and the key position in the virtual keyboard layout module, quickly moves the fingertip wearing the six-axis inertial sensor from the center of the circle to the target key position, and then moves back to the center of the circle to complete a keystroke operation of the target key, and the moving process does not need to pause; The input method of the single-finger virtual keyboard system comprises the following steps: 1) The accelerometer and gyroscope in the six-axis inertial sensor collect the acceleration and angular velocity of the finger movement, and send them to the PC host computer through the microprocessor for processing; 2) The PC software installed on the PC host computer processes the data. The steps are as follows: 2.1) Data preprocessing: The acceleration and angular velocity data sent by the microprocessor are respectively subjected to fixed deviation removal and smoothing filtering to remove noise, and the processed acceleration and angular velocity data are obtained; 2.2) Motion segmentation: The processed acceleration and angular velocity data are used with the acceleration standard deviation as motion features to obtain the start and end times of the keystroke motion; 2.3) Calculate the target displacement: The acceleration and angular velocity data between the start time and the end time of the keystroke movement are integrated using the integral error control algorithm to obtain the target displacement, including the following steps: 2.3.1) The angular velocity between the start time and the end time of the keystroke motion is obtained by using the Runge-Kutta iteration method to obtain the attitude quaternion at each moment between the start time and the end time of the keystroke motion; 2.3.2) Multiply the attitude quaternion at each moment between the start time and the end time of the keystroke movement by the acceleration at the same moment, and then multiply it by the inverse of the attitude quaternion at the same moment to obtain the converted acceleration at each moment; 2.3.3) Integrate the converted acceleration between the start time and the end time of the keystroke motion using the integral error control algorithm to obtain the speed at each moment; 2.3.4) Integrate the velocity between the start and end of the keystroke motion using an integral error control algorithm to obtain the displacement at each moment; 2.3.5) For each displacement between the start and end of the keystroke movement, find the displacement with the longest modulus, which is the target displacement of the user's finger from the center of the circle to the target key; The integral error control algorithm is based on two facts: the motion state at the start and end times is known, and the data at each time has errors, and the errors increase with the increase of the number of integration times. Therefore, the integral error control algorithm first iteratively calculates the forward integral of the data at each time from the start time to the end time, and then iteratively calculates the backward integral of the data at each time from the end time to the start time, and finally linearly weights the forward integral and the backward integral of the data at each time to obtain the final integral at each time; 2.4) Get the target key: According to the relative position of the center of the virtual keyboard layout module and the key position, use the target displacement to compare with the relative position to obtain the position of the key pressed by the user, and then according to the relationship between the position and the key value in the virtual keyboard layout module, get the key value of the target key pressed by the user.

2. A single-finger virtual keyboard system according to claim 1, characterized in that: The six-axis inertial sensor integrates an accelerometer and a gyroscope, and can output acceleration and angular velocity data of the finger in three directions, X, Y, and Z. The direction forward along the tip of the index finger is defined as the Y-axis direction, the direction parallel to the palm and perpendicular to the Y-axis is defined as the X-axis direction, and the direction perpendicular to the XY plane is defined as the Z-axis direction. The X-axis, Y-axis, and Z-axis constitute a local three-dimensional coordinate system of the six-axis inertial sensor.

3. The one-finger virtual keyboard system according to claim 1, characterized in that: In step 2.1), in a stationary state, the acceleration data and angular velocity data under more than 4 different postures are sampled by a six-axis inertial sensor, and a spherical surface is fitted to the acceleration data and the angular velocity data using the least square method, and the center of the spherical surface is the fixed deviation of the acceleration and the angular velocity respectively; The fixed deviation only needs to be estimated once before the system is put into operation, and does not need to be estimated again after the system is put into operation; The filtering adopts a sliding window smoothing filtering method with a fixed length of N, and averages the current data point and N-1 data before the current data point, and the obtained average value is used as the filtering value of the current data point.

4. The one-finger virtual keyboard system according to claim 1, characterized in that: In step 2.2), the keystroke motion starts when the user's finger is at the center of the circle and is ready to move toward the target key; the keystroke motion ends when the user's finger returns from the target key to the center of the circle and is at the center of the circle; When the keystroke motion starts, the user's finger changes from a static state to a dynamic state; when the keystroke motion ends, the user's finger changes from a dynamic state to a static state; Using the acceleration standard deviation as the motion feature, two thresholds are considered: the start motion threshold and the end motion threshold. In order to prevent the system from misjudging the start of a keystroke due to a slight hand shake when the user is still, and to prevent the system from misjudging the end of a keystroke due to a brief hand pause during the motion, the start motion threshold is set to be greater than the end motion threshold. The initial state of the user's finger is static; the state transition process is as follows: If the current state is static and the acceleration standard deviation does not exceed the threshold for starting motion, the current state is determined to be static; If the current state is static and the acceleration standard deviation is greater than the threshold for starting motion, the static state is changed to a moving state, and the current moment is recorded as the start moment of the keystroke motion; If the current state is a moving state and the acceleration standard deviation is less than the end-of-movement threshold, the moving state is changed to a stationary state, and the current moment is recorded as the end-of-keystroke movement moment.

5. The one-finger virtual keyboard system according to claim 1, characterized in that: In step 2.4), obtaining the target key includes the following steps: 2.4.1) Taking the center of the circle in the virtual keyboard layout module as the origin, 12 direction vectors are obtained, and the virtual keyboard layout module is divided into three areas: center, inner circle and outer circle according to the radius length; 2.4.2) Calculate the similarity between the target displacement and the 12 direction vectors using vector cosine to obtain the direction of the target button; 2.4.3) Calculate the modulus of the target displacement and compare it with the three areas to get the area where the target button is located; 2.4.4) By combining the direction of the target key and the area where the target key is located, the position of the target key in the virtual keyboard layout module can be determined, and then the key value of the target key pressed by the user can be obtained based on the relationship between the position and the key value in the virtual keyboard layout module.

Citation Information

Patent Citations

  • Virtual keyboard input method and device, and terminal

    CN106951167A

  • MYO-based text input system and method

    CN110362190A