A method and system for controlling a cursor based on a smart ring

Abstract

The application provides a method and system for controlling a cursor based on a smart ring, the method comprising: acquiring an IMU signal collected by a smart ring worn on a specified finger of a user; estimating an attitude angle of the smart ring based on the IMU signal; processing the IMU signal by using a contact state classifier to identify a contact state between the specified finger and a surface; processing the attitude angle and the IMU signal by using a speed predictor to predict an initial fingertip moving speed and a knuckle moving speed of the specified finger; correcting the initial fingertip moving speed according to the knuckle moving speed and a final fingertip moving speed of the specified finger obtained by correcting a previous frame to obtain a final fingertip moving speed of a current frame; and when the contact state indicates that the specified finger is in contact with the surface, controlling a cursor on a graphical interface to move by using the final fingertip moving speed and a speed direction thereof. A user can control a cursor at any time and anywhere by wearing a smart ring on a finger, thereby improving user experience.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method and system for controlling a cursor based on a smart ring. Background Technology

[0002] When controlling AR / VR head-mounted displays or computer graphical interfaces, users typically use physical devices such as mice, touchpads, or controllers to control the cursor. However, these physical devices are relatively large and inconvenient to carry, making it impossible for users to control the cursor anytime, anywhere, resulting in a poor user experience. Therefore, there is an urgent need for a method that allows users to conveniently control the cursor in a graphical interface anytime, anywhere. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method and system for controlling a cursor based on a smart ring, so that users can control the cursor in the graphical interface anytime and anywhere.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0005] The first aspect of this invention discloses a method for controlling a cursor based on a smart ring, the method comprising:

[0006] Acquire inertial accelerometer (IMU) signals collected by at least one smart ring worn on a designated finger of the user;

[0007] The IMU signal is preprocessed, and the attitude angle of the smart ring is estimated based on the preprocessed IMU signal;

[0008] The preprocessed IMU signal is processed using a preset contact state classifier to identify the contact state between the specified finger and the surface.

[0009] The attitude angle and the preprocessed IMU signal are processed using a preset velocity predictor to predict the initial fingertip movement velocity and the joint movement velocity of the finger joints other than the fingertip in the current frame.

[0010] Based on the knuckle movement speed in the current frame and the final fingertip movement speed of the specified finger obtained from the previous frame, the initial fingertip movement speed in the current frame is corrected to obtain the final fingertip movement speed in the current frame.

[0011] When the contact state indicates that the specified finger is in contact with the surface, the cursor movement on the graphical interface is controlled by the final fingertip movement speed and its direction in the current frame.

[0012] Preferably, the initial fingertip movement speed in the current frame is corrected to obtain the final fingertip movement speed in the current frame based on the knuckle movement speed in the current frame and the final fingertip movement speed of the specified finger obtained in the previous frame, including:

[0013] Based on the knuckle movement speed and the initial fingertip movement speed in the current frame, a first weight parameter, a second weight parameter, and a third weight parameter are calculated, wherein the first weight parameter is used to measure the consistency of the velocity projection direction, the second weight parameter is used to measure coplanarity, and the third weight parameter is used to measure length consistency.

[0014] Calculate the smoothing coefficient based on the first weight parameter, the second weight parameter, and the third weight parameter;

[0015] The final fingertip movement speed of the current frame is determined by using the smoothing coefficient, the initial fingertip movement speed of the current frame, and the final fingertip movement speed of the specified finger obtained by correction in the previous frame.

[0016] Preferably, the IMU signal is preprocessed, and the attitude angle of the smart ring is estimated based on the preprocessed IMU signal, including:

[0017] Filter out high-frequency components in the IMU signal;

[0018] The attitude angle of the smart ring is estimated based on the IMU signal after filtering out high-frequency components.

[0019] Preferably, the designated finger is the index finger;

[0020] Acquiring inertial accelerometer (IMU) signals from at least one smart ring, including:

[0021] Acquire inertial accelerometer (IMU) signals from one or two smart rings worn on the user's index finger.

[0022] Preferred options also include:

[0023] When the contact state does not indicate that the specified finger is in contact with the surface, the cursor on the control graphical interface remains stationary.

[0024] A second aspect of this invention discloses a system for controlling a cursor based on a smart ring, the system comprising:

[0025] An acquisition unit is used to acquire inertial accelerometer (IMU) signals collected by at least one smart ring, which is worn on a designated finger of the user.

[0026] The processing unit is used to preprocess the IMU signal and estimate the attitude angle of the smart ring based on the preprocessed IMU signal.

[0027] The identification unit is used to process the preprocessed IMU signal using a preset contact state classifier to identify the contact state between the specified finger and the surface.

[0028] The prediction unit is used to process the attitude angle and the preprocessed IMU signal using a preset velocity predictor to predict the initial fingertip movement speed and the joint movement speed of the finger joints other than the fingertip in the current frame.

[0029] The correction unit is used to correct the initial fingertip movement speed of the current frame based on the knuckle movement speed of the current frame and the final fingertip movement speed of the specified finger obtained by correction in the previous frame, so as to obtain the final fingertip movement speed of the current frame.

[0030] The control unit is used to control the movement of the cursor on the graphical interface by utilizing the final fingertip movement speed and its direction in the current frame when the contact state indicates that the designated finger is in contact with the surface.

[0031] Preferably, the correction unit includes:

[0032] The first calculation module is used to calculate a first weight parameter, a second weight parameter, and a third weight parameter based on the knuckle movement speed and the initial fingertip movement speed in the current frame. The first weight parameter is used to measure the consistency of the velocity projection direction, the second weight parameter is used to measure coplanarity, and the third weight parameter is used to measure length consistency.

[0033] The second calculation module is used to calculate the smoothing coefficient based on the first weight parameter, the second weight parameter, and the third weight parameter;

[0034] The determining module is used to determine the final fingertip movement speed of the current frame using the smoothing coefficient, the initial fingertip movement speed of the current frame, and the final fingertip movement speed of the specified finger obtained by correction in the previous frame.

[0035] Preferably, the processing unit is specifically used to: filter out high-frequency components in the IMU signal; and estimate the attitude angle of the smart ring based on the IMU signal with the high-frequency components filtered out.

[0036] Preferably, the designated finger is the index finger; the acquisition unit is specifically used to: acquire the inertial accelerometer (IMU) signal collected by one or two smart rings worn on the user's index finger.

[0037] Preferably, the control unit is further configured to: keep the cursor on the graphical interface stationary when the contact state does not indicate that the designated finger is in contact with the surface.

[0038] Based on the above embodiments of the present invention, a method and system for controlling a cursor based on a smart ring are provided. The method includes: acquiring IMU signals collected by at least one smart ring, with at least one smart ring worn on a designated finger of the user; preprocessing the IMU signals and estimating the attitude angle of the smart ring based on the preprocessed IMU signals; processing the preprocessed IMU signals using a contact state classifier to identify the contact state between the designated finger and the surface; processing the attitude angle and the preprocessed IMU signals using a velocity predictor to predict the initial fingertip movement velocity and knuckle movement velocity of the designated finger in the current frame; correcting the initial fingertip movement velocity of the current frame based on the knuckle movement velocity of the current frame and the final fingertip movement velocity of the designated finger obtained in the previous frame to obtain the final fingertip movement velocity of the current frame; and controlling the cursor movement on the graphical interface using the final fingertip movement velocity and its velocity direction when the contact state indicates that the designated finger is in contact with the surface. In this solution, a smart ring is used to collect IMU signals, and a contact state classifier and a velocity predictor are used to process the IMU signals to determine the contact state and the final fingertip movement speed. When the contact state indicates that the specified finger is in contact with the surface, the final fingertip movement speed and its direction are used to control the cursor movement on the graphical interface. Users can control the cursor on the graphical interface anytime and anywhere by wearing the smart ring on their finger, improving the user experience. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 A flowchart of a method for controlling a cursor based on a smart ring, provided as an embodiment of the present invention;

[0041] Figure 2 A flowchart for determining the final fingertip movement speed of the current frame, provided for embodiments of the present invention;

[0042] Figure 3 This is a structural block diagram of a system for controlling a cursor based on a smart ring, provided as an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] As can be seen from the background technology, when users control AR / VR head-mounted display devices or computer graphical interfaces, they typically use physical devices such as mice, touchpads, or controllers to control the cursor in the graphical interface. However, the aforementioned physical devices are relatively large and inconvenient to carry, which prevents users from controlling the cursor in the graphical interface anytime and anywhere, resulting in a poor user experience.

[0046] To enable users to control the cursor in a graphical interface anytime and anywhere, in scenarios such as user movement, interaction with a large screen, wearing AR / VR head-mounted display devices, and interaction with distant computer devices, this solution proposes a method and system for controlling the cursor based on a smart ring. At least one smart ring (one ring is more convenient to carry, while two rings offer higher control precision) is worn on a designated finger (such as the base and middle knuckle of the index finger). Each smart ring contains at least one inertial accelerometer (IMU). After wearing the smart ring, the system identifies the contact state and sliding trajectory of the designated finger on any plane in real time, achieving a touchpad-like interactive input method. This allows users to control the cursor in the graphical interface anytime and anywhere, improving the user experience. The following detailed description of the invention is provided through various embodiments.

[0047] See Figure 1 The flowchart illustrates a method for controlling a cursor based on a smart ring according to an embodiment of the present invention. The method includes:

[0048] Step S101: Acquire the IMU signal obtained by at least one smart ring.

[0049] It should be noted that at least one smart ring (e.g., a smart ring) is worn on the user's designated finger. In the specific implementation step S101, the IMU signal collected by the at least one smart ring worn on the designated finger is acquired.

[0050] In some embodiments, the finger is designated as the index finger (this is just an example, but it can also be other fingers). In this case, the IMU signal is acquired by acquiring the IMU signal (such as the IMU six-axis signal) collected by one or two smart rings worn on the user's index finger.

[0051] It is understandable that a smart ring contains at least one hardware device capable of reading IMU signals in real time; depending on the number of smart rings worn, there are mainly two wearing methods:

[0052] Wearing Option 1: When wearing one smart ring, place it on the middle joint of the index finger (example only). Wearing Option 2: When wearing two smart rings, place them on the middle and base joints of the index finger, respectively.

[0053] Smart rings can send various signals to remote computers or the computing chip inside the smart ring in real time.

[0054] Step S102: Preprocess the IMU signal and estimate the attitude angle of the smart ring based on the preprocessed IMU signal.

[0055] In the specific implementation step S102, the IMU signal is preprocessed, and the attitude angle of the smart ring is estimated based on the preprocessed IMU signal. The preprocessing includes at least the filtering of high-frequency components.

[0056] In some embodiments, the specific implementation of preprocessing the IMU signal and estimating the smart ring's attitude angle is as follows: filtering out high-frequency components from the IMU signal; estimating the smart ring's attitude angle based on the IMU signal with the high-frequency components removed. Specifically, a low-pass filter can be used to filter out high-frequency components from the IMU signal acquired by the smart ring, and then the Mahony algorithm is used to estimate the smart ring's attitude angle based on the "IMU signal with the high-frequency components removed," where the preprocessed IMU signal is the IMU signal with the high-frequency components removed.

[0057] Understandably, if a user wears two smart rings on a designated finger, a pass filter is used to remove high-frequency components from the IMU signals acquired by the two smart rings. Then, the Mahony algorithm is used to estimate the attitude angle of each smart ring based on the "IMU signals with high-frequency components removed" corresponding to the two smart rings.

[0058] Step S103: Use a preset contact state classifier to process the preprocessed IMU signal to identify the contact state between the specified finger and the surface.

[0059] It should be noted that the contact state classifier is obtained by training a machine learning model (such as an RNN) based on sample data.

[0060] In the specific implementation step S103, the preprocessed IMU signal is used as the input feature vector and input to the contact state classifier. The contact state classifier identifies the contact state between the specified finger and the surface. Specifically, the preprocessed IMU signal within a certain time window (e.g., 0.1s) is used as the input feature vector and input to the contact state classifier. The contact state classifier identifies the contact state between the specified finger and the surface. The identified contact state between the specified finger and the surface can be any of the following states: the specified finger is in contact with the surface, the specified finger is suspended and moving towards the surface, the specified finger is suspended and leaving the surface, or other situations where the specified finger is suspended.

[0061] It should be noted that if one smart ring is worn on a designated finger, the preprocessed IMU signal collected by that smart ring is input into the contact state classifier for contact state identification; if two smart rings are worn on a designated finger, the preprocessed IMU signals collected by both smart rings are input into the contact state classifier for contact state identification.

[0062] Step S104: Use a preset velocity predictor to process the attitude angle and the preprocessed IMU signal to predict the initial fingertip movement velocity of the specified finger in the current frame and the joint movement velocity of the finger joints other than the fingertip.

[0063] It should be noted that the velocity predictor is obtained by training a machine learning model (such as RNN) based on sample data; the specified finger can be divided into the following 4 parts: fingertip, distal interphalangeal joint (DIP), proximal interphalangeal joint (PIP), and metacarpophalangeal joint (MP).

[0064] In the specific implementation of step S104, the estimated attitude angle (obtained from step S102) and the preprocessed IMU signal are used as input feature vectors and input to the velocity predictor for velocity prediction. The predicted initial fingertip movement velocity and the joint movement velocities of the finger joints other than the fingertip are obtained in the current frame. Among them, the joint movement velocities of the finger joints other than the fingertip are the joint movement velocities corresponding to the distal interphalangeal joint, proximal interphalangeal joint, and metacarpophalangeal joint.

[0065] In other words, the velocity predictor can predict the initial fingertip movement velocity, the distal interphalangeal joint movement velocity, the proximal interphalangeal joint movement velocity, and the metacarpophalangeal joint movement velocity of the current frame, that is, a total of 4 velocities can be predicted.

[0066] It should be noted that if a smart ring is worn on a designated finger, the attitude angle of that smart ring and the preprocessed IMU signal are used to input a velocity predictor to predict the movement speed of the fingertip and other finger joints.

[0067] If two smart rings are worn on a designated finger, the attitude angles of smart ring 1 and smart ring 2, along with the preprocessed IMU signals, are concatenated into a longer feature vector. This concatenated feature vector is then input into a velocity predictor to predict the movement speed of the fingertip and other finger joints.

[0068] Step S105: Based on the knuckle movement speed of the current frame and the final fingertip movement speed of the specified finger obtained from the previous frame, correct the initial fingertip movement speed of the current frame to obtain the final fingertip movement speed of the current frame.

[0069] In the specific implementation step S105, after predicting the initial fingertip movement speed and the joint movement speed of other finger joints in the current frame (denoted as frame t), the initial fingertip movement speed of the current frame is corrected based on the joint movement speed of the current frame and the final fingertip movement speed of the specified finger obtained in the previous frame (frame t-1), thereby obtaining the final fingertip movement speed of the current frame.

[0070] Step S106: When the contact state indicates that the specified finger is in contact with the surface, the cursor movement on the graphical interface is controlled by the final fingertip movement speed and its direction in the current frame.

[0071] In the specific implementation of step S106, when the contact state classifier identifies that the finger is in contact with the surface, the cursor on the computer's graphical interface is controlled to move according to the speed and direction of the final fingertip movement speed in the current frame. Specifically, the cursor on the computer's graphical interface can be controlled by sending instructions via Bluetooth communication (only an example) according to the speed and direction of the final fingertip movement speed in the current frame.

[0072] In some embodiments, when the contact state does not indicate that the specified finger is in contact with the surface, the cursor on the graphical interface is kept stationary; that is, when the contact state classifier identifies any of the following states: "the specified finger is suspended and moving toward the surface", "the specified finger is suspended and leaving the surface", or "other situations where the specified finger is suspended", the cursor on the graphical interface is kept stationary.

[0073] It should be noted that in implementing this solution, the signal sensing part (reading and sending IMU signals) can be implemented through the sensors built into the smart ring and signal transmission devices such as Bluetooth. The calculation part can be completed in the computer receiving the signal. If the smart ring has a built-in computing unit, the calculation part can also be completed on the chip inside the smart ring.

[0074] In practical applications, the smart ring's sensors read IMU signals and send them to a computer or a chip inside the smart ring. The computer or chip preprocesses the signals; the contact state and the final fingertip movement speed are obtained through a contact state classifier and a speed predictor. When the contact state classifier identifies that the specified finger is in contact with the surface, the cursor on the computer's graphical interface is moved according to the magnitude and direction of the final fingertip movement speed.

[0075] In this embodiment of the invention, a smart ring is used to collect IMU signals, and a contact state classifier and a velocity predictor are used to process the IMU signals to determine the contact state and the final fingertip movement speed. When the contact state indicates that the specified finger is in contact with the surface, the final fingertip movement speed and its direction are used to control the movement of the cursor on the graphical interface. Users can control the cursor on the graphical interface anytime and anywhere by wearing a smart ring on their finger, improving the user experience.

[0076] Regarding the above embodiments of the present invention Figure 1 Step S105 involves correcting the initial fingertip movement speed of the current frame; see [link / reference]. Figure 2 This illustrates a flowchart of an embodiment of the present invention for determining the final fingertip movement speed of the current frame. Figure 2 Includes the following steps:

[0077] Step S201: Calculate the first weight parameter, the second weight parameter, and the third weight parameter based on the knuckle movement speed and the initial fingertip movement speed in the current frame.

[0078] In the specific implementation step S201, the first weight parameter, the second weight parameter, and the third weight parameter are calculated based on the knuckle movement speed and the initial fingertip movement speed of the current frame. The first weight parameter (denoted as C1) is used to measure the consistency of the velocity projection direction, the second weight parameter (denoted as C2) is used to measure coplanarity, and the third weight parameter (denoted as C3) is used to measure length consistency. The calculation process of the first weight parameter (C1), the second weight parameter (C2), and the third weight parameter (C3) is explained below.

[0079] Explanation regarding the calculation of the first weight parameter (C1):

[0080] The first weighting parameter C1 is used to measure the consistency of the velocity projection direction. The range of C1 is 0-1. C1=1 means that the projection direction is completely consistent. The closer C1 is to 1, the closer the projection direction is. Conversely, the closer C1 is to 0, the less close the projection direction is. Specifically, C1 can be calculated by formulas (1)-(4).

[0081]

[0082]

[0083]

[0084]

[0085] In formulas (1)-(4), The projection velocity of the fingertip onto the interaction plane (horizontal or vertical). The projection velocity of the distal interphalangeal joint onto the interaction plane. The projection velocity of the proximal interphalangeal joint (PIP) onto the interaction plane; For example, when i is 1, 2, or 3, The temperatures are 13 degrees, 15 degrees, and 30 degrees respectively.

[0086] As can be seen from the above formulas (1)-(4), the specific calculation method of C1 is as follows: first, calculate the angle between the projected velocities of adjacent joints using the arccos function between the projected velocity vectors, and then perform a weighted summation of the angle between the projected velocities of adjacent joints; during the weighted summation process, if the angle α is less than a threshold alpha, then add 1 to C1, and if it is greater than alpha, then add cos(α)*cos(α) to C1.

[0087] It should be noted that the projection velocity is calculated as follows: the normal direction of the interaction plane is known (horizontal or vertical), and the velocity vector of the joint is predicted by the velocity predictor. Therefore, the projection velocity on the interaction plane can be obtained.

[0088] The above is an explanation of how to calculate the first weight parameter C1.

[0089] Explanation regarding the calculation of the second weighting parameter (C2):

[0090] The second weighting parameter C2 is used to measure coplanarity. C2 ranges from 0 to 1, and the closer C2 is to 1, the better the coplanarity. The method for calculating the normal velocity of each joint is as follows: the direction vectors of the two bones of the smart ring can be obtained through the IMU attitude angle estimation algorithm, thereby determining the plane formed by the bones of the specified finger (such as the index finger); based on the movement velocity of each joint predicted by the velocity predictor (initial fingertip movement velocity and finger joint movement velocity other than the fingertip), and the plane formed by the bones of the specified finger, the normal velocity of the joint relative to the plane of the specified finger can be obtained.

[0091] The second weight parameter C2 is equal to the vector product of the normal velocity differences of three pairs of adjacent joints (DIP and PIP, PIP and MP, DIP and nailtip), which can be calculated by formula (5).

[0092]

[0093] In formula (5), This represents the normal velocity of the distal interphalangeal joint (DIP) relative to the bone it is attached to. This represents the normal velocity of the proximal interphalangeal joint (PIP) relative to the bone it is attached to. The normal velocity of the metacarpophalangeal joint (MP) relative to the bone it is attached to. This represents the normal velocity of the fingertip relative to the bone it is attached to.

[0094] The above is an explanation of how to calculate the second weighting parameter C2.

[0095] Explanation regarding the calculation of the third weighting parameter (C3):

[0096] The third weighting parameter C3 is used to measure length consistency. C3 is calculated as follows: The ratio of the radial velocities of joints (MP and DIP, DIP and PIP) on the same bone is calculated. If this ratio is greater than 1, the reciprocal is taken. Then, the two ratios are multiplied by 1 / 2 and added together. The radial velocity of a joint is calculated as follows: Using the IMU attitude angle estimation algorithm, the unit direction vector of each bone can be obtained. The radial velocity of the joint is obtained by multiplying the velocity vector predicted by the velocity predictor with this unit direction vector.

[0097] Specifically, C3 can be calculated using formula (6).

[0098]

[0099] In formula (6), L1 and L2 are the direction vectors of the proximal phalanx and the intermediate phalanx, respectively. The tangential velocity of the metacarpophalangeal joint (MP) relative to the proximal phalanx is denoted as . The tangential velocity of the distal interphalangeal joint (DIP) relative to the proximal phalanx. The tangential velocity of the distal interphalangeal joint (DIP) relative to the middle phalanx. The tangential velocity of the proximal interphalangeal joint (PIP) relative to the middle phalanx.

[0100] The above is an explanation of how to calculate the third weighting parameter C3.

[0101] Step S202: Calculate the smoothing coefficient based on the first weight parameter, the second weight parameter, and the third weight parameter.

[0102] In the specific implementation of step S202, based on the first weight parameter, the second weight parameter and the third weight parameter calculated in step S201 above, a smoothing coefficient (denoted as C) is calculated. The smoothing coefficient C is in the range of 0-1. Specifically, the smoothing coefficient C is calculated by formula (7).

[0103]

[0104] Step S203: Determine the final fingertip movement speed of the current frame using the smoothing coefficient, the initial fingertip movement speed of the current frame, and the final fingertip movement speed of the specified finger obtained from the correction in the previous frame.

[0105] In the specific implementation of step S203, the final fingertip movement speed of the current frame is determined by using the smoothing coefficient C, the initial fingertip movement speed of the current frame (frame t), and the final fingertip movement speed of the specified finger obtained by the correction of the previous frame (frame t-1); specifically, the final fingertip movement speed of frame t is determined by formula (8).

[0106]

[0107] In formula (8), C is the smoothing coefficient. Let be the final fingertip movement speed in frame t (equivalent to the final output speed). Let $\frac{ ... The final fingertip movement speed is t-1.

[0108] As can be seen from the content of the above formula (8), this scheme reduces the prediction weight of the velocity predictor by using the smoothing coefficient C, and uses the final fingertip movement speed in the (t-1)th frame for smoothing and supplementation.

[0109] Based on the above information regarding the first weight parameter C1, the second weight parameter C2, the third weight parameter C3, and the smoothing coefficient C, this scheme sets the following three physical constraints based on the first weight parameter C1, the second weight parameter C2, and the third weight parameter C3:

[0110] Physical constraints (1) Whether the velocity projection directions of adjacent joints on the plane are close.

[0111] Physical constraints (2), whether the fingertip and the three joints are coplanar.

[0112] Physical constraints (3) Whether the radial velocities of adjacent joints relative to the phalanges are consistent.

[0113] After the initial fingertip movement speed of the current frame is predicted by the velocity predictor, if the above physical constraints (1) to physical constraints (3) are not met, the prediction weight of the velocity predictor is reduced by the smoothing coefficient C. This is equivalent to correcting the prediction result of the velocity predictor by the smoothing coefficient C, thereby achieving high-precision finger contact detection and position tracking.

[0114] The above embodiments of the present invention Figure 1 and Figure 2 This is a detailed description of the plan, through Figure 1 and Figure 2 As can be seen from the content, this solution has the following advantages:

[0115] This solution achieves cursor control by wearing one or two smart rings on designated fingers, replacing traditional touchpads and mice; this solution achieves high-precision finger contact detection and position tracking through machine learning models and physical constraints.

[0116] Corresponding to the method for controlling a cursor based on a smart ring provided in the above embodiments of the present invention, see also... Figure 3 The present invention also provides a structural block diagram of a system for controlling a cursor based on a smart ring. The system includes: an acquisition unit 301, a processing unit 302, a recognition unit 303, a prediction unit 304, a correction unit 305, and a control unit 306.

[0117] Acquisition unit 301 is used to acquire IMU signals collected by at least one smart ring, wherein at least one smart ring is worn on a designated finger of the user.

[0118] In some embodiments, the finger is designated as the index finger; the acquisition unit 301 is specifically used to: acquire the IMU signal collected by one or two smart rings worn on the user's index finger.

[0119] The processing unit 302 is used to preprocess the IMU signal and estimate the attitude angle of the smart ring based on the preprocessed IMU signal.

[0120] In a specific implementation, the processing unit 302 is specifically used to: filter out high-frequency components in the IMU signal; and estimate the attitude angle of the smart ring based on the IMU signal with the high-frequency components filtered out.

[0121] The identification unit 303 is used to process the pre-processed IMU signal using a preset contact state classifier to identify the contact state between a specified finger and a surface.

[0122] The prediction unit 304 is used to process the attitude angle and the preprocessed IMU signal using a preset velocity predictor to predict the initial fingertip movement speed of a specified finger in the current frame and the joint movement speed of the finger joints other than the fingertip.

[0123] The correction unit 305 is used to correct the initial fingertip movement speed of the current frame based on the knuckle movement speed of the current frame and the final fingertip movement speed of the specified finger obtained from the correction in the previous frame, so as to obtain the final fingertip movement speed of the current frame.

[0124] The control unit 306 is used to control the movement of the cursor on the graphical interface by utilizing the final fingertip movement speed and its direction in the current frame when the contact state indicates that a specified finger is in contact with the surface.

[0125] In specific implementation, the control unit 306 is used to keep the cursor on the graphical interface stationary when the contact state does not indicate that the specified finger is in contact with the surface.

[0126] In this embodiment of the invention, a smart ring is used to collect IMU signals, and a contact state classifier and a velocity predictor are used to process the IMU signals to determine the contact state and the final fingertip movement speed. When the contact state indicates that the specified finger is in contact with the surface, the final fingertip movement speed and its direction are used to control the movement of the cursor on the graphical interface. Users can control the cursor on the graphical interface anytime and anywhere by wearing a smart ring on their finger, improving the user experience.

[0127] Preferred, combined Figure 3 The modified unit 305, as shown, includes a first calculation module, a second calculation module, and a determination module; the execution principle of each module is as follows:

[0128] The first calculation module is used to calculate a first weight parameter, a second weight parameter, and a third weight parameter based on the knuckle movement speed and the initial fingertip movement speed of the current frame. The first weight parameter is used to measure the consistency of the velocity projection direction, the second weight parameter is used to measure coplanarity, and the third weight parameter is used to measure length consistency.

[0129] The second calculation module is used to calculate the smoothing coefficient based on the first weight parameter, the second weight parameter, and the third weight parameter.

[0130] The determination module is used to determine the final fingertip movement speed of the current frame by using the smoothing coefficient, the initial fingertip movement speed of the current frame, and the final fingertip movement speed of the specified finger obtained by correction in the previous frame.

[0131] In summary, this invention provides a method and system for controlling a cursor based on a smart ring. The smart ring collects IMU signals, and a contact state classifier and a velocity predictor process the IMU signals to determine the contact state and the final fingertip movement speed. When the contact state indicates that the specified finger is in contact with a surface, the final fingertip movement speed and its direction are used to control the cursor movement on the graphical interface. Users can control the cursor on the graphical interface anytime and anywhere by wearing a smart ring on their finger, improving the user experience.

[0132] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0133] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0134] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for controlling a cursor based on a smart ring, characterized in that, The method includes: Acquire inertial accelerometer (IMU) signals collected by at least one smart ring worn on a designated finger of the user; The IMU signal is preprocessed, and the attitude angle of the smart ring is estimated based on the preprocessed IMU signal; The preprocessed IMU signal is processed using a preset contact state classifier to identify the contact state between the specified finger and the surface. The attitude angle and the preprocessed IMU signal are processed using a preset velocity predictor to predict the initial fingertip movement velocity and the joint movement velocity of the finger joints other than the fingertip in the current frame. Based on the knuckle movement speed in the current frame and the final fingertip movement speed of the specified finger obtained from the previous frame, the initial fingertip movement speed in the current frame is corrected to obtain the final fingertip movement speed in the current frame. When the contact state indicates that the specified finger is in contact with the surface, the cursor movement on the graphical interface is controlled by the final fingertip movement speed and its direction in the current frame. Based on the knuckle movement speed in the current frame and the final fingertip movement speed of the specified finger obtained from the previous frame, the initial fingertip movement speed in the current frame is corrected to obtain the final fingertip movement speed in the current frame, including: Based on the knuckle movement speed and the initial fingertip movement speed in the current frame, a first weight parameter, a second weight parameter, and a third weight parameter are calculated, wherein the first weight parameter is used to measure the consistency of the velocity projection direction, the second weight parameter is used to measure coplanarity, and the third weight parameter is used to measure length consistency. Calculate the smoothing coefficient based on the first weight parameter, the second weight parameter, and the third weight parameter; The final fingertip movement speed of the current frame is determined by using the smoothing coefficient, the initial fingertip movement speed of the current frame, and the final fingertip movement speed of the specified finger obtained by correction in the previous frame. The first weight parameter is calculated as follows: first, the angle between the projected velocities of adjacent joints is calculated using the arccos function between the projected velocity vectors, and then the angle between the projected velocities of adjacent joints is summed with weights. The second weighting parameter is equal to the vector product of the normal velocity differences of three pairs of adjacent joints; The specific calculation method for the third weighting parameter is as follows: calculate the ratio of the radial velocity of two pairs of joints on the same bone. If the ratio is greater than 1, take the reciprocal. Then multiply the two ratios by 1 / 2 and add them together.

2. The method according to claim 1, characterized in that, The process includes preprocessing the IMU signal and estimating the attitude angle of the smart ring based on the preprocessed IMU signal, including: Filter out high-frequency components in the IMU signal; The attitude angle of the smart ring is estimated based on the IMU signal after filtering out high-frequency components.

3. The method according to any one of claims 1-2, characterized in that, The designated finger is the index finger; Acquiring inertial accelerometer (IMU) signals from at least one smart ring, including: Acquire inertial accelerometer (IMU) signals from one or two smart rings worn on the user's index finger.

4. The method according to any one of claims 1-2, characterized in that, Also includes: When the contact state does not indicate that the specified finger is in contact with the surface, the cursor on the control graphical interface remains stationary.

5. A system for controlling a cursor based on a smart ring, characterized in that, The system is used to implement the method for controlling a cursor based on a smart ring as described in any one of claims 1 to 4, the system comprising: An acquisition unit is used to acquire inertial accelerometer (IMU) signals collected by at least one smart ring, which is worn on a designated finger of the user. The processing unit is used to preprocess the IMU signal and estimate the attitude angle of the smart ring based on the preprocessed IMU signal. The identification unit is used to process the preprocessed IMU signal using a preset contact state classifier to identify the contact state between the specified finger and the surface. The prediction unit is used to process the attitude angle and the preprocessed IMU signal using a preset velocity predictor to predict the initial fingertip movement speed and the joint movement speed of the finger joints other than the fingertip in the current frame. The correction unit is used to correct the initial fingertip movement speed of the current frame based on the knuckle movement speed of the current frame and the final fingertip movement speed of the specified finger obtained by correction in the previous frame, so as to obtain the final fingertip movement speed of the current frame. The control unit is used to control the movement of the cursor on the graphical interface by utilizing the final fingertip movement speed and its direction in the current frame when the contact state indicates that the designated finger is in contact with the surface.

6. The system according to claim 5, characterized in that, The correction unit includes: The first calculation module is used to calculate a first weight parameter, a second weight parameter, and a third weight parameter based on the knuckle movement speed and the initial fingertip movement speed in the current frame. The first weight parameter is used to measure the consistency of the velocity projection direction, the second weight parameter is used to measure coplanarity, and the third weight parameter is used to measure length consistency. The second calculation module is used to calculate the smoothing coefficient based on the first weight parameter, the second weight parameter, and the third weight parameter; The determining module is used to determine the final fingertip movement speed of the current frame using the smoothing coefficient, the initial fingertip movement speed of the current frame, and the final fingertip movement speed of the specified finger obtained by correction in the previous frame.

7. The system according to claim 5, characterized in that, The processing unit is specifically used to: filter out high-frequency components in the IMU signal; and estimate the attitude angle of the smart ring based on the IMU signal after the high-frequency components have been filtered out.

8. The system according to any one of claims 5-7, characterized in that, The designated finger is the index finger; the acquisition unit is specifically used to: acquire the inertial accelerometer (IMU) signal collected by one or two smart rings worn on the user's index finger.

9. The system according to any one of claims 5-7, characterized in that, The control unit is also configured to: keep the cursor on the graphical interface stationary when the contact state does not indicate that the designated finger is in contact with the surface.

Images