Digital pen cap

Abstract

A system and method are provided for a digital pen cap that can be positioned at an end of a stylus. The digital pen cap comprises: a sensor for measuring the position and orientation of the stylus tip; a controller configured to process the sensor data and output the processed data; the memory is used for storing the processed data; and a transmitter for transmitting the processed data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 357,807, filed on July 1, 2022, entitled “Digital Pen Cap,” which is assigned to the assignee of this application and is incorporated herein by reference in its entirety. Technical Field

[0003] The present invention relates to the field of electronic writing and drawing. Background Art

[0004] Electronic writing facilities allow a user to write on a surface in a traditional manner and have the writing digitally recorded or transmitted to a computing device. Storing writing digitally reduces paper storage and facilitates computer organization of various writings. Electronic writing systems typically require a specialized writing surface and / or image capture device to record the writing. In some systems, a pen with a digital camera captures the strokes on a specialized dot paper as an image and sends the image to a host computer for processing. Another system includes infrared and ultrasonic clips for attachment to the paper. Other technologies for digitizing writing, such as optical character recognition (OCR), use offline processing to digitize the image.

[0005] Conventional electronic writing instruments may place specific requirements on a writing instrument, writing surface, and / or computing device. Such conventional electronic writing instruments may not function if the corresponding components are missing or incompatible (e.g., from another manufacturer). Therefore, it is desirable to reduce the number of specialized hardware components in an electronic writing system. Summary of the invention

[0006] A simplified summary of one or more aspects is presented below in order to provide a basic understanding of these aspects. This summary is not an extensive overview of all contemplated aspects, and is neither intended to identify key or important elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to a more detailed description presented later.

[0007] In some aspects, the technology described herein relates to a cap for positioning on an axis to record movement of the axis, comprising: a housing configured to be attached to a distal end of the axis opposite to a writing end of the axis; a plurality of sensors configured to measure a position and orientation of the distal end of the axis and output the position and orientation as sensor data; a controller configured to process the sensor data and output the processed data; a memory configured to store the processed data; and a transmitter configured to transmit the processed data.

[0008] In some aspects, the technology described herein relates to a method of transcribing writing using a writing instrument, the writing instrument having a shaft, the shaft having a writing end and a distal end opposite the writing end, comprising: receiving sensor data, the sensor data comprising multiple samples of the position and orientation of a removable cap, the removable cap comprising multiple inertial sensors attached to the distal end of the shaft; extracting a feature set from the sensor data within a sample window; and classifying the feature set into characters based on a trained classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] For a more complete understanding of the present disclosure and its features and advantages, please refer to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts, and in which:

[0010] Figure 1A-1B A schematic diagram showing a digital pen cap according to some aspects of the present disclosure is shown;

[0011] Figure 2A-2B A block diagram showing the functions and data of a digital pen cap system according to some aspects of the present disclosure;

[0012] Figure 3 Another block diagram of a digital pen cap system according to some aspects of the present disclosure is shown;

[0013] Figures 4A-4D A view showing a digital pen cap according to some aspects of the present disclosure;

[0014] Figure 5 shows a digital pen cap at the end of a stylus according to some aspects of the present disclosure; and

[0015] Figure 6 is a flow chart of an example method of transcribing writing using a writing instrument having a shaft having a writing end and a distal end opposite the writing end. DETAILED DESCRIPTION

[0016] Systems and methods for digital pen caps are disclosed. Conventional electronic writing facilities impose specific requirements on writing implements, writing surfaces, and / or computing devices. These requirements often require two or more devices to operate together. For example, some digital pens may only be used with a specific computing tablet that has a surface (e.g., including a sensor) that interacts with specific functions of the digital pen (e.g., a transmitter). If the corresponding components are missing or incompatible (e.g., from another manufacturer), such conventional electronic writing facilities may not operate. Therefore, there is a need to reduce the number of dedicated hardware components in electronic writing systems.

[0017] In one aspect, a digital pen cap includes one or more inertial sensors for tracking a stroke pattern of an axis. The digital pen cap can be placed at the end of an axis such as a pen, pencil, or stylus, and the inertial sensor tracks the movement of the axis. The movement tracked by the inertial sensor in the pen cap is converted to track the movement of the axis on a writing surface. The movement of the pen cap can be classified according to sub-motions. The digital pen cap includes a sensor, a processor, a transceiver, and a housing module.

[0018] The digital pen cap is located on the shaft and can be any pen, pencil or stylus-like device. The pen cap includes an inertial sensor that captures the user's movements and / or strokes when writing or drawing. The pen cap is designed to recognize patterns and / or strokes and digitize the writing and drawing. The pen cap can wirelessly transmit the recognized patterns and / or strokes to another device, thereby digitizing the writing or drawing made using the shaft to which the pen cap is attached.

[0019] The digital pen cap can be used with a traditional writing implement or a digital pen. In the case of a traditional writing facility, the digital pen cap can effectively create a digital copy of what is written on paper. The digital pen cap does not require any specific writing surface, thereby reducing the compatibility issues of traditional digital writing. In some embodiments, the digital pen cap can operate without any writing surface. In addition, the digital pen cap can communicate with the computing device using an application programming interface (API) that does not require dedicated hardware for the computing device, thereby reducing the compatibility requirements between the digital pen cap and the computing device.

[0020] The digital pen cap system includes an inertial subsystem, a processor subsystem, a communication subsystem, and a memory subsystem. The inertial sensor in the pen cap is located at the end of the shaft of the pen, pencil, or stylus and tracks its movement. The digital pen cap system analyzes the sensor data to recognize patterns and interprets the patterns as letters, characters, words, sentences, or drawings. The pen cap can also identify the distance of the pen tip, pencil, or stylus from the writing surface.

[0021] In some embodiments, the inertial subsystem includes an adjustable mechanism, such as a slider or screw, for adjusting the position and orientation of the inertial sensor. Such an adjustable mechanism allows the user to calibrate and use the digital pen cap regardless of the size of the pen, pencil, or stylus. In addition, the adjustment function allows the user to adjust the position of the sensor for better motion detection.

[0022] In various embodiments, the pen cap includes a switch, a power button for switching between various modes, and function settings, such as writing or drawing. The pen cap may include various indicators, such as red, green, and blue (RGB) light emitting diodes (LEDs) to indicate different modes, signal quality, or signal strength of input data from the sensor. In some examples, the battery compartment is located within the pen cap and the battery can be replaced by the user. In other examples, the battery can be rechargeable and does not need to be replaced. The pen cap can be adjusted to fit on pens, pencils, or styluses of different sizes. In various embodiments, the pen cap determines the distance from the pen cap to the tip of the writing / drawing tool. In some examples, the pen cap determines the distance to the tip of the writing / drawing tool based on sensor data and an algorithm.

[0023] Figure 1A A front view 100 of an example pen cap 102 according to various aspects of the present disclosure is shown. The pen cap 102 includes a sensor 104, a first LED 106, a second LED 108, a switch 110, a battery 112, and an elongated hole 114. In various examples, the sensor 104 is an inertial sensor board and is configured to detect inertial motion of the pen cap 102. In some examples, the sensor 104 includes an accelerometer and a gyroscope. The accelerometer detects a rate of change of velocity of the sensor 104 and a change in velocity of motion of the sensor 104. The gyroscope detects an orientation of the sensor 104, a change in orientation of the sensor 104, and an angular velocity of the sensor 104. In some examples, the gyroscope is a microchip packaged MEMS gyroscope.

[0024] In various examples, the first LED 106 is a battery indicator LED. In one example, the first LED 106 is on when the battery is charged, and it is off when the battery is dead. In some examples, the first LED 106 flashes when the battery is low. In some examples, the first LED 106 is green when the battery is charged, and it turns red when the battery is low.

[0025] In various embodiments, the second LED 108 is a mode indicator LED. In one example, the mode indicator LED 108 changes color according to the selected mode. Various modes include turning on, off, recording, recording and transmitting drawn lines, and detecting characters in writing and transmitting characters. These modes can also indicate the use of different sensors.

[0026] In various examples, the switch 110 is a button. The switch 110 can be used to open the pen cap 102, close the pen cap 102 (if it is already open), and / or switch between various operating modes of the pen cap 102. The user can press the switch 110 to select a mode or open and close the pen cap 102.

[0027] In various examples, the battery 112 provides power to the pen cap 102. Figure 1A In the illustrated aspect, the battery 112 is round and is located on the top of the pen cap. In other embodiments, the battery is an elongated battery and is located on the side of the pen cap 102. In some embodiments, the battery 112 is a rechargeable battery and the pen cap 102 can be charged without removing or replacing the battery. In other embodiments, the battery 112 is a disposable battery and the battery compartment is accessible to the user. The user can remove the battery 112 from the pen cap 102 and replace the battery 112 with a new battery 112.

[0028] In various examples, the elongated hole 114 extends approximately the length of the pen cap 102. The pen cap 102 includes a sliding mechanism attached to the sensor 104, and the sliding mechanism can be used to move the sensor 104 up and down the elongated hole 114 in the pen cap 102. In this way, the position of the sensor 104 is adjustable along the length of the pen cap 102. In some examples, the orientation of the sensor 104 is also adjustable. In various embodiments, the adjustability of the position of the sensor 104 allows the user to calibrate the sensor 104 and use the pen cap 102 with any selected pen, pencil, or stylus.

[0029] Figure 1B A side view 120 of a pen cap 102 is shown in accordance with various aspects of the present disclosure. Figure 1B The sensor 104 and the elongated hole 114 are shown. Figure 1B Arrows are included to illustrate potential movement of the sensor 104 within the elongated aperture 114 along the length of the cap 102 .

[0030] Figure 2A A block diagram 200 is shown of the functionality of the pen cap 102. Figure 2A As shown, the pen cap 102 includes a plurality of sensors 202 for feature extraction and handwriting recognition. The sensors 202 include inertial sensors, such as one or more of an accelerometer and a gyroscope. Accelerometers measure dynamic acceleration and static acceleration caused by motion, such as tilt. These measurements are based on the position of a detection mass that is capacitively sensed and decoded to provide acceleration data. Accelerometers typically measure speed in meters per second squared (m / s 2 ) or gravity (G) as units to measure the rate of change of velocity, where a single gravity is equivalent to 9.8m / s 2(Although this may vary slightly with altitude). In some examples, the output units of the accelerometer sensor of the pen cap described herein are milligrams (mg). The gyroscope measures the angular velocity, tilt and / or lateral direction of an object in the yaw, pitch and roll directions. The rotational speed of the sensor is converted into an electrical signal. The 16-bit analog-to-digital converter in the accelerometer and gyroscope samples the mechanical motion and reports the data digitally at an output data rate of 1kHz via serial communication. The output data provides a binary value for each of the x, y, z (yaw, pitch and roll) axes. The inertial sensor is designed to sense six degrees of freedom. In particular, the sensor 202 can sense forward and / or backward, upward and / or downward, left and / or right movement along any of the three perpendicular axes (x-axis, y-axis, z-axis). In addition, the sensor 202 senses rotation or movement around any of the x-axis, y-axis and z-axis. The sensor senses the movement speed of the pen cap in any direction, the movement direction of the pen cap, the movement speed of the pen cap, and the acceleration and / or deceleration of the movement of the pen cap. The sensor also senses the angular acceleration of the pen cap.According to various examples, the inertial sensor senses the movement of the pen cap relative to a previous position.

[0031] In various embodiments, the sensor data may be used for machine learning. Specifically, the pre-processed data is input to the machine learning module 320 ( Figure 3 ), which further processes the data. In various examples, the machine learning module is designed to digitally transcribe the user's handwriting. The machine learning module 320 performs machine learning to predict letters based on sensor data (including inertial sensor data, such as accelerometer and gyroscope data). In some examples, the machine learning module uses extracted features to predict letters. Machine learning can include (1) preprocessing (block 204), (2) feature derivation / feature extraction (block 206), and (3) classification (block 208).

[0032] At block 204, sensor data from the inertial sensor 202 is acquired and preprocessed. The machine learning preprocessing step at block 204 includes bandpass filtering, for example, with a cutoff frequency of 20 Hz. For example, data from all three axes is processed through a low pass filter to eliminate incidental motion such as physiological tremors. In some embodiments, the cutoff frequency can be adapted to a particular user based on the writing style of the particular user. For example, the cutoff frequency can range from 3 Hz to 30 Hz. The filtered measurements of each sensor at the sampling rate can be recorded into a data structure. In some embodiments, the measurements can be stored as changes in each of the six degrees of freedom (e.g., x, y, z, yaw, pitch, and roll). The preprocessed data can be stored in memory 306 ( Figure 3 ) or transmitted to another device (e.g., computing device 350 ( Figure 3In some examples, the data is transmitted using Bluetooth. The filtered data is then used for further processing (at the pen cap 102 or other device), including feature derivation / feature extraction at block 206 and classification at block 208.

[0033] In some embodiments, the preprocessing in block 204 includes an initial step of aiming the sensor. Generally speaking, aiming is a calibration procedure that includes determining differences in sensor rotation. For example, aiming can be adjusted based on differences in alignment of the pen cap on different axes. In various examples, the rotation axis of an accelerometer, a gyroscope, and / or another inertial measurement unit can be determined. In some embodiments, when the pen cap includes a camera, the rotation axis of the camera can be determined. In some examples, the aiming step includes removing the mean from the data of each measurement to center the measurement on zero. In an accelerometer, while deriving features, trends can be identified and used to determine noise. In addition, in some examples, a piecewise linear model can be fitted and trends can be eliminated by dividing the data into smaller segments. The preprocessed data can be stored in the memory 306 or transmitted to other devices. In some embodiments, the preprocessed data can be transmitted to a cloud network for storage and / or processing.

[0034] The machine learning module 320 performs feature extraction at block 206. According to various implementations, the feature extraction step at block 206 includes determining the mean, median, kurtosis, skewness, signal power, and interquartile range (IQR) statistical features of the three axes of the accelerometer. Kurtosis refers to a measure of the "tail" of the probability distribution of the measured values. Skewness refers to a measure of the asymmetry of the probability distribution of the measured values ​​about the mean. Signal power refers to the absolute sum of squares of the time domain samples divided by the signal length, or equivalently, the square of the RMS level. The derived features create a feature space that is used as an input to the classification model at the classification block 208. In various examples, the feature extraction in block 206 derives a set of statistical features of a sample window in real time, which can be, for example, 200 samples. In some embodiments, a sample window of 200 samples may be a default value. The size of the sample window can be adjusted according to a specific user, for example, based on the user's writing speed during the calibration process. For example, the writing speed can be measured when the user writes a set of prescribed words or sentences. In some implementations, for a sampling rate of 1 kHz, the size of the sample window may vary between 100 and 400 samples.

[0035] As described above, at block 206, data from the inertial sensor 202 is input to the feature extraction module 324 ( Figure 3). At block 206, the feature extraction module 324 detects selected patterns in the input data and identifies these patterns as being associated with selected features. In various embodiments, the feature extraction module 324 at block 206 uses a sample window of the measurement data to detect features. The sample window of the measurement data is updated over time. Typically, feature extraction is used to reduce the total amount of data so that the amount of data output from the feature extraction module is less than the amount of data input to the feature extraction module. For example, the number of features in the sample window is less than the original data of the sample window sample. At block 208, the data output from the feature extraction module 324 is input to the classification model 326. In some embodiments, the classification model 326 can be located on an external device (e.g., a computing device 350 ( Figure 3 ). Emitting feature sets may be more effective than emitting raw or preprocessed samples.

[0036] Figure 2B is a graph showing feature extraction values ​​according to various aspects of the present disclosure. Figure 2B As shown, specific feature values ​​include, for example, mean, median, kurtosis, skewness, signal power, and IQR (interquartile range). In some embodiments, feature extraction is used for sub-motion analysis and identification of hand movements. Features such as kurtosis measure randomness by producing lower noise values ​​(-1.2) and higher specific data values.

[0037] Back to Figure 2A , in block 208, the classification model 326 predicts the character of the feature set. In some embodiments, a support vector machine (SVM) is used as the classification model 326 to classify the derived features into characters. In other examples, the classification model may include a neural network, a decision tree, a bagging classifier, or a random forest. In some embodiments, for example, gradient boosting may be used to combine multiple classification models.

[0038] In various examples, the classification model 326 at block 208 can be trained using training data of a set of characters obtained from a subject using a pen cap. For example, the set of characters may correspond to letters in an alphabet. Separate classification models can be used for different alphabets or languages. The subject can generate training data by writing prescribed words and sentences. In the training data, individual characters can be separated by noise or motion patterns (such as spaces between characters). The feature space of each character can be labeled with a class label corresponding to the character of the specified word and sentence. For training and testing purposes, the training data can be divided into 70%-30%. In some examples, among the various classification models, the support vector machine (SVM) has the highest accuracy. Therefore, in various examples, SVM is used as a machine learning model for classification tasks.

[0039] In some embodiments, block 206 includes sub-motion analysis. Sub-motion analysis finds Gaussian curves and decomposes the curves into small sub-motions. The sub-motions are provided as features to the classification model. In some embodiments, the classifier engine at block 208 has two steps. The first step is a preliminary classification based on sub-motions. The second step is a secondary classification, which further classifies the sub-motions into characters based on specific feature values.

[0040] At block 210, data from the classification model 336 is input to the handwriting recognition module 328 ( Figure 3 ). The handwriting recognition module can be used to interpret the handwriting data. In some embodiments, the handwriting recognition module 328 can include a dictionary-based word prediction selection. For example, the handwriting recognition module can predict options for completing a word based on the initial characters of the word. In block 206, the predicted options can be used to reduce the feature space for character recognition of subsequent characters. Additionally or alternatively, the handwriting recognition module can verify or correct the words output by the classification model.

[0041] In some embodiments, the handwriting recognition module 328 outputs the handwriting recognition data to a memory for storage at block 212. In some examples, the memory is integrated into the digital pen cap. In other examples, the memory is separate from the digital pen cap and the handwriting recognition data is transferred to an external memory. In some examples, a Bluetooth connection is used to transfer data from the digital pen cap to the external memory. In some examples, the Bluetooth connection is a Bluetooth Low Energy (BLE) connection, a low-power wireless communication technology.

[0042] Figure 3 300 is another block diagram illustrating elements of a digital pen cap 102 according to various embodiments of the present disclosure. The digital pen cap 102 includes a sensor 302, a controller 304, a memory 306, and a transmitter 308. The sensor 302 includes an inertial sensor, such as an accelerometer 312 and a gyroscope 314, and senses the movement of the digital pen cap. The sensor 302 transmits data to the controller 304, for example, via a wired bus. In some embodiments, the controller 304 processes the received data. In some examples, the controller 304 outputs the data to the memory 306 where the data is stored. In some examples, the controller 304 outputs the data to the transmitter 308. The transmitter 308 sends the data to, for example, a computing device 350. In various examples, the transmitter 308 is a Bluetooth transmitter.

[0043] In one aspect, the controller 304 includes a machine learning module 320 and a handwriting recognition module 328. As described above, the machine learning module 320 may include a pre-processing module 322 for performing block 204, a feature extraction module 324 for performing block 206, and a classification model 326 for performing block 208. In some embodiments, the pre-processing module 322 may include a low-pass filter. The feature extraction module 324 may include circuits and / or software executed by one or more processors to calculate the above-mentioned Figure 2B Each feature discussed. The classification model 326 can be a machine learning model, such as an SVM that is trained to classify features into characters. In some embodiments, the machine learning module 320 can include multiple classification models 326, each of which is trained for a specific alphabet or language. In some embodiments, the classification model 326 is pre-trained, for example, based on writing samples from multiple topics. In some embodiments, the classification model 326 can be trained or adjusted based on a training set generated by a user based on a specified text (e.g., a set of words or sentences written by a user with a pen cap 102 attached to the pen barrel). Training or adjusting the classification model 326 for a user can improve the accuracy of character classification for a particular user. For example, an SVM can be adjusted by changing a configurable parameter, while a neural network can have adjustable weights that can be adjusted through additional training.

[0044] The transmitter 308 transmits data to the computing device 350. The transmitter 308 can transmit any data stored in the memory 306. In some implementations, the computing device 350 includes an application with an API 360. The application is configured to receive data from the pen cap 102. In some implementations, the API 360 is configured to receive processed data such as characters or words from the pen cap 102. In some implementations, the computing device 350 or an application thereon can include one or more of the machine learning module 320 or the handwriting recognition module 328. The API 360 can receive raw or partially processed data from the pen cap 102 and perform further processing. For example, the API 360 can receive raw data (e.g., measurements from the sensor 302), pre-processed data (e.g., output from the pre-processing module 322), features (e.g., output from the feature extraction module 324), characters (e.g., output from the classification model 326), or words (e.g., output from the handwriting recognition module 328). Computing device 350 may execute machine learning module 320 and / or handwriting recognition module 328 to perform further processing, for example, as described above with respect to Figure 2A described.

[0045] According to various implementations, a processor (e.g., a processor of a controller 304 or a computing device 350) collects data from an inertial sensor 302, including an accelerometer 312 and a gyroscope 314, and uses the data to derive pen cap position, velocity, and direction information. In one example, the processor is a microprocessor. In some examples, the processor is part of a controller (and / or microcontroller). In some embodiments, the processor filters the data with a second-order low-pass filter. In one example, the second-order low-pass filter has a cutoff frequency of 4 Hz, thereby filtering out physiological tremors. In some examples, the data is further filtered using a data window for selecting hand movements. In some embodiments, when the pen cap is used, continuous data is passed to the microcontroller in real time, and statistical features are calculated in a feature extraction module 324 for every 200 samples. The extracted features are passed to a machine learning model (e.g., a classification model 326), which classifies letters based on the extracted feature values.

[0046] In some examples, the processor collects raw data from the sensor and stores the raw data locally in a flash memory (e.g., memory 306). In some examples, the flash memory in the pen cap has one of a storage capacity of about 256MB, 512MB, 1024MB, or more. In some implementations, the data from the flash memory is wirelessly transmitted to an external computing device 350. In various examples, the data can be transmitted using one of a Bluetooth connection, a USB connection, or a USB C communication.

[0047] In some implementations, the data is sent to an application program interface (API) 360 in an encrypted format. In one example, the encrypted format is a JavaScript Object Notation (JSON) object. The API 360 includes algorithms that accept encrypted data and decrypt and decode the encrypted data.

[0048] In some embodiments, the digital pen cap 102 includes a rechargeable power source and can be charged using one of a USB Type C charger and a wireless charging option. In one example, the wireless charging option is a magnetic charger. In some embodiments, the digital pen cap includes a removable battery that can be replaced by the user.

[0049] Figures 4A-4C A side view of a digital pen cap according to various aspects of the present disclosure is shown. Figure 4D A bottom view of a digital pen cap according to various aspects of the present disclosure is shown. Figure 4A 4 shows a first side view 400 of a digital pen cap 402. The first side view 400 shows a sensor plate 404 attached to the pen cap 402. According to various examples, the sensor plate 404 can be vertically movable relative to the pen cap 402, such as Figure 1AThe sensor plate 404 extends a majority of the length of the pen cap 402. In various embodiments, the sensor plate 404 can be any selected size to contain sensors for detecting movement of the pen cap 402.

[0050] Figure 4B A second side view 410 of the digital pen cap 402 is shown. The second side view 410 shows the side of the pen cap 402 opposite to the first side view 400. The second side view 410 shows the clamping side of the pen cap. The circuit board 406 is included on the clip side of the sensor board. According to some embodiments, the circuit board on the clip side of the digital cap includes a transmitter for transmitting signals from the sensor cap 402. In some examples, the transmitter transmits the signal wirelessly to an external device (e.g., computing device 350). In some examples, the transmitter transmits the signal via a Bluetooth connection. In some embodiments, the circuit board 406 may include a plurality of protruding elements that may provide a connection (e.g., for a wired connection).

[0051] Figure 4C Another side view 420 of the pen cap 402 is shown, which includes the sensor board 404 side of the pen cap and the clip side of the pen cap including the circuit board 406. The side view 420 shows the barrel 408 of the pen cap 402. According to various examples, the barrel of the pen cap 402 can be adjusted to accommodate any size of stylus. In one example, the diameter of the barrel 408 of the pen cap 402 can be reduced by inserting a second tube (e.g., a rubber sleeve) into the barrel 408 to effectively narrow the barrel 408.

[0052] Figure 4D A bottom view of the pen cap 402 including the sensor board 404 , the circuit board 406 , and the pen barrel 408 is shown.

[0053] Figure 5 A digital pen cap 502 is shown located on the distal end of a stylus 508 in accordance with various embodiments of the present disclosure. As described above, the digital pen cap 502 includes a sensor. The sensor in the digital pen cap 502 determines a distance 504 between the cap and the writing surface. In addition, the sensor in the cap 502 determines an angle 506 between the cap 502 and the distance line. Using the angle 506 and the distance 504, the digital pen cap 502 determines the position of the tip 510 (i.e., the writing end) of the stylus 508. As previously described, the digital pen cap 502 measures and records its movement along an arbitrary axis, and uses these recorded movements to determine the movement of the tip 510 of the stylus 508.

[0054] Figure 66 is a flow chart of an example method 600 for transcribing writing using a writing instrument having a shaft having a writing end and a distal end opposite the writing end. The method 600 can be performed by the pen cap 102, or alternatively by a user or a computing device 350 in communication with the pen cap. For example, in some embodiments, the method 600 can also include the configuration of the pen cap 102.

[0055] In block 610, method 600 may optionally include inserting a flexible rubber layer into the interior of the cap to reduce the effective diameter of the cap. For example, a user may insert a flexible rubber layer into the interior of cap 102 to reduce the effective diameter of the cap.

[0056] In block 620 , method 600 may optionally include attaching the cap to the distal end of the shaft. For example, a user may attach pen cap 102 to the distal end of stylus 508 .

[0057] In block 630, the method 600 includes receiving sensor data, the sensor data including a plurality of samples of the position and orientation of the removable cap, the removable cap including a plurality of inertial sensors attached to the distal end of the shaft. For example, in some embodiments, the controller 304 receives sensor data, the sensor data including a plurality of samples of the position and orientation of the removable cap, the removable cap including a plurality of inertial sensors 302 attached to the distal end of the shaft. For example, the controller 304 may receive samples from the sensor 302 via a wired bus. In other embodiments, the computing device 350 may receive the samples via the transmitter 308.

[0058] In some embodiments, aiming based on the difference in the position of the sensors relative to the axis can be optionally included at block 632. For example, the pre-processing module 322 can perform aiming of multiple sensors. In some embodiments, the pre-processing module 322 can apply a low pass filter to the multiple samples.

[0059] At block 640, the method 600 includes extracting a feature set from the sensor data over the sample window. For example, the controller 304 and / or the computing device 350 may execute the feature extraction module 324 to extract a feature set from the sensor data within the sample window.

[0060] In block 650, method 600 may optionally include changing the configurable parameters of the user's support vector machine based on the training sample of the specified characters written by the user. For example, controller 304 and / or computing device 350 may change the configurable parameters of the user's support vector machine (i.e., classification model 326) based on the training sample of the specified characters written by the user.

[0061] At block 660, the method 600 includes classifying the feature set as a character based on the trained classification model. For example, the controller 304 and / or the computing device 350 may execute the classification model 326 to classify the feature set as a character.

[0062] In block 670, method 600 may optionally include predictively selecting one or more words based on the characters and the dictionary. For example, controller 304 and / or computing device 350 may execute handwriting recognition module 328 to predictively select one or more words based on the characters and the dictionary. The selection may also be based on contextual embeddings of the one or more characters.

[0063] In block 680, method 600 may optionally include selecting a subsequent character based at least in part on the one or more words. For example, controller 304 and / or computing device 350 may execute handwriting recognition module 328 to select a subsequent character based at least in part on the one or more words. For example, handwriting recognition module 328 may select a subsequent character that classification model 326 predicts has the highest likelihood of being the next in one of the selected words.

[0064] In block 690, method 600 may optionally include transmitting the sensor data, the feature set, the character, or a combination thereof to an external computing device. For example, transmitter 308 may send the sensor data, the feature set, the character, or a combination thereof to external computing device 350. For example, depending on the data to be transmitted, block 690 may optionally be performed before any of blocks 630, 640, 660, or 670.

[0065] The following numbered clauses summarize various aspects of the disclosure:

[0066] Item 1. A cap for positioning on an axis to record movement of the axis, comprising: a housing configured to be attached to a distal end of the axis opposite to a writing end of the axis; a plurality of sensors configured to measure a position and orientation of the distal end of the axis and output the position and orientation as sensor data; a controller configured to process the sensor data and output the processed data; a memory configured to store the processed data; and a transmitter configured to transmit the processed data.

[0067] Clause 2. The cap of clause 1, wherein to process the sensor data, the controller is configured to perform feature extraction on the sensor data, and wherein the processed data includes the extracted features.

[0068] Clause 3. The cap of clause 2, wherein feature extraction comprises aiming based on a difference in position of said sensor relative to said axis.

[0069] Clause 4. The cap of clause 2 or 3, wherein the extracted features include two or more of the mean, median, kurtosis, skewness, signal power, and interquartile range.

[0070] Clause 5. The cap of any one of clauses 1-4 further includes a machine learning module, which is configured to: receive the processed data; extract features from the processed data; and classify the features into characters based on a trained classification model.

[0071] Clause 6. The hat of clause 5, wherein the trained classification model is a support vector machine.

[0072] Clause 7. The cap of clause 6, wherein the controller is configured to change a configurable parameter of a support vector machine for a user based on a training sample of prescribed characters written by the user.

[0073] Clause 8. The cap of any of clauses 1-7, wherein the plurality of sensors includes at least one gyroscope and at least one accelerometer.

[0074] Clause 9. The cap of any of clauses 1-8, further comprising a flexible rubber layer removably connected to an interior of the housing to reduce an effective diameter of the housing.

[0075] Clause 10. The cap of any of clauses 1-9, further comprising a battery configured to provide power to the plurality of sensors, the controller, the memory, and the transmitter.

[0076] Clause 11. The cap of any of clauses 1-10, wherein the transmitter is configured to transmit short-range wireless signals.

[0077] Item 12. A method for transcribing writing using a writing instrument, the writing instrument having an axis, the axis having a writing end and a distal end opposite the writing end, the method comprising: receiving sensor data, the sensor data comprising multiple samples of the position and orientation of a removable cap, the removable cap comprising multiple inertial sensors attached to the distal end of the axis; extracting a feature set from the sensor data within a sample window; and classifying the feature set into characters based on a trained classification model.

[0078] Clause 13. The method of clause 12, wherein extracting the feature set includes targeting based on a difference in position of the sensor relative to the axis.

[0079] Clause 14. The method of clause 12 or 13, wherein the extracted features include two or more of the mean, median, kurtosis, skewness, signal power, and interquartile range.

[0080] Clause 15. The method of any of clauses 12-14, wherein the trained classification model is a support vector machine.

[0081] Clause 16. The method of clause 15, further comprising changing a configurable parameter of a support vector machine of the user based on a training sample of prescribed characters written by the user.

[0082] Clause 17. The method of any of clauses 12-16, wherein the plurality of sensors includes at least one gyroscope and at least one accelerometer.

[0083] Clause 18. The method of any of clauses 12-17, further comprising: inserting a flexible rubber layer into the interior of the cap to reduce the effective diameter of the cap; and attaching the cap to the distal end of the shaft.

[0084] Clause 19. The method of any of clauses 12-18, further comprising: predictively selecting one or more words based on the characters and a dictionary; and selecting subsequent characters based at least in part on the one or more words.

[0085] Clause 20. The method of any of clauses 12-19, further comprising: transmitting the sensor data, the feature set, the character, or a combination thereof to an external computing device.

[0086] In the above discussion, sensors, amplifiers, switches, transistors, memory and / or other components can be easily replaced, substituted or otherwise modified to suit specific circuit requirements. In addition, it should be noted that the use of complementary electronic devices, hardware, software, etc. provides equally feasible options for implementing the teachings of the present disclosure.

[0087] In one example aspect, any number of the circuits in the figures may be implemented on a board of an associated electronic device. The board may be a general purpose circuit board that may house various components of an electronic system internal to the electronic device and may also provide connectors for other peripheral devices. More specifically, the board may provide electrical connections through which other components of the system may communicate electrically. Any suitable processor (including digital signal processors, microprocessors, supporting chipsets, etc.), computer-readable non-volatile storage elements, etc. may be appropriately coupled to the board based on specific configuration needs, processing requirements, computer design, etc. Other components, such as external memory, additional sensors, audio / video display controllers, and peripheral devices, may be connected to the board via cables as plug-in cards or may be integrated into the board itself.

[0088] In various aspects, the functions described herein may be implemented in emulated form as software or firmware running within one or more configurable (e.g., programmable) elements arranged in a structure that supports the functions. The software or firmware providing the emulation may be provided on a non-transitory computer-readable storage medium that includes instructions that allow a processor to perform the functions.

[0089] In another example aspect, the circuits of the accompanying drawings may be implemented as stand-alone modules (e.g., devices having related components and circuits configured to perform a specific application or function) or as plug-in modules inserted into dedicated hardware of an electronic device. Note that certain aspects of the present disclosure may be readily included in part or in whole in a system-on-chip (SOC) package. SOC stands for an IC that integrates the components of a computer or other electronic system into a single chip. It may contain digital, analog, mixed signal, and often radio frequency functions: all of which can be provided on a single chip substrate. Other aspects may include a multi-chip module (MCM), which has multiple separate ICs located within a single electronic package and configured to interact closely with each other through the electronic package. In various other aspects, the clock and filtering functions may be implemented in one or more silicon cores in application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and other semiconductor chips.

[0090] All specifications, dimensions, and relationships (e.g., number of processors, logical operations, etc.) outlined herein are for exemplary and teaching purposes only. Such information may be significantly changed without departing from the spirit of the present disclosure or the scope of the appended claims. These specifications apply only to one non-limiting example, and therefore, they should be interpreted as such. In the foregoing description, example aspects have been described with reference to specific processors and / or component arrangements. Various modifications and changes may be made to these aspects without departing from the scope of the appended claims. Therefore, the specification and drawings should be regarded as illustrative, not restrictive.

[0091] Note that the activities discussed above with reference to the accompanying figures apply to any integrated circuit involving signal processing, particularly those using sampled analog, some of which may be related to processing real-time data. Certain aspects may involve multi-DSP signal processing, floating-point processing, signal / control processing, fixed-function processing, microcontroller applications, etc.

[0092] In some cases, the features discussed herein may be applicable to medical systems, scientific instrumentation, wireless and wired communications, radar, industrial process control, audio and video equipment, current sensing, instrumentation (which may be highly precise), and other digital processing-based systems.

[0093] Note that through many of the examples provided herein, interactions may be described in terms of two, three, four, or more electrical components. However, this is done for clarity and example purposes only. It should be understood that the system may be integrated in any suitable manner. Along similar design alternatives, any of the components, modules, and elements shown in the drawings may be combined in a variety of possible configurations, all of which are clearly within the broad scope of this specification. In some cases, it may be easier to describe one or more functions of a given set of flows by referencing only a limited number of electrical components. It should be understood that the drawings and the circuits they teach are easily expandable and can accommodate a large number of components and more complex / complex arrangements and configurations. Therefore, the examples provided should not limit the scope of the circuit or inhibit its broad teachings that may be applied to countless other architectures.

[0094] Note that in this specification, references to various features (e.g., elements, structures, modules, components, steps, operations, characteristics, etc.) included in "one aspect", "example aspect", "on one hand", "on the other hand", "some aspects", "various aspects", "other aspects", "alternative aspects", etc.) are intended to mean that any such features are included in one or more aspects of the present disclosure, but may or may not be combined in the same aspect.

[0095] Numerous other variations, substitutions, changes, alterations, and modifications may be ascertained by those skilled in the art, and the present disclosure is intended to encompass all such variations, substitutions, alterations, alterations, modifications, and modifications that fall within the scope of the appended claims. To assist the United States Patent and Trademark Office (USPTO) and any reader of any patent issued by this application in interpreting the claims appended hereto, applicants wish to note that applicants: (a) do not intend to invoke 35 U.S.C. § 112(f) as it exists on the date of filing this application, unless the term "means" or "step" is specifically used in a particular claim; and (b) do not intend, by any statement in the specification, to limit the present disclosure in any manner not reflected in the appended claims.

[0096] Note that all optional features of the apparatus described above may also be implemented with respect to the methods or processes described herein, and that details in the examples may be used anywhere in one or more embodiments.

[0097] In these cases (as described above), means for "for" may include (but are not limited to) using any suitable components discussed herein, as well as any suitable software, circuits, hubs, computer code, logic, algorithms, hardware, controllers, interfaces, links, buses, communication paths, etc. In a second example, the system includes a memory that also includes machine-readable instructions that, when executed, cause the system to perform any of the activities described above.

Claims

1. A cap for positioning on a shaft to record the movement of the shaft, comprising: a housing configured to be attached to a distal end of the shaft opposite the writing end of the shaft; a plurality of sensors configured to measure a position and an orientation of a distal end of the shaft and output the position and orientation as sensor data; a controller configured to process the sensor data and output the processed data; A memory configured to store the processed data; and The transmitter is configured to transmit the processed data. 2 . The cap of claim 1 , wherein to process the sensor data, the controller is configured to perform feature extraction on the sensor data, and wherein the processed data includes the extracted features.

3. The cap according to claim 2, wherein the feature Extracting includes targeting based on a difference in position of the sensor relative to the axis.

4. The cap according to claim 2 or 3, wherein the extracted features include two or more of mean, median, kurtosis, skewness, signal power and interquartile range.

5. The cap of any preceding claim, further comprising a machine learning module configured to: receiving the processed data; extracting features from the processed data; and The features are classified into characters based on the trained classification model. The cap of claim 5 , wherein the trained classification model is a support vector machine.

7. The cap of claim 6, wherein the controller is configured to change a configurable parameter of a support vector machine for a user based on a training sample of prescribed characters written by the user.

8. The cap of any preceding claim, wherein the plurality of sensors comprises at least one gyroscope and at least one accelerometer.

9. The cap of any preceding claim, further comprising a flexible rubber layer removably connected to the interior of the housing to reduce the effective diameter of the housing.

10. The cap of any one of the preceding claims, further comprising a battery configured to provide power to the plurality of sensors, the controller, the memory, and the transmitter.

11. A cap according to any preceding claim, wherein the transmitter is configured to transmit short range wireless signals.

12. A method of transcribing writing using a writing instrument, the writing instrument having a shaft, the shaft having a writing end and a distal end opposite the writing end, the method comprising: receiving sensor data, the sensor data comprising a plurality of samples of position and orientation of a removable cap comprising a plurality of inertial sensors attached to a distal end of the shaft; extracting a feature set from the sensor data within a sample window; and The feature set is classified into characters based on the trained classification model.

13. The method of claim 12, further comprising aiming based on a difference in position of the sensor relative to the axis.

14. The method according to claim 12 or 13, wherein the extracted features include two or more of mean, median, kurtosis, skewness, signal power and interquartile range.

15. The method according to any one of claims 12 to 14, wherein the trained classification model is a support vector machine.

16. The method of claim 15, further comprising changing a configurable parameter of a support vector machine of the user based on a training sample of prescribed characters written by the user.

17. The method of any one of claims 12 to 16, the plurality of sensors comprising at least one gyroscope and at least one accelerometer.

18. The method according to any one of claims 12 to 17, further comprising: inserting a flexible rubber layer into the interior of the cap to reduce the effective diameter of the cap; and The cap is attached to the distal end of the shaft.

19. The method according to any one of claims 12 to 18, further comprising: predictively selecting one or more words based on the characters and a dictionary; and A subsequent character is selected based at least in part on the one or more words.

20. The method of any one of claims 12 to 19, further comprising transmitting the sensor data, the feature set, the character, or a combination thereof to an external computing device.

Images