Method and system for handwriting recognition using a wearable pressure sensing device

Abstract

Writing recognition using a wearable pressure sensing device includes receiving pressure measurement data from a pressure sensor disposed on a body part of a user. The pressure measurement data indicates a change in pressure of the body part due to an interaction of the body part with a medium indicating a writing gesture of the user. Based on the pressure measurement data, a start boundary and an end boundary of each of a plurality of writing symbols are detected. At least one feature of the pressure measurement data associated with the plurality of writing symbols is extracted. A symbol pattern is detected based on the extracted feature, and at least one letter is detected based on the symbol pattern. A word is detected based on the detected at least one letter.

Description

Technical Field

[0001] The present invention generally relates to methods, systems and computer program products for handwriting recognition. More specifically, the present invention relates to methods, systems and computer program products for handwriting recognition using a wearable pressure sensing device attached to a body part of a user. Background Art

[0002] Writing and handwriting recognition have played an important role in human culture. Handwriting recognition refers to the ability of a computer to receive and interpret handwritten communications. Until recently, writing has been limited to two basic aids: a writing surface (e.g., paper or a tablet) and a writing implement (e.g., a pen, pencil, or stylus). Electronic handwriting recognition typically requires the use of a specific writing surface and / or writing implement. For example, some implementations of handwriting recognition require the use of a touch-sensitive writing surface or screen. Writing surfaces and writing implements are typically always available and frequently used. Even when writing surfaces and writing implements are available, they may be limited to specific environments, such as a home environment, and / or require specific lighting conditions. Summary of the invention

[0003] Illustrative embodiments provide a method, system, and computer program product. An embodiment of a method for handwriting recognition using a wearable pressure sensing device includes receiving pressure measurement data from a pressure sensor disposed on a body part of a user. In this embodiment, the pressure measurement data is indicative of a change in pressure of the body part due to the interaction of the body part with a medium indicating a writing gesture of the user. The embodiment also includes: detecting a start boundary and an end boundary of each of a plurality of handwriting symbols based on the pressure measurement data; and extracting at least one feature of the pressure measurement data associated with the plurality of handwriting symbols. The embodiment also includes: detecting a symbol pattern based on the extracted features; detecting at least one letter based on the symbol pattern; and detecting a word based on the detected at least one letter.

[0004] Embodiments also include: receiving motion data from a motion sensor disposed on a body part of the user, wherein the detection of the symbol pattern is also based on the motion data. Embodiments also include reducing the dimensionality of the pressure measurement data before detecting the start boundary and the end boundary. One embodiment also includes preprocessing the pressure measurement data before reducing the dimensionality of the pressure measurement data. In one embodiment, the preprocessing of the pressure measurement data includes one or more of the following: filtering bias and undesired frequency content from the pressure measurement data; scaling the pressure measurement data; and removing temporal artifacts from the pressure measurement data.

[0005] Embodiments also include detecting commands based on the extracted features. Embodiments also include detecting symbols of a virtual keyboard based on the extracted features. Embodiments also include detecting a medical condition of a user based on the extracted features.

[0006] In one embodiment, the at least one feature comprises one or more of a spatial feature, a temporal feature, and a spectral feature of the pressure measurement data. In one embodiment, the detection of the symbol pattern is based on prior statistical knowledge. In one embodiment, the detection of the at least one letter is performed using at least one of machine learning on the extracted features or on the raw data of the pressure sensor together with additional sensor data.

[0007] Embodiments include a computer usable program product. The computer usable program product includes one or more computer readable storage devices and program instructions stored on at least one of the one or more storage devices.

[0008] Embodiments include a computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors through at least one of the one or more memories. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Certain novel features which are believed to be characteristic of the invention are set forth in the appended claims. However, the invention itself and its preferred mode of use, further objects and advantages will be best understood from the following detailed description of illustrative embodiments read in conjunction with the accompanying drawings, in which:

[0010] Figure 1 depicts a block diagram of a network of data processing systems in which the illustrative embodiments may be implemented;

[0011] Figure 2 depicts a block diagram of a data processing system in which the illustrative embodiments may be implemented;

[0012] Figure 3 depicts a block diagram of an example configuration for handwriting recognition using a wearable pressure sensing device in accordance with an illustrative embodiment;

[0013] Figure 4A depicts an example pressure sensor configuration for a wearable sensing device in accordance with an illustrative embodiment;

[0014] Figure 4B Depicted is a schematic diagram of an illustrative embodiment of the present invention. Figure 4A An example of a pressure sensor measurement with a pressure sensor configuration;

[0015] Figure 5 depicts example pre-processing operations for a wearable sensing device according to an illustrative embodiment;

[0016] Fig. 6A depicts an example of handwriting recognition through aggregation of basic shapes in accordance with an illustrative embodiment;

[0017] Figure 6B depicts another example of handwriting recognition through aggregation of basic shapes in accordance with an illustrative embodiment;

[0018] Figure 7 depicts an example of handwriting recognition using a wearable pressure sensing device in accordance with an illustrative embodiment;

[0019] Figure 8 depicts a flow chart of an example process for handwriting recognition using a wearable pressure sensing device in accordance with an illustrative embodiment;

[0020] Fig. 9 A flowchart depicting an example process for using a wearable pressure sensing device for handwriting recognition to implement a virtual keyboard in accordance with an illustrative embodiment; and

[0021] Fig.10 Depicted is a flow diagram of an example process for handwriting recognition using a wearable pressure sensing device to be used to detect a medical condition. DETAILED DESCRIPTION

[0022] The illustrative embodiments recognize that currently available tools or solutions do not meet these needs or do not provide adequate solutions to these needs.The illustrative embodiments used to describe the present invention generally address and solve the above-mentioned problems and other problems related to handwriting recognition.

[0023] Embodiments may be implemented as software applications. Applications implementing embodiments may be configured as modifications to existing handwriting recognition systems, separate applications and / or devices operating in conjunction with handwriting recognition systems, standalone applications and wearable sensing devices, or some combination thereof.

[0024] Various embodiments describe devices, systems, and methods for writing and handwriting recognition that do not require a writing surface and a writing tool. In one or more embodiments, a user performs a gesture related to writing using a body part (e.g., a finger) equipped with a wearable sensing device, which includes one or more pressure sensors attached to the body part. In a specific embodiment, the pressure sensor can be any sensor capable of detecting pressure changes in a finger or other body part interacting with any surface, such as a piezoresistor, a strain gauge sensor (SGS), a piezoelectric sensor, a capacitive sensor, or an LED / photodetector configured to display pressure changes of an external tissue based on a color change. In a specific example, the pressure sensor is deployed on a nail, and the parameters of the local pressure, shear, and torsion of the nail are derived when interacting with any surface. In one or more embodiments, the pressure sensor can provide measurement data using measurements of one or more of pressure, tension, and compression of a body part.

[0025] In one embodiment, a system includes: a wearable sensing device having one or more pressure sensors configured to be set on a body part; a medium; and a memory and a processing unit. In one or more embodiments, the body part can be any body part that can be attached to the pressure sensor, such as a fingernail of a hand. In one or more embodiments, the medium can be any medium for writing, including but not limited to any surface (e.g., a flat or non-flat surface) that causes a pressure change in the pressure sensor, such as a two-dimensional surface such as a table or a wall when the body part is pressed again. In a specific embodiment, the medium can also be a non-rigid surface, such as cloth or even air, where the movement of the body part in the air causes a pressure change in the body part. The pressure sensor of the wearable sensing device may include any suitable technology for sensing the pressure applied to the body part, for example, a piezoresistive sensor or an optical sensor (e.g., a camera). In some embodiments, the wearable sensing device may further include one or more additional sensors, such as an accelerometer or a gyroscope.

[0026] In one or more embodiments, when a user makes a gesture associated with handwriting, the wearable sensing device digitizes physical pressure measurements and preprocesses the digitized pressure measurements to normalize the pressure measurements for further processing. In one embodiment, the preprocessed pressure measurements are sent to a memory and processing unit. In this embodiment, the memory and processing unit are configured to decode the pressure measurements associated with the gesture into handwriting output and store the results, as further described herein.

[0027] One or more embodiments provide a method and apparatus for writing using pressure sensors and / or other sensors (such as accelerometers or gyroscopes) arranged on body parts to provide handwriting recognition of gestures indicative of writing. Specific embodiments provide human-machine translation of letters and numbers. Other specific embodiments provide a virtual keyboard for typing. Other embodiments provide for determining the health and health trends of a user using handwriting recognition after the system is trained or personalized to the user, through quantitative trends and testing for the user, and trend data over time under various usage conditions.

[0028] One or more embodiments described herein may utilize handwriting recognition to diagnose medical conditions, such as quantitative trends of medical conditions based on handwriting characteristics and testing and quantitative trends of Unified Huntington's Disease Rating Scale (UHDRS) or Unified Parkinson's Disease Rating Scale (UPDRS) assessments.

[0029] In other embodiments, the principles described herein may be applicable to uses other than position sensing of body parts (e.g., human fingernails or toenails), or other position sensing of human-machine interfaces for security, gaming, human health monitoring, testing, trending, and animal health monitoring applications.

[0030] Various embodiments described herein provide a handwriting device that does not require a pen, a surface for handwriting, a keyboard, or a touch screen. To recognize handwriting, one or more embodiments provide for the aggregation of motion features (e.g., extracted from acceleration measurements) and dynamic features (e.g., extracted from pressure measurements).

[0031] The illustrative embodiments are described for certain types of transmissions, sensors, measurements, devices, data processing systems, environments, components, and applications by way of example only. Any particular manifestation of these and other similar articles is not intended to limit the present invention. Any suitable manifestation of these and other similar articles may be selected within the scope of the illustrative embodiments.

[0032] Furthermore, the illustrative embodiments may be implemented for any type of data, data source, or access to a data source over a data network. Any type of data storage device may provide data to embodiments of the present invention locally on a data processing system or over a data network within the scope of the present invention. Where embodiments are described using a mobile device, any type of data storage device suitable for use with the mobile device may provide data to the embodiments locally on the mobile device or over a data network within the scope of the illustrative embodiments.

[0033] The illustrative embodiments are described only as examples using specific codes, designs, architectures, protocols, layouts, schematics, and tools, and are not limited to the illustrative embodiments. In addition, in some cases, for clarity of description, specific software, tools, and data processing environments are used to describe the illustrative embodiments only as examples. The illustrative embodiments can be used in combination with other comparable or similar purpose structures, systems, applications, or architectures. For example, other comparable mobile devices, structures, systems, applications, or architectures therefor can be used in combination with this embodiment of the present invention within the scope of the present invention. The illustrative embodiments can be implemented in hardware, software, or a combination thereof.

[0034] The examples in this disclosure are only for clarity of description and are not limited to the illustrative embodiments. Additional data, operations, actions, tasks, activities, and manipulations can be expected from this disclosure and can be expected within the scope of the illustrative embodiments.

[0035] Any advantages listed herein are merely examples and are not intended to be limited to the illustrative embodiments. Additional or different advantages may be achieved through specific illustrative embodiments. In addition, specific illustrative embodiments may have some, all, or none of the advantages listed above.

[0036] With reference to the accompanying drawings, in particular with reference to Figure 1 and 2 , these figures are exemplary diagrams of data processing environments in which illustrative embodiments may be implemented. Figure 1 and Figure 2 This is merely an example and is not intended to assert or imply any limitation with respect to the environments in which different embodiments may be implemented. Specific implementations may make many modifications to the depicted environment based on the following description.

[0037] Figure 1 A block diagram of a network of data processing systems in which the illustrative embodiments may be implemented is depicted. Data processing environment 100 is a network of computers in which the illustrative embodiments may be implemented. Data processing environment 100 includes network 102. Network 102 is a medium used to provide communication links between various devices and computers connected together in data processing environment 100. Network 102 may include connections such as wires, wireless communication links, or fiber optic cables.

[0038] Client or server are merely example roles for certain data processing systems connected to network 102 and are not intended to exclude other configurations or roles of these data processing systems. Server 104 and server 106 are coupled to network 102 along with storage unit 108. Software applications may be executed on any computer in data processing environment 100. Clients 110, 112, and 114 are also coupled to network 102. Data processing systems, such as server 104 or 106, or client 110, 112, or 114, may contain data and may execute software applications or software tools thereon.

[0039] The wearable sensing device 120 includes one or more pressure sensors configured to be disposed on one or more body parts of the user 122 for detecting pressure changes within the body parts. In one or more embodiments, the wearable sensing device 120 also includes one or more accelerometers for sensing movement of the body parts.

[0040] This is for example purposes only and does not imply any limitation on this architecture. Figure 1 Certain components that can be used in the example implementation of the embodiment are depicted. For example, servers 104 and 106 and clients 110, 112, 114 are described as servers and clients only as examples and do not imply a limitation on the client-server architecture. As another example, as shown in the figure, the embodiment can be distributed over multiple data processing systems and data networks, while within the scope of the illustrative embodiment, another embodiment can be implemented on a single data processing system. Data processing systems 104, 106, 110, 112, and 114 also represent example nodes in clusters, partitions, and other configurations suitable for the embodiment.

[0041] Device 132 is an example of a device described herein. For example, device 132 may take the form of a smart phone, a smart watch, a tablet computer, a laptop computer, a fixed or portable form of client 110, a wearable computing device, or any other suitable device. In certain embodiments, wearable sensing device 102 and device 132 communicate directly with each other, for example, via peer-to-peer Bluetooth. Figure 1 Any software application executed in another data processing system in is configured to execute in device 132 in a similar manner. Figure 1 Any data or information stored or generated in another data processing system in may be configured to be stored or generated in device 132 in a similar manner.

[0042] Applications 105 implement the embodiments described herein. Database 109 may be stored in memory 108 as shown, or may be provided by another source (not shown). Servers 104 and 106, storage unit 108, and clients 110, 112, and 114 and device 132 may be coupled to network 102 using wired connections, wireless communication protocols, or other suitable data connectivity. Clients 110, 112, and 114 may be, for example, smart phones, personal computers, vehicle computers, or network computers.

[0043] In the depicted example, server 104 may provide data such as boot files, operating system images, and applications to clients 110, 112, and 114. In this example, clients 110, 112, and 114 may be clients to server 104. Clients 110, 112, 114, or some combination thereof, may include their own data, boot files, operating system images, and applications. Data processing environment 100 may include additional servers, clients, and other devices not shown.

[0044] In the depicted example, data processing environment 100 may be the Internet. Network 102 may represent a collection of networks and gateways that communicate with each other using Transmission Control Protocol / Internet Protocol (TCP / IP) and other protocols. The heart of the Internet is a backbone of data communication links between major nodes or hosts, which include thousands of commercial, government, educational, and other computer systems that route data and messages. Of course, data processing environment 100 may also be implemented as many different types of networks, such as an intranet, a local area network (LAN), or a wide area network (WAN). Figure 1 This is intended as an example, not as an architectural limitation to the different illustrative embodiments.

[0045] Among other uses, data processing environment 100 can be used to implement a client-server environment in which illustrative embodiments can be implemented. A client-server environment enables software applications and data to be distributed across a network, thereby enabling applications to function by using interactivity between client data processing systems and server data processing systems. Data processing environment 100 can also employ a service-oriented architecture in which interoperable software components distributed over a network can be packaged together into a consistent business application. Data processing environment 100 can also take the form of a cloud and employ a cloud computing model of service delivery to enable convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly configured and released with minimal management effort or interaction with service providers.

[0046] refer to Figure 2 , which depicts a block diagram of a data processing system in which the illustrative embodiments may be implemented. Data processing system 200 is an example of a computer, such as Figure 1 Servers 104 and 106 or clients 110, 112, and 114, or another type of device, may have computer usable program code or instructions therein for implementing the processes of the illustrative embodiments.

[0047] Data processing system 200 also represents a data processing system or configuration thereof, such as Figure 1 Data processing system 132 in which computer usable program code or instructions implementing the processes of the exemplary embodiments may be placed. Data processing system 200 is described as a computer only by way of example and not limitation. Implementations in other device forms (such as Figure 1 Device 132 in the data processing system 200 may modify the data processing system 200, for example by adding a touch interface, or even eliminating certain depicted components from the data processing system 200, without departing from the general description of the operation and functionality of the data processing system 200 described herein.

[0048] In the depicted example, data processing system 200 employs a hub architecture that includes north bridge and memory controller hub (NB / MCH) 202 and south bridge and input / output (I / O) controller hub (SB / ICH) 204. Processing unit 206, main memory 208, and graphics processor 210 are coupled to north bridge and memory controller hub (NB / MCH) 202. Processing unit 206 may include one or more processors and may be implemented using one or more heterogeneous processor systems. Processing unit 206 may be a multi-core processor. In some embodiments, graphics processor 210 may be coupled to NB / MCH 202 via an accelerated graphics port (AGP).

[0049] In the depicted example, a local area network (LAN) adapter 212 is coupled to the south bridge and I / O controller hub (SB / ICH) 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read-only memory (ROM) 224, a universal serial bus (USB) and other ports 232, and a PCI / PCIe device 234 are coupled to the south bridge and I / O controller hub 204 via bus 238. A hard disk drive (HDD) or solid-state drive (SSD) 226 and a CD-ROM 230 are coupled to the south bridge and I / O controller hub 204 via bus 240. The PCI / PCIe devices 234 may include, for example, an Ethernet adapter, an add-in card, and a PC Card for a notebook computer. PCI uses a card bus controller, while PCIe does not. ROM 224 may be, for example, a flash binary input / output system (BIOS). Hard disk drive 226 and CD-ROM 230 may use, for example, an integrated drive electronics (IDE), a serial advanced technology attachment (SATA) interface, or variations such as external SATA (eSATA) and micro SATA (mSATA). Super I / O (SIO) device 236 may be coupled to south bridge and I / O controller hub (SB / ICH) 204 via bus 238.

[0050] Memories such as main memory 208, ROM 224, or flash memory (not shown) are some examples of computer usable storage devices. Hard disk drive or solid state drive 226, CD-ROM 230, and other similarly available devices are some examples of computer usable storage devices that include computer usable storage media.

[0051] An operating system runs on the processing unit 206. The operating system Figure 2 The operating system coordinates and provides control over the various components in data processing system 200. The operating system may be a commercial operating system for any type of computing platform, including but not limited to server systems, personal computers, and mobile devices. Object-oriented or other types of programming systems may operate in conjunction with the operating system and provide calls to the operating system from programs or applications executing on data processing system 200.

[0052] for operating systems, object-oriented programming systems, and such Figure 1Instructions for an application or program, such as application 105 in , are located on a storage device, such as in the form of code 226A on hard drive 226, and may be loaded into at least one of one or more memories, such as main memory 208, for execution by processing unit 206. The processes of the illustrative embodiments may be performed by processing unit 206 using computer-implemented instructions, which may be located in a computer memory, such as main memory 208, read-only memory 224, or one or more peripheral devices.

[0053] Furthermore, in one case, code 226A may be downloaded from remote system 201B via network 201A, where similar code 201C is stored on storage device 201D. In another case, code 226A may be downloaded to remote system 201B via network 201A, where downloaded code 201C is stored on storage device 201D.

[0054] Figure 1-2 The hardware in the may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical drives, etc., may be used in addition to or in place of Figure 1-2 Additionally, the processes of the illustrative embodiments may be applied to a multi-processor data processing system.

[0055] In some illustrative examples, data processing system 200 may be a personal digital assistant (PDA), which is typically configured with flash memory to provide non-volatile memory for storing operating system files and / or user-generated data. The bus system may include one or more buses, such as a system bus, an I / O bus, and a PCI bus. Of course, the bus system may be implemented using any type of communication structure or architecture that provides data transfer between different components or devices attached to the structure or architecture.

[0056] The communication unit may include one or more devices for sending and receiving data, such as a modem or a network adapter. A memory may be, for example, main memory 208 or a cache, such as that found in north bridge and memory controller hub 202. A processing unit may include one or more processors or CPUs.

[0057] Figure 1-2 The examples described in and above are not meant to imply architectural limitations. For example, data processing system 200 may be a tablet computer, laptop computer, watch, vehicle computer, or telephone device in addition to taking the form of a mobile or wearable device.

[0058] Where a computer or data processing system is described as a virtual machine, virtual device, or virtual component, the virtual machine, virtual device, or virtual component operates in the manner of data processing system 200 using virtualized representations of some or all of the components described in data processing system 200. For example, in the virtual machine, virtual device, or virtual component, processing unit 206 is represented as a virtualized instance of all or some number of hardware processing units 206 available in the host data; main memory 208 is represented as a virtualized instance of all or some portion of main memory 208 available in the host data processing system; and disk 226 is represented as a virtualized instance of all or some portion of disk 226 available in the host data processing system. In this case, the host data processing system is represented by data processing system 200.

[0059] refer to Figure 3 , which depicts a block diagram of an example configuration 300 for handwriting recognition using a wearable pressure sensing device, according to an illustrative embodiment. The wearable sensing device 302 is Figure 1 The storage and processing unit 318 is an example of a wearable sensor device 120. Figure 1 An example of a client 114, a device 132, and a server 104. Application 105 is Figure 1 The client device 320 is an example of an application 105 and may be included in one or more of the clients 110, 112, 114 and the device 132. Figure 1 An example of a client device 110 of FIG. In this embodiment, a wearable sensing device 302 is disposed on a body part 304 (such as a fingernail of a hand, a toenail of a user). In one or more embodiments, a user applies pressure to a media surface 306, such as a writing surface or a tablet computer, with a body part 304 using a motion indicative of writing.

[0060] In this embodiment, the wearable sensing device 302 includes one or more pressure sensors 308, other sensors including an accelerometer 310, a pre-processing component 312, and a transceiver 314. In one or more embodiments, the one or more wearable sensing devices 302, the memory and processing unit 318, and the client device 320 can be integrated into a single device. In one or more embodiments, the other sensors can include one or more of an accelerometer, a gyroscope, a magnetometer, or a temperature sensor. In a particular embodiment, the wearable sensing device 302 can include a power source such as a battery, which can be charged by movement of a body part such as a hand.

[0061] In one embodiment, a user with a wearable sensing device 302 disposed on a body part 304 makes a gesture on a media surface 306 using a motion indicating a written shape such as a symbol, letter, and / or command. As a result of the gesture, a pressure sensor 308 of the wearable sensing device 302 receives a pressure measurement signal from the body part 304 and provides the pressure measurement signal to a pre-processing component 312. In this embodiment, other sensors 310 obtain motion data (e.g., accelerometer data) of the motion of the body part during the gesture and provide the motion data to the pre-processing component 312. Also, the pressure measurement signal is input to a pre-processing stage in the wearable sensing device.

[0062] In an embodiment, the preprocessing component 312 of the wearable sensing device 302 preprocesses the pressure measurement input and the motion data input to normalize the sensor readings and compensate for variations between different repetitions and configurations. In one embodiment, the preprocessing includes removing one or more of a bias (e.g., a direct current (DC) component) from the pressure input data, compensating for variations in the output voltage of each sensor, filtering to exclude unwanted frequency content, such as by a Kalman filter, scaling the pressure measurement input data, and removing time artifacts.

[0063] In this embodiment, the pre-processing component 312 sends the pre-processed pressure and motion data to the transceiver 314, and the transceiver 314 sends the pre-processed pressure and motion data to the storage and processing unit 318 via the transmission sensor data medium 316. In certain embodiments, the transmission sensor data medium 316 is a wireless transmission medium. In other certain embodiments, the transmission sensor data medium 316 is a wired transmission medium.

[0064] In this embodiment, the application 105 of the storage and processing unit 318 reduces the dimensionality of the pressure measurement data and / or motion data. In one or more embodiments, the application 105 reduces the pressure measurement data and / or motion data to a lower data level to make the data more representable, or to remove artifacts of the data. In one or more embodiments, the application 105 reduces the dimensionality of the pressure measurement data and / or motion data by transferring the data to an independent component library. In a specific embodiment, the application 105 uses one or more dimensionality reduction algorithms such as principal component analysis (PCA) or independent component analysis (ICA) or feature selection algorithms to reduce the dimensionality of the pressure measurement data and / or motion data along the x and y axes.

[0065] In an embodiment, the application 105 detects the start and end boundaries of the written symbol from the pressure measurement data and / or the motion data. In order to perform writing recognition, it is necessary to accurately detect the start and end of the written symbol. In one embodiment, the state in which the body part 304 is in a stable state (not necessarily the state of minimum pressure change of the pressure sensor 308) is regarded as a reference state or a closed state. Deviation from the closed state is defined as an open state. In a specific embodiment, the deviation from the reference state is performed by an edge detection algorithm, for example by thresholding the signal at a steady-state level, or using a machine learning tool, which provides the probability of the start / end based on the signal pattern by using a trained data tool, so no threshold is required. In a specific embodiment in which multiple pressure sensors are deformed simultaneously, an envelope consisting of the average sum of the pressure measurement signals can be used to detect deviations from the reference state.

[0066] In an embodiment, the application 105 extracts features for each gesture based on the start and end boundaries of the detected written symbol. In addition to using the raw pressure measurement data and / or motion data, the application 105 also extracts spatial, temporal, and spectral features from the pressure measurement data and / or motion data. The spatial features are based on prior information about the location of the pressure sensor 308. The temporal features can include the original signal pattern in the boundaries of the pressure measurement data, as well as any statistical measurements based on the raw data, including statistical moments, such as mean, standard deviation, signal peak amplitude, average amplitude, and duration. Spectral features include information about signal behavior at different frequencies (e.g., peak and median frequencies). In a particular embodiment, another set of temporal features can be derived by the relationship between consecutive gestures. For example, the application 105 can derive the delay between adjustment gestures by the start and end of the gesture, or can obtain the correlation of the signal waves between consecutive gestures.

[0067] In an embodiment, the application 105 includes a classifier component that uses a classification algorithm to identify the symbol indicated by the extracted gesture features. In one embodiment, the classification algorithm uses the extracted features to decode information in the two-dimensional symbol of the gesture, including shapes, letters, punctuation, and commands such as backspace, space, etc. In a specific embodiment, the classifier component can use a deep learning-based classification technique such as a neural network.

[0068] In certain embodiments, the classification algorithm may be supervised based on the output of a training or calibration process, may be unsupervised based on prior knowledge about the gesture, or may be supervised by a combination of supervision and unsupervision. In certain embodiments, the classification algorithm may include an additional sensor that receives motion data (which may include an accelerometer or a gyroscope).

[0069] In one embodiment, a trained symbol database is used to store patterns of different symbols that make up letters, punctuation marks, or commands. In a specific embodiment, the database also stores symbol features to be used by the classifier. In one embodiment, the application 105 classifies symbols based on prior knowledge related to gestures, taking into account physiological motion constraints of the body part 304, including typical values, maximum and minimum feature values. In an embodiment, the prior knowledge is converted into functions of the symbol. In one embodiment, the application 105 aggregates motion data from other sensors 310, such as accelerometer and gyroscope data, and provides it to the classifier component in addition to the pressure sensor data to enable the classifier to more accurately detect the beginning and end of the symbol, estimate the position of the finger in space, and thus provide more accurate writing recognition.

[0070] In an embodiment, the application 105 uses a letter detection algorithm to determine one or more letters, punctuation marks, and commands from the symbols. In a specific embodiment, the letter detection algorithm uses the extracted features (including features between symbols such as delays between symbols) and other sensor data to estimate the letters / punctuation marks / commands. In a specific embodiment, the letter detection algorithm uses prior knowledge based on a training / calibration phase or prior statistical knowledge about the symbols that make up the letters / punctuation marks / commands. In a specific embodiment, the writing indicated by gestures can simulate keyboard letters / punctuation marks. In a specific embodiment, depending on the visual feedback, commands can be given, for example, repeat a word, delete a word, etc. In a specific embodiment, the letter detection algorithm is based on a machine learning algorithm such as a hidden Markov model (HMM) or a neural network, which can embed shapes into letters.

[0071] In an embodiment, application 105 uses a word detection algorithm to detect words based on detected letters, detected punctuation marks and their respective features. For example, the beginning and end of a new word can be identified by utilizing the time delay between letters, or by conventional or special punctuation marks. In a specific embodiment, application 105 applies an automatic correction mechanism to detect words (e.g., using automatic spelling and sentence context) based on statistical prior word knowledge. In another specific embodiment, for each word, alternatives, similar spelling considerations and statistical confidence of letters can be presented to the user in a graphical user interface with special symbols for selecting the correct word. In another specific embodiment, a graphical user interface is presented, which provides control commands of the system, such as deleting words, retyping words or selecting from multiple word selections. In a specific embodiment, word prior knowledge can include a complete word complete dictionary, a priori statistical probability of the probability of letters in a word, conditional probability of letters based on previous letters, sentence context, visual or auditory prompts and / or the relationship between different words. In a specific embodiment, the word / sentence detection algorithm is based on a machine learning algorithm, such as a hidden Markov model (HMM) or a neural network, which can embed letters into words and sentences.

[0072] In this embodiment, after recognizing a word, the storage and processing unit 318 sends the recognized word to the client device 320. Figure 3 In a particular embodiment of the present invention, the client device 320 includes a display 322, a processor 324, a memory 326, and a client application 328. The processor 324 is configured to execute software instructions stored in the memory 326 to perform one or more operations of the client device 320 described herein. In this embodiment, the client application 328 is configured to receive the recognized word and display the recognized word within a graphical user interface of the display 322.

[0073] refer to Figure 4A , which depicts an example pressure sensor configuration 400 for a wearable sensing device 302 according to an illustrative embodiment. In the embodiment of FIG. 4 , the wearable sensing device 302 includes a first (left) pressure sensor 308A, a second (middle) pressure sensor 308B, and a third (right) pressure sensor 308C disposed on a body part 304. Figure 4AIn a specific embodiment of the present invention, the first pressure sensor 308A, the second pressure sensor 308B, and the third pressure sensor 308C are disposed on the nail plate of the finger. In a specific embodiment, the first pressure sensor 308A, the second pressure sensor 308B, and the third pressure sensor 308C are each strain gauge sensors. The pressure measurement output 402 is shown as a representation of the combined pressure measurement of each of the first pressure sensor 308A, the second pressure sensor 308B, and the third pressure sensor 308C when the body part 304 is pressed against the surface. In this embodiment, the darker areas of the pressure measurement output represent greater pressure applied to the surface.

[0074] refer to Figure 4B , which depicts an example of a Figure 4A 4 is an example of pressure sensor measurements 450 for the pressure sensor configuration 400. In this embodiment, a rotation degree 452 axis is plotted relative to a sensor deformation axis 454 for each of the first pressure sensor (left SGS) 308A, the second pressure sensor (middle SGS) 308B, and the third pressure sensor (right SGS) 308C, and illustrates a non-bent state of the third pressure sensor (right SGS) 308C and a maximum bent state of the first pressure sensor (left SGS) 308A in which the maximum force is applied. Figure 4B Further included are average force graphs and normal deformation graphs for each of the first pressure sensor (left SGS) 308A, the second pressure sensor (middle SGS) 308B, and the third pressure sensor (right SGS) 308C.

[0075] refer to Figure 5 , which depicts example pre-processing operations 500 of a wearable sensor device 302 according to an illustrative embodiment. Figure 5 In the embodiment of FIG. 1 , a first pressure sensor measurement curve 502 for a first pressure sensor (left SGS) 308A, a second pressure sensor measurement curve 504 for a second pressure sensor (middle SGS) 308B, and a third pressure sensor measurement curve 506 for a third pressure sensor (right SGS) 308C are shown, which include pre-processing graphs of the measurement values ​​of each of the pressure sensors before and after pre-processing. Figure 5 In the embodiment of , pre-processing includes filtering, reducing bias, reducing unexpected frequency content, and reducing noise in the pressure measurement signal for each of the first pressure sensor (left SGS) 308A, the second pressure sensor (middle SGS) 308B, and the third pressure sensor (right SGS) 308C. Figure 5 In a specific embodiment of the invention, a Kalman filter is applied to eliminate bias.

[0076] refer to Fig. 6A, which depicts an example 600 of handwriting recognition through aggregation of basic shapes, according to an illustrative embodiment. Fig. 6A In the embodiment of FIG. 6 , a written symbol shape 602 and an associated pressure sensor signal value 604 are shown. Fig. 6A In an embodiment of the present invention, uppercase handwriting recognition by aggregation of basic symbol shapes is shown, wherein each uppercase letter is recognized by detecting the symbols that are aggregated to form the letter. Fig. 6A In the embodiment of FIG. 6 , an example 606 of a capital letter "IBM" formed by handwriting through basic symbol gestures and corresponding x and y pressure sensor values ​​is shown. The pressure sensor values ​​are compared with a database / dictionary of patterns obtained from training or calibration to decode the specific letter "IBM". In this example, by using a dictionary and pattern matching algorithm, the application 105 decodes the symbols (shapes) 1, 2, 3, and then combines them into a letter that forms a capital letter I by using temporal features and statistical knowledge.

[0077] refer to Figure 6B , which depicts another example 650 of handwriting recognition through aggregation of basic shapes in accordance with an illustrative embodiment. Figure 6B In the embodiment of the invention, the recognition of cursive alphabet handwriting by the aggregation of basic symbol shapes is shown, wherein each cursive letter is recognized by detecting each letter pattern and the beginning and end of each letter. Figure 6B In the embodiment of , cursive letter shapes 654 are shown with associated pressure sensor signal values ​​652. Figure 6B In the embodiment of FIG. 6 , an example 656 of handwriting of cursive letters forming “ibm” by basic symbol gestures and corresponding example x and y pressure sensor values ​​are shown. The pressure sensor values ​​are compared with a database of letter patterns obtained from training or calibration to decode the specific letters “ibm”.

[0078] refer to Figure 7, which depicts an example of writing recognition using a wearable pressure sensing device according to an illustrative embodiment. In example 700, pressure sensor measurements 702 from the wearable sensing device 302 for each of the first pressure sensor (left SGS) 308A, the second pressure sensor (middle SGS) 308B, and the third pressure sensor (right SGS) 308C are shown during the writing of the phrase "Call me Ishmael" by a user having the wearable sensing device 302 arranged on a body part 304 through a gesture. The example 700 also shows acceleration measurements 704 from the accelerometer of the wearable sensing device 302 during the writing of the phrase. In this embodiment, the pressure sensor measurements 704 are provided to a pressure / dynamic feature extraction component 706 of the application 105, which is configured to extract one or more pressure / dynamic features from the pressure sensor measurements 704. The acceleration measurements 704 are provided to a motion feature extraction component 708, which is configured to extract one or more motion features, such as spacing, timing, or dedicated gestures between words, from the acceleration measurements 704. In an embodiment, the extracted pressure / dynamic features and the extracted motion features are provided to a classifier component 710 of the application 105. The classifier component 710 is configured to classify the pressure measurements 702 and the acceleration measurements 704 using the extracted pressure / dynamic features and the extracted motion features to produce an output 712 of “Call me Ishmael”.

[0079] refer to Figure 8 , which depicts a flow chart of an example process for using a wearable pressure sensing device for handwriting recognition according to an illustrative embodiment. Process 800 may be performed at Figure 1 or Figure 3 The wearable sensing device 302 may be implemented in the application 105. In block 802, the wearable sensing device 302 receives pressure sensor data from one or more pressure sensors disposed on the body part 304. In block 804, the wearable sensing device 302 receives accelerometer data from signals of one or more accelerometers of the wearable sensing device 302.

[0080] In block 806, the wearable sensing device 302 pre-processes the pressure sensor data and the accelerometer data to normalize the sensor readings and / or compensate for variations between different iterations and configurations. In one embodiment, the pre-processing includes one or more of: removing a DC component from the pressure input data to compensate for variations in the output voltage of each sensor, scaling the pressure measurement input data, and removing temporal artifacts.

[0081] In block 808, the application 105 reduces the dimensionality of the pre-processed pressure sensor data and / or accelerometer data. In one or more embodiments, the application 105 reduces the pressure measurement data and / or motion data to a lower data level to make the data more representable, or to remove artifacts from the data. In certain embodiments, the application 105 reduces the dimensionality of the pressure measurement data and / or motion data using one or more dimensionality reduction algorithms such as PCA or ICA.

[0082] In box 810, the application 105 detects the start and end boundaries of the written symbol from the pressure sensor data and / or accelerometer data. In box 812, the application 105 extracts features for each gesture based on the detected start and end boundaries of the symbol. In one or more embodiments, the features include spatial, temporal, and spectral features of the pressure measurement data and / or motion data. The spatial features are based on prior information about the location of the pressure sensor 308. The temporal features include the original signal pattern within the pressure measurement data boundaries, and any statistical measures based on the original data, including statistical moments, such as mean, standard deviation, signal peak amplitude, average amplitude, and duration. Spectral features include information about the behavior of the signal at different frequencies (e.g., peak and median frequencies). In a particular embodiment, another set of temporal features can be derived from the relationship between consecutive gestures. For example, the application 105 can derive the delay between adjustment gestures through the start and end of the gesture, or can obtain the correlation of the signal waves between consecutive gestures.

[0083] In box 814, the application 105 detects symbol patterns from the extracted gesture features. In box 816, the application 105 uses the detected symbols to detect letters, punctuation, and / or commands based on the clustering of the detected symbols. In a specific embodiment, the application 105 uses a priori knowledge based on a training / calibration phase, or a priori statistical knowledge about the symbols that make up letters / punctuation / commands. In a specific embodiment, letter detection is based on a machine learning algorithm such as an HMM or a neural network.

[0084] In box 818, the application 105 uses a word detection algorithm to detect words based on the detected letters, the detected punctuation marks and their respective features. In a specific embodiment, the application 105 detects words based on statistical prior word knowledge. In box 820, the application 105 outputs the detected words. Thereafter, the process 800 ends.

[0085] refer to Fig. 9 , which depicts a flow chart of an example process for implementing handwriting recognition for a virtual keyboard using a wearable pressure sensing device according to an illustrative embodiment. Process 900 may be performed at Figure 1 or Figure 3is implemented in the application 105. Fig. 9 In an embodiment of the present invention, a virtual keyboard is implemented, wherein the user's gestures are interpreted as typing on the virtual keyboard. In block 902, the wearable sensing device 302 receives pressure sensor data from one or more pressure sensors disposed on a body part 304. In block 904, the wearable sensing device 302 receives accelerometer data from one or more accelerometers of the wearable sensing device 302.

[0086] In block 906, the wearable sensing device 302 pre-processes the pressure sensor data and the accelerometer data to normalize the sensor readings and / or compensate for variations between different iterations and configurations. In one embodiment, the pre-processing includes one or more of: removing a DC component from the pressure input data to compensate for variations in the output voltage of each sensor, scaling the pressure measurement input data, and removing temporal artifacts.

[0087] In block 908, the application 105 reduces the dimensionality of the pre-processed pressure sensor data and / or accelerometer data. In one or more embodiments, the application 105 reduces the pressure measurement data and / or motion data to a lower data level to make the data more representable or to remove artifacts of the data. In certain embodiments, the application 105 reduces the dimensionality of the pressure measurement data and / or motion data using one or more dimensionality reduction algorithms such as PCA or ICA.

[0088] In block 910, the application 105 detects the starting position of the virtual keyboard finger reference and the start and end boundaries of the symbol from the pressure sensor data and / or accelerometer data. In block 912, the application 105 extracts features of each finger position and pressure measurement data associated with the gesture. In one or more embodiments, the features include spatial, temporal, and spectral features of the pressure measurement data and / or motion data.

[0089] In block 914, the application 105 detects a symbol pattern from the extracted gesture features. In block 916, the application 105 detects letters, sentences, punctuation, and / or commands based on the set of detected symbols using the detected symbols. In a particular embodiment, the application 105 detects letters, sentences, punctuation, and / or commands using one or more of a trained keyboard / letter dictionary with prior knowledge of letters, words, sentences, punctuation, and commands, prior knowledge about keyboard configuration, and accelerometer data.

[0090] In box 918, application 105 detects words and sentences based on detected letters, detected letters, sentences and punctuation marks. In a specific embodiment, application 105 detects words based on statistical prior word knowledge. In box 920, application 105 outputs detected words and sentences. Thereafter process 900 ends.

[0091] Reference Fig.10 , which depicts a flow chart of an example process for performing handwriting recognition for diagnosis of a degenerative medical condition using a wearable pressure sensing device in accordance with an illustrative embodiment. In certain embodiments, the degenerative medical condition is measured by a Unified Huntington's Disease Rating Scale (UHDRS) score or a Unified Parkinson's Disease Rating Scale (UPDRS) score. In other embodiments, the degenerative medical condition may use any other suitable medical score indicative of a degenerative medical condition. Process 1000 may be performed at Figure 1 or Figure 3 The wearable sensing device 302 may be implemented in an application 105 of the present invention. In an embodiment, a UHDRS, UPDRS, or other medical condition test is implemented, wherein the user's gestures are interpreted as a UHDRS, UPDRS, or other personal medical score rating metric. In box 1002, the wearable sensing device 302 receives pressure sensor data from one or more pressure sensors disposed on a body part 304. In box 1004, the wearable sensing device 302 receives additional sensor data such as accelerometer data from one or more additional sensors of the wearable sensing device 302.

[0092] In block 1106, the wearable sensing device 302 pre-processes the pressure sensor data and other sensor data to normalize sensor readings and / or compensate for variations between different repetitions and configurations. In one embodiment, the pre-processing includes one or more of: removing biases such as DC component bias from the pressure input data to compensate for variations in the output voltage of each sensor, scaling the pressure measurement input data, and removing temporal artifacts.

[0093] In block 1008, the application 105 reduces the dimensionality of the preprocessed pressure sensor data and / or the additional sensor data. In one or more embodiments, the application 105 reduces the pressure measurement data and / or the motion data to a lower data level to make the data more representable, or to remove artifacts of the data. In a particular embodiment, the application 105 reduces the dimensionality of the pressure measurement data and / or the motion data using one or more dimensionality reduction algorithms such as PCA or ICA.

[0094] In block 1010, the application 105 detects the start and end boundaries of letters or other symbols from the pressure sensor data and / or accelerometer data. In block 1012, the application 105 extracts features for the pressure measurement data associated with each sensor location and gesture relative to the prior knowledge database / training. In one or more embodiments, the features include spatial, temporal, and spectral features of the pressure measurement data and / or motion data.

[0095] In box 1014, the application 105 associates the extracted features with a clinical score. In a particular embodiment, the application 105 associates the extracted features with a clinical score based on one or more of a trained system from a system trained from a UHDRS, UPDRS or other medical test, a score database from a personalized UHDRS, UPDRS or other medical, and / or strain gauge and accelerometer data from a calibration reference. In box 1016, the application 105 estimates a UHDRS score, a UPDRS score or other medical score that can be associated with the writing features. In box 1018, the application 105 outputs a UHDRS, UPDRS or other medical diagnostic score. Thereafter, the process 1000 ends.

[0096] Thus, in illustrative embodiments, computer-implemented methods, systems or apparatuses, and computer program products are provided for handwriting recognition using wearable pressure sensing devices and other related features, functions, or operations. Where an embodiment or a portion thereof is a description of a type of device, the computer-implemented method, system or apparatus, computer program product, or a portion thereof, is adapted or configured for use with a suitable and comparable representation of that type of device.

[0097] Where embodiments are described as being implemented in an application, it may be considered within the scope of the exemplary embodiments to deliver the application in a software as a service (SaaS) model. In the SaaS model, the capabilities of the application implementing the embodiments are provided to users by executing the application in a cloud infrastructure. Users may access the application using a variety of client devices through a thin client interface such as a web browser (e.g., web-based email) or other lightweight client applications. Users do not manage or control the underlying cloud infrastructure including the network, servers, operating system, or storage of the cloud infrastructure. In some cases, users may not even be able to manage or control the functionality of the SaaS application. In certain other cases, the SaaS implementation of the application may allow limited possible exceptions to user-specific application configuration settings.

[0098] The present invention may be a system, method and / or computer program product that is integrated at any possible level of technical detail, by itself or in part with other data, sensors, computer systems, algorithms, trends or machine learning / artificial intelligence. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0099] A computer-readable storage medium may be a tangible device that can retain and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device (e.g., a punch card or a raised structure in a groove on which instructions are recorded), and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be understood as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0100] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network). A local area network and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0101] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc. and procedural programming languages ​​such as "C" programming languages, or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may be personalized electronic circuits by executing computer-readable program instructions using the state information of computer-readable program instructions to perform aspects of the present invention.

[0102] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It will be understood that each frame of the flow chart diagram and / or block diagram and the combination of frames in the flow chart diagram and / or block diagram can be implemented by computer-readable program instructions.

[0103] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in the flowchart and / or block diagram box. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can instruct the computer, programmable data processing device, and / or other equipment to function in a specific manner, so that the computer-readable storage medium with instructions stored therein. Includes a manufactured product that includes instructions for implementing various aspects of the functions / actions specified in the flowchart and / or block diagram box.

[0104] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device, so that a series of operating steps are performed on the computer, other programmable device, or other device to produce an implemented computer processing process, so that the instructions executed on the computer, other programmable device, or other device implement the functions / actions specified in the flowchart and / or block diagram blocks.

[0105] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each square frame in the flow chart or block diagram can represent a module, a fragment or a part of an instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function indicated in the square frame may not occur in the order indicated in the figure.For example, depending on the function involved, the two boxes shown in succession can be performed substantially at the same time, or these boxes can sometimes be performed in reverse order.It should also be noted that each square frame of the block diagram and / or flow chart and the combination of the square frames of the block diagram and / or flow chart can be realized by performing a specified function or action or based on a hardware-based special-purpose system or a combination of hardware and computer instructions for performing special purposes.

Claims

1. A method for handwriting recognition using a wearable pressure sensing device, comprising: receiving pressure measurement data from a pressure sensor disposed on a body part of a user, the pressure measurement data indicating changes in pressure of the body part due to interaction of the body part with a medium indicative of a writing gesture of the user; detecting a start boundary and an end boundary of each of a plurality of written symbols based on the pressure measurement data by detecting a reference state in which the body part is in a stable state as a closed state, and by detecting an open state based on a deviation from the closed state, wherein the detection of the deviation from the closed state is performed using an edge detection technique; extracting at least one feature of the pressure measurement data associated with the plurality of written symbols, wherein the at least one feature comprises a temporal feature of the pressure measurement data, the temporal feature comprising a raw signal pattern within a boundary of the pressure measurement data, and a statistical measure based on the raw data of the pressure sensor, the statistical measure comprising a statistical moment; detecting a symbol pattern based on the extracted features; detecting at least one letter based on the symbol pattern; and A word is detected based on the detected at least one letter.

2. The method of claim 1, further comprising: Motion data is received from a motion sensor disposed on the body part of the user, wherein the detection of the symbol pattern is also based on the motion data.

3. The method of claim 1, further comprising: The dimensionality of the pressure measurement data is reduced prior to said detecting the start boundary and the end boundary.

4. The method of claim 3, further comprising: The pressure measurement data are pre-processed before said reducing the dimensionality of the pressure measurement data.

5. The method of claim 4, wherein: Preprocessing of the pressure measurement data includes one or more of: filtering bias and undesired frequency content from the pressure measurement data; scaling the pressure measurement data; and removing temporal artifacts from the pressure measurement data.

6. The method of claim 1, further comprising: A command is detected based on the extracted features.

7. The method of claim 1, further comprising: Symbols of the virtual keyboard are detected based on the extracted features.

8. The method of claim 1, wherein: The at least one characteristic comprises one or more of a spatial characteristic and a spectral characteristic of the pressure measurement data.

9. The method of claim 1, wherein: The detection of the symbol pattern is based on a priori statistical knowledge.

10. The method of claim 1, wherein: The detecting of the at least one letter is performed using at least one of machine learning on the extracted features or on the raw data of the pressure sensor together with additional sensor data.

11. A system for handwriting recognition comprising means adapted to perform all the steps of the method according to any one of claims 1-10.

12. A computer program product for handwriting recognition, comprising instructions for executing all the steps of the method according to any one of claims 1 to 10.

Images