Sensors

Abstract

A sensor device is provided. The sensor device includes an elastic light-transmitting layer that changes optical properties in response to deformation. First and second light sources respectively emit first and second incident light toward a skin surface of a user, and the first incident light is emitted through said layer. A light detector detects the first and second reflected light reflected from different epidermal depths. The sensor device can be used to detect movement of subcutaneous tissue. A model generated from one or more sensor devices and sensed data associated with different muscle states or physical positions of the user can be used to estimate a physical position or movement of a body part of the user. Such estimated physical position or movement can be used to operate other devices, such as humanoid robotics, powered exoskeletons, and active prosthetic limbs.

Description

[Technical field]

[0001] The present invention relates to sensor devices used to detect movement of subcutaneous tissue from the skin surface, and more particularly, but not exclusively, to sensors and sensor combinations used to estimate the physical position or movement of a user and control a robotic device based on the estimated position or movement. [Background technology]

[0002] The following contains information that may be helpful in understanding the present invention. There is no admission that any of the information provided herein is prior art to or relevant to the presently described or claimed invention, or that any of the publications or documents specifically or implicitly referenced are prior art. Any discussion of prior art throughout this specification should in no way be construed as an admission that such prior art is widely known or forms part of the general knowledge in the art.

[0003] Human-machine interfaces translate user input into machine-actionable output. Traditional machine control relies on conscious human interaction. Examples of conscious control interfaces include keyboards, joysticks, and touchscreen displays. In contrast, innate machine interfaces translate physiological output into a form usable for machine control. For example, electromyography (EMG) interfaces measure electrical activation patterns of skeletal muscles that represent muscle movement. Electrical signals obtained from electromyography (EMG) measurements can be processed to distinguish and / or infer different types of movement and identify user intent. When non-invasive sensing is required, surface electromyography (sEMG) can be utilized. sEMG interfaces may require complex electronics for data acquisition and processing, and may require precise placement of gel-based electrodes to obtain accurate data. The effectiveness of gel-based electrodes is sensitive to moisture.

[0004] Force myography (FMG) interfaces can also be used to capture pressure from muscle contractions to identify user movements, such as hand gestures and different types of grips a user's hands may assume. Because precisely placed gel-based electrodes are not required, FMG interfaces are less susceptible to inaccuracies in interface placement or moisture. However, FMG interfaces do not detect muscle activation direction, but rather based on changes in sensed muscle mass, which can limit their accuracy.

[0005] Another type of interface that is available is an electroencephalogram (EEG) interface configured, for example, as a helmet or other wearable.

[0006] Vision-based systems can also be used to attempt to identify user movements, but they are susceptible to occlusions and changes in lighting and can require significant training to provide adequate accuracy.

[0007] Innate machine interfaces can interpret user intent without conscious user interaction. Machine interfaces can also be more intuitive than traditional controls when the machine mimics human movements. As such, machine interfaces are particularly suited to applications where conscious interaction with the machine is a disadvantage. Humanoid robotics, powered exoskeletons, and active prosthetic limbs are some examples where innate machine interfaces can be used. For example, an electromyography (EMG) interface can sense the electrical activation of a user's forearm muscles (the muscles responsible for hand gestures and / or various types of grips) and generate control outputs for a grasping device (such as a robotic gripper, a powered prosthetic hand, or an exoskeleton glove). Similar examples can be found for devices that mimic lower limbs or other appendages.

[0008] Robotic grippers, actuated prosthetic hands, and exoskeleton gloves are examples of soft robotic grasping devices that attempt to replicate, assist, or enhance human manipulation and / or grip. Robotic grippers have been developed for a variety of applications, including fruit picking, vehicle assembly, and material handling. There are two basic categories of robotic grippers: vacuum grippers and actuated grippers (such as pneumatic, hydraulic, and servo-electric grippers). Some actuated gripping systems can also be used in other applications, such as prosthetic hands and grip-augmenting gloves, which tend to mimic anthropomorphic characteristics (i.e., human-like characteristics).

[0009] Actuated prosthetic hands aim to restore the form and function of the human hand. Partial prosthetic hands are used when the recipient has lost one or more fingers. Total prosthetic hands are used when the recipient has lost the entire hand. Both types of prosthetic hands interface with the recipient in some way and translate the recipient's intentions into finger and / or hand movements. This can be accomplished through a mechanical system that transfers forces from another part of the recipient's body to the prosthetic hand (i.e., an active prosthetic hand), or through sensors (e.g., electromyography (EMG) sensors controlling a motorized system).

[0010] Grip-enhancing gloves can be used to increase the functionality of a recipient's hand. They can be used to increase fatigue resistance to demanding gripping tasks (e.g., repetitive or heavy lifting), aid in rehabilitation (e.g., for stroke patients), and improve long-term mobility and / or strength in recipients with neurodegenerative and / or musculoskeletal diseases (e.g., arthritis, cerebral palsy, and Parkinson's disease). These devices typically include artificial tendons (e.g., cables or pneumatic / hydraulic lines) that extend from some form of actuator to the fingertips of the glove. Activating the actuator pulls the fingers toward the palm, replicating the recipient's natural grip.

[0011] The form and function of a gripping device is often defined by its intended use: for example, a gripping device intended to replace or augment the human hand is expected to be wearable (e.g., battery-powered, lightweight and sized appropriately for the recipient), whereas grippers used on industrial assembly lines often prioritize gripping and / or lifting forces.

[0012] Incorporation by Reference U.S. Provisional Patent Application No. 63 / 272,669, entitled "SENSOR," filed on October 27, 2021, is incorporated by reference in its entirety into this specification, i.e., the specification, claims, drawings, Appendix 1, and Appendix 2 should be considered part of the incorporated disclosure. Summary of the Invention [Problem to be solved by the invention]

[0013] It is an object of the present invention to provide an improved sensor device that addresses or ameliorates one or more disadvantages or limitations associated with the prior art, or at least provides the public with a useful choice. [Means for solving the problem]

[0014] In a first aspect, the present disclosure provides a sensor device for detecting subcutaneous tissue movement from a skin surface of a user, the sensor device comprising: an elastic light-transmitting layer having optical properties that change in response to deformation of the light-transmitting layer; a first light source configured to emit a first incident light through the light transmitting layer toward the skin surface; a second light source configured to emit a second incident light toward the skin surface; a photodetector configured to detect a first reflected light representative of a reflection of the first incident light by a skin tissue layer or at or near the skin surface and a second reflected light representative of a reflection of the second incident light by a subcutaneous tissue layer; Equipped with.

[0015] The photodetector is disposed on the light-transmitting layer and detects the first reflected light and the second reflected light through the light-transmitting layer.

[0016] At least a portion of the side of the light transmissive layer that is configured to contact the skin surface of the user is reflective and configured to reflect light of the first incident light.

[0017] The first light source is disposed on the light-transmitting layer and emits a first incident light through the light-transmitting layer.

[0018] The light-transmitting layer has a thickness of approximately 5 mm between a side of the light-transmitting layer configured to contact the user's skin surface and an opposing side on which the first light source is provided, thereby increasing the intensity of the detected first reflected light.

[0019] The second light source is disposed on the user's skin surface and configured to emit a second incident light directly through the user's skin surface.

[0020] The light-transmitting layer, the first light source, the second light source, and the light detector define a sensor module, and the sensor device includes a plurality of sensor modules, each sensor module being used to detect movement of subcutaneous tissue associated with a different muscle group of the user.

[0021] The first incident light has a first wavelength and the second incident light has a second wavelength, the first wavelength being shorter than the second wavelength.

[0022] The first wavelength is in the range of about 500 nm to about 565 nm, and the second wavelength is in the range of about 625 nm to about 1,400 nm.

[0023] The light-transmitting layer is configured to elastically deform, and deformation of the light-transmitting layer affects one of the path and intensity of light traversing the light-transmitting layer.

[0024] The first and second light sources are configured to non-simultaneously emit first and second incident light beams, respectively, toward a skin surface of a user.

[0025] The sensor device further comprises: a band configured to apply a biasing force to the light transmissive layer against a skin surface of the user; a processor configured to estimate a movement of the subcutaneous tissue based on the detected first reflected light and the second reflected light; The device is provided with at least one of the following:

[0026] The processor is configured to provide the detected first reflected light and second reflected light values ​​as inputs to a model and to determine a gesture-state of the user's body part from an output of the model.

[0027] The first reflected light represents a reflection of the first incident light by a skin tissue layer of tissue, the skin layer being the epidermis layer.

[0028] In another aspect, the present disclosure provides a method of estimating a muscle contraction state of a target muscle using a sensor apparatus as described herein, the method comprising: receiving, using a processor, sensed values ​​of each of the first reflected light and the second reflected light at both a first time point and a second time point; using a processor to estimate a deformation of a skin area adjacent the user's skin based on a change in the sensed value of the first reflected light; using a processor to estimate a deformation of a subcutaneous region adjacent the skin region of the user based on the change in the sensed value of the second reflected light; using a processor to estimate a muscle contraction state of the target muscle based on the estimated skin deformation and the estimated subcutaneous deformation; Includes.

[0029] The aforementioned steps are repeated at different times to provide multiple temporal estimates of muscle contraction states of the target muscle, and the multiple temporal estimates of the muscle contraction states are used to infer a gestural movement of the user's body part associated with the target muscle.

[0030] In another aspect, the present disclosure provides a method for producing a method for manufacturing a pharmaceutical composition comprising: a light-transmitting layer that is elastically deformable to cause a corresponding change in the optical properties of light traversing the light-transmitting layer; a first light emitting component configured to emit light through the light transmitting layer toward the skin site for reflection by a corresponding epidermal skin portion; a second light emitting component configured to emit light toward the skin site for reflection by a corresponding non-epidermal skin portion; a photodetector configured to detect light reflected by the epidermal and non-epidermal skin portions; A sensor device including:

[0031] The non-epidermal skin portion is the subcutaneous tissue portion.

[0032] The light-transmitting layer is adapted to be placed on the skin site and configured to position the first light-emitting component 5 mm away from the skin site, thereby increasing the intensity of light reflected by the epidermal skin portion and detected by the photodetector.

[0033] The photodetector is configured to detect the reflected light through the light transmissive layer.

[0034] In another aspect of the present disclosure, a method is provided that includes estimating subcutaneous tissue movement from a composite light measurement, the composite light measurement including light at a first wavelength reflected from a surface of the skin and light at a second wavelength reflected from subcutaneous tissue below the surface of the skin.

[0035] In another aspect of the present disclosure, a sensor is provided, the sensor comprising: The device comprises a first light source configured to be disposed on or adjacent to the recipient's skin and to direct incident light at a first wavelength through at least an upper layer of the recipient's skin; a second light source configured to direct incident light at a second wavelength onto the recipient's skin proximate to the first light source; a compliant pad made of a transparent or translucent elastomer, the compliant pad configured to be disposed between the second light source and the recipient's skin; and at least one photodetector configured to receive light reflected from the recipient's skin and / or subcutaneous tissue and measure the intensity of the light at the first wavelength and the intensity of the light at the second wavelength.

[0036] In another aspect of the present disclosure, a method is provided, the method comprising: obtaining a first deformation measurement, the first deformation measurement representing a deformation of an epidermal layer of the skin; obtaining a second deformation measurement, the second deformation measurement representing deformation of a dermis layer of the skin adjacent the epidermis layer represented by the first deformation measurement; combining the first deformation measurement and the second deformation measurement to generate an estimate of the movement of the subcutaneous tissue; Includes.

[0037] In another aspect of the present disclosure, a belt is provided, the belt including a first optical sensor configured to detect displacement of skin beneath the belt and a second optical sensor configured to detect displacement of soft tissue beneath the skin surface.

[0038] In another aspect of the present disclosure, an apparatus is provided, the apparatus comprising a strap having a first optical sensor disposed adjacent an inner surface of the strap and a second optical sensor circumferentially spaced from the first optical sensor, the second optical sensor being offset outwardly from the inner surface of the strap relative to the first optical sensor, the strap comprising a compressible elastomeric pad disposed between the second optical sensor and the inner surface of the strap.

[0039] In another aspect of the present disclosure, a band is provided comprising at least two optical sensors configured to detect displacement of soft tissue proximate to the band, the at least two optical sensors operating in different frequency ranges of the optical spectrum, and the band is configured to position the at least two optical sensors at different distances from the skin.

[0040] In another aspect, the present disclosure provides a method including estimating subcutaneous tissue movement from a composite light measurement, the composite light measurement including light of a first wavelength reflected from a surface of the skin and light of a second wavelength reflected from subcutaneous tissue below the surface of the skin.

[0041] The method includes obtaining a deformation estimate of an epidermis layer of the skin from the intensity of light captured at a first wavelength, and obtaining a deformation estimate of a dermis layer of the skin from the intensity of light captured at a second wavelength.

[0042] The method includes combining deformation estimates of the epidermal and dermal layers of the skin to generate an estimate of the motion of subcutaneous tissue.

[0043] The method includes calculating an estimate of muscle and / or tendon contraction from the intensity of light at the first wavelength and the intensity of light at the second wavelength present in the combined light measurement.

[0044] The method includes actuating the artificial limb in response to the calculated estimate of muscle contraction.

[0045] This method is directing light from a first light source onto a surface of the skin for a first time while simultaneously measuring the intensity of the light reflected from the surface of the skin with a photodetector; directing light from a second light source onto the surface of the skin for a second period of time while measuring the intensity of the light reflected from the subcutaneous tissue with a photodetector; wherein the first time and the second time are not simultaneous.

[0046] In another aspect, the present disclosure provides a method for producing a method for manufacturing a pharmaceutical composition comprising: a first light source disposed on or adjacent to the recipient's skin and configured to direct incident light at a first wavelength through at least an upper layer of the recipient's skin; a second light source configured to direct incident light of a second wavelength onto the skin of the recipient proximal to the first light source; a compliant pad made of a transparent or translucent elastomer, the compliant pad configured to be placed between the second light source and the skin of the recipient; at least one photodetector configured to receive light reflected from the skin and / or subcutaneous tissue of the recipient and measure an intensity of the light at the first wavelength and an intensity of the light at the second wavelength; The present invention provides a sensor including:

[0047] The sensor comprises a band that holds the first light source, the second light source, the compliant pad, and at least one photodetector in proximity to the skin, and the band is configured to preload the compliant pad against the recipient's skin by applying pressure to the recipient's skin.

[0048] The band is adapted to encircle the recipient's limb.

[0049] The compliant pad is configured to direct light from the second light source to the skin of the recipient.

[0050] The compliant pad is configured to absorb a variable amount of light at the second wavelength when the pad is compressed, the amount of light that the compliant pad absorbs being dependent on the deformation of the elastomer.

[0051] The second light source has a wavelength greater than 650 nm and the first light source has a wavelength less than 550 nm.

[0052] In another aspect, the present disclosure provides a method for producing a method for manufacturing a pharmaceutical composition comprising: obtaining a first deformation measurement, the first deformation measurement representing a deformation of an epidermal layer of the skin; obtaining a second deformation measurement, the second deformation measurement representing deformation of a dermis layer of the skin adjacent the epidermis layer represented by the first deformation measurement; combining the first deformation measurement and the second deformation measurement to generate an estimate of the movement of the subcutaneous tissue; the method comprising estimating a first deformation measurement from the intensity of light reflected from the skin at a first wavelength; and estimating a second deformation measurement from the intensity of light reflected from the skin at a second wavelength.

[0053] The method includes interleaving light from a first light source emitting light at a first wavelength with light from a second light source emitting light at a second wavelength, and measuring the intensity of the light from the first light source and the intensity of the light from the second light source with a single photodetector.

[0054] In another aspect, the present disclosure provides a belt including a first optical sensor configured to detect displacement of skin beneath the belt and a second optical sensor configured to detect displacement of soft tissue beneath the skin surface.

[0055] A first optical sensor is held on the surface of the skin and operates at wavelengths between 625 nm and 1 mm, and a second optical sensor is held adjacent to the skin and operates at wavelengths between 10 nm and 565 nm.

[0056] In another aspect, the present disclosure provides an apparatus including a strap comprising a first optical sensor disposed adjacent an inner surface of the strap and a second optical sensor circumferentially spaced from the first optical sensor, the second optical sensor being offset outwardly from the inner surface of the strap compared to the first optical sensor, the strap comprising a compressible elastomeric pad disposed between the second optical sensor and the inner surface of the strap.

[0057] The second light sensor is offset from the inner surface of the strap by more than 2 mm, and an elastomeric pad occupies the space between the second light source and the inner surface of the strap.

[0058] The first light sensor comprises a first light source having a first wavelength and a first light detector, and the second light sensor comprises a second light source having a second wavelength and a second light detector.

[0059] The strap is configured to apply a radial compressive force to the recipient's limb, thereby positioning the first optical sensor in contact with or immediately adjacent to the recipient's skin.

[0060] In another aspect, the present disclosure provides a band including at least two optical sensors configured to detect displacement of soft tissue proximate to the band, the at least two optical sensors operating in different frequency ranges of the light spectrum, and the band configured to position the at least two optical sensors at different distances from the skin.

[0061] The band includes a compliant elastomeric material disposed between a first optical sensor of the at least two optical sensors and an inner circumference of the band, the compliant elastomeric material configured to contact the skin and to deform in response to movement of soft tissue immediately beneath the contacted skin.

[0062] The compliant elastomeric material includes a dopant configured to modify a transmittance of light through the compliant elastomeric material from a first optical sensor of the at least two optical sensors in response to a deformation of the compliant elastomeric material.

[0063] A first sensor of the at least two sensors is configured to measure deformation of the compliant elastomeric material as a proxy for displacement of the soft tissue, and a second sensor of the at least two sensors is positioned closer to the skin than the first sensor of the at least two sensors, and the second sensor of the at least two sensors is configured to measure displacement of the soft tissue beneath the skin.

[0064] A first light sensor of the at least two light sensors includes a red or infrared LED light source and a second light sensor of the at least two light sensors includes a green, blue, violet, or ultraviolet LED light source.

[0065] In another aspect, the present disclosure provides a sensor device including a light-transmitting layer elastically deformable to cause a corresponding change in optical properties of light traversing the light-transmitting layer, a first light-emitting component configured to emit light through the light-transmitting layer toward a skin site for reflection by a corresponding epidermal skin portion, a second light-emitting component configured to emit light toward the skin site for reflection by a corresponding non-epidermal skin portion, and a photodetector configured to detect light reflected by the epidermal and non-epidermal skin portions.

[0066] The photodetector may preferably be configured to detect light reflected through the light transmitting layer.

[0067] Preferably, the optical properties may include at least one of a path and an intensity.

[0068] The light-transmitting layer is adapted to be placed on the skin site and is preferably configured to position the first light-emitting component 5 mm away from the skin site, thereby increasing the intensity of light reflected by the epidermal skin portion and detected by the photodetector.

[0069] The first and second light emitting components may be configured to emit light alternately.

[0070] The non-epidermal skin portion can be a subcutaneous tissue portion.

[0071] Preferably, the sensor device further comprises a reflective layer configured to reflect light reflected by either the epidermal skin portion and the non-epidermal skin portion and not detected by the photodetector towards the skin site so as to be reflected by corresponding portions of the epidermal skin portion and the non-epidermal skin portion.

[0072] The sensor device may further comprise at least one of a fixation component adapted to maintain contact between the light transmissive layer and the skin site and a processor configured to determine movement of subcutaneous tissue at the skin site based on light detected by the light detector.

[0073] According to another aspect, the present disclosure provides a method of determining movement of subcutaneous tissue, the method including: determining, using a processor, a deformation of an epidermal skin portion of a skin site based on light reflected by the epidermal skin portion and detected by a photodetector; determining, using a processor, a deformation of a non-epidermal skin portion of the skin site based on light reflected by the non-epidermal skin portion and detected by the photodetector; and determining, using a processor, a movement of subcutaneous tissue based on the determined deformation.

[0074] According to another aspect, a method for determining muscle movement is provided, the method including receiving feature data representative of detection of light reflected by an epidermal skin layer and light reflected by a non-epidermal skin layer, and performing a regression operation on the feature data to determine muscle movement.

[0075] Preferably, the received feature data represents raw data generated by the at least one photodetector.Preferably, the received feature data represents raw photodetector data.

[0076] The regression operation preferably relates to one of a Random Forest (RF) process, a Convolutional Neural Network (CNN) process, and a Temporal Multi-Channel Vision Transformer (TMC-Vit) process.

[0077] Preferably, the method further comprises determining one of a gesture and a force based on the determined muscle movement.

[0078] In another aspect, the present disclosure provides a method for determining muscle movement, the method including receiving feature data representative of detection of light reflected by an epidermal skin layer and light reflected by a non-epidermal skin layer, and performing a regression operation on the feature data to determine muscle movement.

[0079] The received feature data preferably represents raw data generated by at least one optical detector.

[0080] The method may further include determining a gesture based on the determined muscle movement.

[0081] The term "and / or" can mean "and" or "or."

[0082] The terms "property" and "characteristic" may be used interchangeably herein.

[0083] As used herein, "(s)" following a noun refers to the plural and / or the singular form of that noun.

[0084] As used in this specification and the claims, the words "comprise", "comprises", "comprising" and similar terms should not be construed in an exclusive or exhaustive sense. In other words, they are intended to mean "including, but not limited to". When interpreting each statement in this specification that includes the term "comprise", "comprises" or "comprising", there may be features other than those prefaced by this term.

[0085] As used herein, the term "axis" refers to an axis of rotation about which a line or plane can be rotated to form a symmetrical shape. For example, a line rotated about an axis of rotation forms a surface, while a plane rotated about an axis of rotation forms a solid.

[0086] For the purposes of this specification, the term "plastic" shall be taken to refer generically to a wide range of synthetic or semi-synthetic polymeric materials, generally consisting of hydrocarbon-based polymers.

[0087] Any method detailed herein corresponds to a disclosure of an apparatus and / or system configured to perform one, more than one, or all of the method actions. Similarly, any apparatus and / or system disclosure detailed herein corresponds to a method of making and / or using the apparatus and / or system, including a method of using the apparatus in accordance with the functions detailed herein. Any apparatus and / or system disclosure detailed herein also corresponds to a disclosure of providing the apparatus and / or system in other ways.

[0088] For purposes of this specification and the claims, when method steps are described in a sequence, the sequence does not necessarily imply that the steps are chronologically arranged in that sequence, unless there is no other logical way to interpret the sequence.

[0089] It should be noted that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present invention and without diminishing its attendant advantages. Accordingly, such changes and modifications are intended to be included in the present invention.

[0090] It is to be understood that the disclosed aspects are provided by way of example only, and that changes, modifications, and additions may be made without departing from the scope of the present disclosure. Furthermore, where known equivalents exist for specific features, such equivalents are incorporated as if specifically set forth herein. Thus, where reference is made herein to integers or components that have known equivalents, those integers are incorporated herein as if individually set forth.

[0091] The present invention broadly includes the parts, elements and features referred to or shown in the specification of this application, individually or collectively, any or all combinations of two or more of said parts, elements or features.

[0092] Unless otherwise specified or feasible by the art, any one or more teachings detailed herein with respect to one embodiment or example may be combined with one or more teachings of any other teachings detailed herein with respect to other embodiments or examples, including overlapping or repetition of any given teaching of a component with any similar components.

[0093] Some exemplary embodiments include the use of devices to perform some or all of the method actions detailed herein. In some exemplary embodiments, these devices include or are logic circuits or electronics such as a processor, which includes or can access a memory component. Alternatively and / or in addition, a computer chip can be configured or programmed to perform one or more of the method actions detailed herein. In some embodiments, there is a system that includes a processor and / or a microchip, or some form of electronic logic circuitry and / or a sensor configured to perform at least some of the method actions detailed herein. The logic circuitry can be part of or be a laptop computer or other type of computing device (desktop and / or server or mainframe) that is programmed or configured to perform at least some of the method actions detailed herein and enables the teachings detailed herein. The computing device can be a smartphone or smart device. It should also be noted that in some embodiments, some functions are in wireless and / or wired communication with such computing devices.

[0094] Other aspects of the invention will become apparent from the following description, given by way of example only, with reference to embodiments illustrated in the accompanying drawings, in which:

[0095] The embodiments will now be described with reference to the accompanying drawings. [Brief description of the drawings]

[0096] [Figure 1] FIG. 1 is an exploded view of a multi-layer sensor module including two light sources of different wavelengths and a single photodetector. [Diagram 2] FIG. 1 is a schematic diagram of an exemplary armband with five optical sensors circumferentially arranged around the armband. [Figure 3A] 2 is a cross-sectional view of the multi-layer sensor module shown in FIG. 1 illustrating one path that light can take from each of the two light sources to the photodetector. [Figure 3B] 2 is a cross-sectional view of the multi-layer sensor module shown in FIG. 1 illustrating one path that light can take from each of the two light sources to the photodetector. [Figure 4] 1 is a series of graphs showing wavelength dependent output from a multi-layer photosensor for three separate finger movements. [Diagram 5] FIG. 1 is a schematic diagram of a grip enhancing glove showing the sheathed cable tendons extending from the differential to the index finger of the glove. [Figure 6] FIG. 1 is a schematic diagram of a grip enhancing glove showing the routing and termination of the cable tendons in the glove. [Figure 7] FIG. 1 is a schematic diagram of a grip enhancing glove showing five cable tendons running through each finger of the glove. [Figure 8] FIG. 1 is a schematic diagram of a grip-enhancing glove being used by a recipient to grasp a cylindrical object. [Figure 9] Figure 9A shows five photographs of each hand gesture, and Figure 9B shows three charts of signal measurements of the hand gestures in Figure 9A. [Figure 10] Two diagrams of accuracy measurements. [Figure 11] 13 is a table showing the accuracy of three gesture decoding models. [Figure 12] 1 is a table of correlations and accuracy for three regression models. [Figure 13] 1 is a line graph of estimated versus actual gripping force. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0097] In one aspect, the present disclosure provides a sensor device. Elsewhere herein, the sensor device may be referred to as an optical sensor, or a sensor module. The sensor device may be used to detect a physical state of a user's tissue, the sensor device being placed at or near the user's skin surface. The sensor device may be used to detect a property of the user's tissue by emitting light and detecting reflected light. More specifically, in some examples, the sensor device may be used to detect changes in a property or properties of the user's tissue over time by detecting changes in the emission and reflected light.

[0098] The sensor device can include multiple light sources (eg, light emitting components), each emitting incident light for reflection by the user's tissue, and a photodetector that detects the reflected light.

[0099] The light emitted by each light source can include one or more different wavelengths, for example, a first light source can emit light at a first wavelength or range of wavelengths and a second light source can emit light at a second wavelength or range of wavelengths different from the first light source.

[0100] The user of the sensor device can be a human, although in other examples the user may not be a human but may be another animal.

[0101] Humans and other animals have body tissues that include a skin surface, beneath which is a dermal tissue layer, then a subcutaneous tissue layer. The dermal tissue layers include, from the skin surface inward, the skin surface, the epidermis layer, and the dermis layer. The subcutaneous tissue layer includes the subcutaneous tissue layer, then the muscle. The skin surface of interest, the associated dermal tissue layer of interest, and the associated subcutaneous tissue layer of interest are collectively referred to herein as a "skin site."

[0102] The properties of the tissue that the sensor device can be used to detect include one or more physical states of the tissue. The tissue can be, for example, a subcutaneous tissue layer. For example, the sensor device can be used to detect a contraction state, or a change in the contraction state, of a muscle or muscle group below the skin surface on which the sensor device is located. The contraction state of the muscle can be associated with a particular optical property, e.g., a reflective property, of the muscle tissue. The reflective property of the muscle tissue can change due to, for example, one or more of a displacement or remodeling of the muscle, a change in the local volume of the muscle in various contraction states, a change in local stiffness, or a change in local density.

[0103] The contraction state of a muscle may be associated with certain optical properties of other non-muscle tissue layers. For example, the contraction state of a muscle may be associated with certain optical properties of one or more of the adjacent subcutaneous tissue layers, dermis layers, or epidermis layers. The contraction state of a muscle may be associated with certain optical properties of the skin surface under which the muscle is located. The optical properties of the tissue layers adjacent to the muscle of interest may be affected by physical changes of the muscle in different contraction states. For example, the muscle may change shape or volume locally in different contraction states. These changes may deform the surrounding tissue layers, and these deformations may result in changes in the optical properties of the tissue layers. For example, contraction of muscle tissue may cause outward pressure on the adjacent subcutaneous tissue, dermis, and epidermis. This may reduce the thickness of one or more of these tissue layers and change their optical properties. Contraction of muscle tissue may also cause deformation of the skin surface, either by stretching the skin surface in-plane or by locally deforming the skin tissue outward, or both. Such changes in the skin surface may cause changes in the optical properties of components of the sensor device that are placed in contact with the skin surface.

[0104] In other examples, the sensor device may additionally or alternatively be usable to detect other properties of tissue, more particularly subcutaneous tissue. For example, the sensor device may be usable to detect the heart rate of a user to whom the sensor is applied. Since changes in blood flow in tissue are associated with the continuous beating of an animal's heart, changes in blood flow may cause changes in the optical properties of the tissue through which the blood flows and / or adjacent tissue layers.

[0105] In other examples, the sensor device may additionally or alternatively be usable to detect other properties of the subcutaneous tissue, such as the presence or concentration of different elements or compounds that change the optical properties of the tissue. For example, the sensor device may be usable to detect blood oxygen level of blood in the subcutaneous tissue due to the optical properties exhibited by the tissue at different oxygen saturation levels.

[0106] The use of a sensor device(s) to identify one or more characteristics of a user's tissue, such as the activity of the user's muscles, in accordance with the present disclosure may be characterized as lightmyography (LMG), as opposed to electromyography (EMG) or forcemyography (FMG) techniques.

[0107] Either the optical properties of the tissue layers or other components between either or both of the light source and the photodetector and the skin surface may be associated with certain characteristics of the reflected light from the incident light of the light source(s) by the different tissue layers. The reflected light for the layers represents the reflection of the incident light by the respective layers. These properties may include the intensity of the reflected light. In some examples, these properties may include the distance from the skin layer of the subject to each of the light source and the photodetector. When the properties of the tissue change, the change in the properties may be associated with a corresponding change in the reflected properties of the light from the incident light of the light source(s) from within the layer of the tissue. In other words, the change in the properties of the tissue results in a corresponding change in the light reflected by (or at) the skin layer of interest and detected by the photodetector.

[0108] By affecting a particular physical state of the user's tissue, for example by contracting a muscle to a particular contraction state while using the sensor device, the associated properties of the light reflected from within the tissue can be obtained. Similarly, by imparting different changes to the properties of the user's tissue, for example by contracting a muscle between two contraction states, the resulting changes in the associated properties of the light reflected from within the tissue can be obtained. Thus, the sensor device can detect the physical state of the tissue below the skin surface based on the properties of the light reflected from within the tissue and received by the sensor device.

[0109] If a sensor device is used to detect the physical state of a particular muscle, the determined contraction state of that muscle may be used to infer the relative position of a body part of a user, for example, the flexed or extended position of a user's forearm and upper arm relative to one another. Similarly, if a sensor device is used to sense the physical state of a particular muscle, changes in the sensed contraction state of that muscle may be used to infer changes in the relative position of different body parts of a user, for example, the flexed or extended position of a user's forearm and upper arm relative to one another.

[0110] Data collected from the sensed reflected light characteristics from the sensor device associated with a particular tissue region, e.g., a particular muscle, can be used to generate a model of the reflected light characteristics associated with a particular physical state or change in physical state of the user's tissue, particularly the subcutaneous tissue. For example, the sensor device can be used to collect data of the reflected light characteristics associated with a selected contraction state of a particular muscle, or a change in the selected contraction state of a particular muscle. The collected data can then be used to train a model of the characteristic sensed characteristics associated with the selected contraction state or change in the selected contraction state of that muscle.

[0111] For example, the data can be used to train a machine learning model. Such a machine learning model can be trained with sensor data associated with a particular physical location or movement of a user. Once trained, the sensor data of reflected light characteristics sensed when a sensor device is applied to a user can be input into the trained machine learning model and classified by the model to obtain an estimate of the current physical location or movement of the user's body part.

[0112] Thus, by training a model of the characteristics of reflected light associated with different contraction states of different muscles, a trained model can be generated that is capable of classifying sensor data from one or more sensor devices and generating estimates of the contraction states or particular changes in contraction states of different individual muscles.

[0113] In some examples, multiple sensor devices are used, each sensing different properties of one or more of different regions of the user's tissue. For example, multiple sensor devices may be used, each determining a contraction state of a corresponding different muscle of the user. A data-based model of the properties of reflected light can be trained based on sensed values ​​from the multiple sensor devices, and the trained model can then be used to estimate a physical position or movement of a body part of the user associated with the operation of the multiple different muscles. For example, multiple sensor devices may be provided to determine the contraction states of different muscles, such as different muscles of the user's forearm. If a trained model of the properties of reflected light associated with the different contraction states of each of those muscles has been developed, the sensed values ​​from each sensor device are input into and classified by the trained model, and from those classifications, an estimate of the contraction state of each muscle can be obtained.

[0114] Since muscle contraction states or changes in muscle contraction states may be associated with movements of the user's body part to which the sensor device is applied, estimation of the physical state of one or more muscles or other subcutaneous tissue regions may estimate a physical position assumed by the user's body part. For example, if a detectable physical state of a muscle(s) is associated with a particular physical position assumed by the user's body part, estimation of these physical states by one or more sensor devices may estimate the user's physical position by inputting the sensed values ​​into a trained model. Similarly, if a detectable change in the physical state of a muscle is associated with a particular movement, e.g., gesture, of the user, estimation of these physical states by the sensor device(s) may estimate a movement or gesture made by the user by inputting the sensed values ​​into a trained model.

[0115] The estimated physical positions of a user's body parts, or the estimated movements or gestures of a user, can be used to control devices such as humanoid robotics, powered exoskeletons, and active prosthetic limbs. For example, a sensor device or multiple sensor devices can be used to estimate the user's movements and manipulate a robot to support or mimic those movements.

[0116] In one tested configuration, multiple sensor devices were placed on an armband and a machine learning model, the lightmyography model, was trained using data from several different hand gestures from a number of different users. The hand gestures included a rest position, a pinching gesture with the index finger and thumb, a tripod gesture, a fist or clenched fist pose, and an extension pose in which the fingers of the hand are fully extended. For comparison, a matching model was also trained using sensor data from EMG. Using the trained model, the lightmyography model was found to be more accurate in classifying hand gestures than was provided by the trained EMG model.

[0117] EMG sensor data may require processing (e.g., RMS) to be used as training data. This may involve computational power and additional time. Compared to the use of EMG data, sensor data from the photodetector of the sensor device(s) of the present disclosure can be advantageously utilized directly as training data in the context of model training without the need for intermediate processing. In the context of application, the sensor data of the photodetector can be provided directly to a trained model without any intermediate processing.

[0118] In the case where the sensor device is used to provide input data to the trained model to receive an estimated physical location or movement of a user to which the sensor device is applied, a processor and a storage medium may be provided. The storage medium may store the trained model, and the processor may receive the sensor data, input the sensor data to the trained model, receive the classification, and estimate the physical location or movement accordingly. The processor may store the output estimate. The processor may, in some examples, communicate the output estimate to a user, for example, via a user interface. The processor may, in some examples, communicate the output to another device, for example, to a server or controller of a robotic device that is operated based on the estimated physical location or movement of the user.

[0119] When the processor is used in combination with the sensor device(s), the processor may be collocated with the sensor device(s). In other examples, the processor is not collocated with the sensor device(s) but may receive sensor data from the sensor device(s).

[0120] When the storage medium is used in combination with the sensor device and the processor, the storage medium may be located together with the sensor device or the processor. In other examples, the storage medium may be located apart from the sensor device or the processor, e.g., remotely, but communicatively connected to the processor.

[0121] A method for estimating a muscle contraction state of a target muscle may utilize one or more sensor devices. According to such a method, sensed values ​​of the first and second reflected light at both a first time point and a second time point may be received. The sensed values ​​may be received, for example, by a processor. Once the sensed values ​​are received, the processor may be used to perform an estimation of a deformation of a skin area adjacent to the user's skin to which the one or more sensor devices are applied. The estimation may be performed based on the sensed value of the first reflected light. More specifically, the estimation may be performed based on a classification of the sensed value of the first reflected light by a trained model. The estimation may be performed using a process or deformation estimation of a subcutaneous area of ​​the user to which the one or more sensor devices are applied. The estimation may be performed based on the sensed value of the second reflected light. More specifically, the estimation may be performed based on a classification of the sensed value of the second reflected light by a trained model. The processor may be used to perform an estimation of a muscle contraction state of the or each muscle adjacent to the one or each optical sensor. The estimation may be performed based on a combination of an estimated skin deformation and an estimated subcutaneous deformation.

[0122] The characteristics of the reflected light sensed by the sensor device may include characteristics of the reflected light originally incident from two different light sources.

[0123] The characteristics of the reflected light sensed by the sensor device may include characteristics of the reflected light at two different wavelengths.

[0124] The characteristics of the reflected light sensed by the sensor device may include characteristics of reflected light from two different light sources, where the reflected light from the different light sources is at different wavelengths.

[0125] The sensor device may include an elastic light-transmitting layer having optical properties that change as a result of deformation of the light-transmitting layer. The elastic light-transmitting layer may be elastically deformed. The light-transmitting layer may be referred to elsewhere herein as an elastomeric layer or an elastomeric pad. The light-transmitting layer may have a first side and a second side and may have a thickness between the two sides. The second side of the light-transmitting layer may be positioned against a skin surface of a user in use.

[0126] The optical properties of the tissue or light transmitting layer that may change with deformation of the tissue or layer may include one or more of the transmittance, absorption, scattering, or reflectance of light incident on or passing through the tissue or layer. The optical properties of the tissue or light transmitting layer may also include the perceived color of the tissue or layer.

[0127] In at least some examples, the sensor device may detect the intensity of light reflected from (or by) different tissue layers.

[0128] The deformation of the light-transmitting layer may include stretching in the plane of the light-transmitting layer, thereby changing the thickness between the first and second sides of the light-transmitting layer. The deformation may additionally or alternatively include compression between the first and second sides of the light-transmitting layer, thereby changing the thickness of the light-transmitting layer. Such a change in the thickness of the light-transmitting layer changes the optical properties of the layer and accordingly affects the light passing through the light-transmitting layer. For example, an increase in the thickness of the light-transmitting layer may be associated with an increase in the attenuation of light passing through the layer, while a decrease in the thickness of the light-transmitting layer may be associated with a decrease in the attenuation of light passing through the layer.

[0129] When the light-transmitting layer is deformed, the change in shape of the light-transmitting layer due to the deformation may additionally or independently change the optical properties of the light-transmitting layer. For example, when the thickness of the light-transmitting layer changes due to the deformation, an increase or decrease in thickness may cause a decrease or increase in intensity, respectively, for a given intensity of light incident from one side of the light-transmitting layer, which may be sensed at the other side of the light-transmitting layer.

[0130] The use of a light-transmitting layer in the sensor device allows one or both of the light source and the light detector to be spaced apart from the user's skin surface during use. This allows incident light from the light source to be reflected from the skin surface and received by the light detector. This spacing arrangement is suitable for shorter wavelengths (e.g., green). When deformation of the light-transmitting layer occurs, the change in the optical properties of the light-transmitting layer can enhance or change the properties of the light reflected from the tissue and passing out of the light-transmitting layer. That is, the change in the optical properties of the light-transmitting layer can cause a corresponding change in the optical properties of the light passing through the light-transmitting layer.

[0131] The light-transmitting layer can be made of an elastic material. For example, the light-transmitting material can include silicone with a Shore hardness of 00-30. Portions of the light-transmitting layer can be dyed to change the effect of ambient light on light detection. For example, the light-transmitting layer can have portions dyed black to reduce the effect of such ambient light.

[0132] The thickness of the light transmitting layer can be selected to provide a desired effect on either or both the incident and reflected light passing through the light transmitting layer. Increasing the thickness of the light transmitting layer may be associated with the transmission of light having longer wavelengths. The thickness of the light transmitting layer can be selected to provide a desired area of ​​reflection of the incident light on the skin surface or other tissue layer.

[0133] In some examples, the light transmissive layer may include regions of different thicknesses, each with a different light source and / or associated light detector.

[0134] As described below, the light transmitting layer can include one or more dopants to change the optical properties of the light transmitting layer. The one or more dopants can also change the way in which the optical properties of the light transmitting layer change as its thickness changes. That is, the light transmitting layer can be doped or otherwise configured to change the relationship between the thickness and the optical properties of the light transmitting layer.

[0135] The first light source of the sensor device may be configured to emit the first incident light through the light transmitting layer toward the skin surface (or the skin site of interest). To this end, the first light source may be disposed on or facing a first side of the light transmitting layer opposite the skin surface of the user during use. In some examples, the first light source may be partially or completely embedded within the light transmitting layer. When the sensor device is configured such that the light transmitting layer or a portion thereof is between the first light source and the skin surface of the user, at least the first incident light from the optical sensor passes through the light transmitting layer.

[0136] The second light source of the sensor device may be configured to emit the second incident light. In some examples, the second light source may be configured to emit the second incident light through a light-transmitting layer. In other examples, the second light source may be configured to emit the second incident light without through a light-transmitting layer. For example, the second light source may be configured to emit the second incident light directly to a skin surface of a user.

[0137] The optical detector of the sensor device can be configured to detect a first reflected light representative of a reflection of the first incident light. The optical detector can be configured to detect a second reflected light representative of a reflection of the second incident light. The first reflected light and the second reflected light may be reflected by different tissue layers of the user. For example, the first reflected light may be reflected by a dermal tissue layer and the second reflected light may be reflected by a subcutaneous tissue layer.

[0138] The light detector may be disposed on or toward a first side of the light transmitting layer, the side facing away from the user's skin surface in use. In some examples, the light detector may be partially or completely embedded within the light transmitting layer. If the light transmitting layer, or a portion thereof, is disposed between the light detector and the user's skin surface, the first reflected light and the second reflected light may pass through the light transmitting layer before being received by the light detector.

[0139] In some examples, the sensor device may include a first photodetector and a second photodetector for detecting the first and second reflected light, respectively. The first photodetector may be disposed at or toward the first side of the light-transmitting layer to detect the first reflected light through the light-transmitting layer, and the second photodetector may be positioned to avoid detecting the second reflected light through the light-transmitting layer. For example, the second photodetector may be disposed on the skin surface either at the second side of the light-transmitting layer or laterally adjacent to the light-transmitting layer to directly detect the second reflected light.

[0140] The wavelength(s) of light emitted by the first light source and the second light source may be different from one another, and because light of different wavelengths may penetrate to different depths within tissue before being reflected, the different wavelengths of light emitted by the first light source and the second light source may enable the sensor device to receive reflected light from different tissue layers.

[0141] Shorter wavelengths penetrate less into tissue and increase the intensity of the reflected light, resulting in a more sensitive response and allowing for more accurate sensing of physical changes in tissue. Longer wavelengths penetrate deeper into tissue and reflect less light, but can sense physical changes in deeper layers of tissue.

[0142] In some examples, the sensor device may be configured such that the first reflected light is reflected at or adjacent to the skin surface of the user. For example, the wavelength of the light emitted by the first optical sensor may be such that at least a portion of the first incident light is reflected by the skin surface as the first reflected light. In such examples, the wavelength of the light emitted by the first optical sensor may correspond to green light having a wavelength of about 500 nm to about 565 nm.

[0143] In another example, the skin surface may be provided with a reflective element that reflects the first incident light. The reflective element may act to increase the reflection of the incident light thereon. In such an example, the wavelength of light emitted by the first light sensor may be any wavelength or wavelengths that are reflected by the reflective surface. The reflective surface may be provided, for example, on a second side (the side facing the skin) of the light-transmitting layer. The second side of the light-transmitting layer may be provided with a reflective coating. The reflective coating may be a flexible reflective coating. The reflective element may be disposed in the path of the incident light to allow the incident light to be transmitted through the skin surface while improving the intensity of the resulting reflected light. The reflective element in one configuration is suitable for use with incident light of a shorter wavelength (e.g., green) than incident light of a longer wavelength (e.g., infrared). In examples using incident light of different wavelengths, each reflective element portion is appropriately associated with incident light of an appropriate and compatible wavelength. The reflective surface may be used to advantageously reduce the adverse effects of factors such as the reflectivity, roughness, and color of human skin. With a shinier surface that is more reflective, a higher signal response or SNR can be achieved, making it easier to distinguish between different muscle movements.

[0144] In some examples, the sensor device may be configured such that the second reflected light is reflected at or within the subcutaneous tissue layer. In such examples, the wavelength of the light emitted by the second optical sensor may have a wavelength of about 625 nm to about 1,400 nm.

[0145] When the sensor device has a reflective surface on the skin surface, the first photodetector may be configured to receive the first reflected light from the reflective surface, and the second photodetector may be configured to receive the second reflected light from the subcutaneous tissue layer. The second photodetector may be provided such that the second reflected light received by the second photodetector passes through the light-transmitting layer. In another example, the second photodetector may be provided such that the second reflected light received by the second photodetector does not pass through the light-transmitting layer. For example, the second photodetector may be provided directly facing the user's skin surface.

[0146] When the sensor device includes a first light source and a second light source, the light source may be configured to alternately emit the first incident light and the second incident light, respectively. The alternating emissions may be configured to emit non-simultaneously with respect to one another. Such a configuration facilitates differentiation between the first reflected light and the second reflected light at the photodetector(s) of the sensor device.

[0147] When the sensor device has a first light source disposed at a first distance from the user's skin surface and a second light source disposed at a different second distance from the user's skin, the sensor device emits light from a plurality of different distances from the skin surface. Such a sensor device may be called a multi-layer sensor device or a multi-layer optical sensor. In some examples, the first distance is a distance equal to the thickness of the light-transmitting layer, the second distance is zero, and the second light source emits light at the skin surface.

[0148] The light source of the sensor device may be provided by any light emitting device. In at least some examples, the light source of the sensor device is provided as a light emitting diode (LED).

[0149] In some examples, the sensor device may be configured to block extraneous ambient light from being received by the photodetector(s). For example, a housing or other light-impermeable element may be provided around the light source, the photodetector(s), and the light-transmitting layer to block ambient light when the sensor device is placed in contact with the skin surface of a user. Such a light-blocking arrangement blocks unintentional detection of ambient light by the photodetector.

[0150] The technology disclosed herein, e.g., sensor devices and combinations thereof, is generally applicable for sensing movement, deformation, and / or displacement of soft tissue adjacent to a skin surface. Some exemplary applications of the technology include wearable human-machine interfaces, physiological sensors (such as heart rate monitors), and / or soft tissue pressure sensors. Specific examples of the technology include human-machine interfaces. In these examples, the disclosure relates to systems and methods for sensing muscle activation beneath the skin. The muscle activation can be used to control various devices, such as computer systems, bionic / prosthetic grippers, and / or lower limb orthotics. However, the technology is readily adaptable to other soft tissue sensing and / or measurement applications.

[0151] In at least some examples, a wearable band, such as an arm band or leg band, is used to hold multiple multi-layered optical sensors or sensor devices next to the user's skin. The optical sensors can detect muscle movement from deformation of the skin proximate to the band. In at least some examples, the optical sensor comprises at least two light sources, such as LEDs, that project light onto the skin, and at least one photodetector positioned to receive light from the light sources. The sensor output can be used to control devices such as humanoid robotics, powered exoskeletons, and active prosthetic limbs.

[0152] For example, a band may be worn by a user around the forearm to provide data for interpreting grip type (e.g., fist grip / pinching grip / cylindrical grip) and / or gestures for an active prosthetic or exoskeleton glove. In at least some examples, three or more optical sensors or sensor devices are circumferentially positioned around the band to detect forearm muscle movements that affect finger and / or wrist movements.

[0153] In at least some examples, the optical sensor is comprised of an elastomer layer, such as silicone, disposed between one of the light sources and the skin. The thickness of the elastomer can be from about 1 mm to about 20 mm. For example, the thickness of the elastomer layer can be from 3 mm to 12 mm. In other examples, other light-transmissive elastic layers may be used.

[0154] The intermediate elastomer layer forms a layered structure of light sources within the sensor. For example, a light source held immediately adjacent to or in contact with the skin is offset from a light source behind the elastomer layer by approximately the thickness of the elastomer layer. The elastomer layer can direct and / or guide the light between the light source and the skin. In at least some examples, the light sources in the different layers emit different wavelengths of light. For example, the light source adjacent to the skin can be configured to emit light with a longer wavelength than the light source that is offset from the skin. In some opposite examples, the light source adjacent to the skin can emit light having a wavelength shorter than about 500 nm, while the light source separated from the skin by the elastomer layer can emit light having a wavelength longer than about 650 nm.

[0155] The elastomeric layer can include an elastomeric pad interposed between one or more light sources and the skin to adjust the distance between the light source and the skin, manipulate the transmittance of the gap between the light source and the skin, and / or control other attributes that can be used to tune the response of the sensor.

[0156] For example, the elastomeric pad can incorporate a dopant that changes the absorptivity of the material in response to deformation. In some examples, the elastomeric pad can have one or more sections that reflect, scatter, or absorb at least a portion of the light emitted from the light source.

[0157] For example, the elastomeric pad can have a reflective surface configured to cover a portion (e.g., a third, a quarter, half) of the skin surface such that a corresponding portion of the light emitted from the light source is reflected by the reflective surface and a remaining portion of the light emitted from the light source is reflected by the epidermis layer of the skin.

[0158] In some examples, the mechanical properties of the elastomeric pad can be used to tune the response of the sensor, for example, by tuning the deformation characteristics of the elastomeric pad to produce nonlinear transmittance and / or absorptance characteristics, thereby controlling the dynamic range and / or resolution of the sensor.

[0159] At least one photodetector is positioned to receive light reflected from the user's skin, in some examples a single photodetector may be positioned to receive light from two or more light sources.

[0160] For example, a single photodetector may be offset from the skin and positioned laterally between adjacent light sources. In at least some examples, an elastomeric layer is interposed between the photodetector and the skin surface. The photodetector may be positioned on the same layer as one of the light sources or on a layer intermediate the light sources. For example, the photodetector may be offset from the skin by an intermediate elastomeric layer such that the photodetector is closer to the skin than one light source and farther from the skin than another light source.

[0161] In at least some examples, the sensor is configured to interleave light pulses from two or more light sources that share a common light detector. For example, the sensor may interleave the periods during which each light source is emitting light, such that the light detector does not receive light from multiple light sources simultaneously. The sensor may be configured to measure (or detect) the intensity of light incident on the light detector simultaneously with each light pulse. In some examples, the light captured by the light detector in successive periods is reflected from different types of soft tissue and / or different layers of the skin. The sensor may be configured to alternate between measurements from the surface of the skin and measurements from below the surface of the skin to create a composite light measurement. In some examples, each light source has a dedicated light detector, and measurements from different types of soft tissue and / or layers of the skin may be taken simultaneously.

[0162] In at least some examples, the sensor is configured to detect movement and / or deformation in different layers, sections, and / or depths of the soft tissue. For example, the light sources can be configured to have different degrees of penetration of the light they emit into the skin. In at least some examples, red or infrared LEDs are held adjacent to the surface of the skin, such that light emitted from the LEDs and subsequently captured by the photodetector is reflected by tissues below the skin surface, such as subcutaneous fat and / or muscle. Light with longer wavelengths can achieve deeper skin penetration and provide supplemental data that can help decode (e.g., estimate) the user's muscle movements. Longer wavelength light can also be used to obtain health information, which is a potential advantage of light-based myography over traditional EMG techniques. By using shorter wavelength LEDs (such as green, blue, violet, or ultraviolet LEDs), more superficial measurements (e.g., from the epidermal layer of the skin) can be obtained. Shorter wavelength light penetrates the skin less, resulting in higher reflectance and a more sensitive response. In at least some instances, differential tissue measurements can be obtained and / or enhanced by the physical placement of the light source relative to the skin surface, for example, the gap between the light source and the skin surface can be adjusted to modulate the depth of light penetration.

[0163] In some examples, the sensor can be used to estimate subcutaneous tissue movement from a combined light measurement. The combined light measurement can include light at a first wavelength reflected from a surface of the skin and light at a second wavelength reflected from tissue below the surface of the skin, such as muscle, connective tissue, and / or subcutaneous fat. For example, the sensor can be configured to obtain a deformation estimate of an epidermal layer of the skin from an intensity of light captured at the first wavelength and a deformation estimate of a dermal layer of the skin from an intensity of light captured at the second wavelength. The combined deformation estimate of the epidermal and dermal layers of the skin can serve as a surrogate for muscle and / or connective tissue movement in at least some applications. For example, an estimate of muscle contraction can be calculated from the intensity of light at the first wavelength and the intensity of light at the second wavelength from the combined light measurement. In at least some examples, an artificial limb, an exoskeleton glove, and / or a robotic gripper can be actuated or otherwise controlled in response to the calculated muscle contraction estimate.

[0164] FIG. 1 illustrates an exploded view of an exemplary multi-layer sensor module 200. The sensor module 200 has two LED light sources 202, 203 and a single photodetector 205. The photodetector 205 may be provided in the form of a photodiode. In the illustrated embodiment, the photodiode is co-located with the first LED light source 202. The LED light sources 202, 203 and the photodetector 205 are secured within the housing 201 when the sensor module is assembled. In the illustrated embodiment, the housing 201 has an elastomeric layer covering the skin-facing surface of the sensor module. The thickness of the elastomeric layer varies across the skin-facing surface of the housing 201. For example, the thickness of the elastomeric layer adjacent the first LED light source 202 is about 4 mm to about 8 mm, whereas the thickness of the elastomeric layer adjacent the second LED light source is typically negligible (e.g., less than about 1 mm thick). In some examples, the elastomeric layer does not extend into the area adjacent the second LED light source 203. The backing plate 204 securely holds the LED light source within the housing 201. In at least some examples, the backing plate 204 biases the LED light source toward the elastomeric layer, thereby creating a tight interference fit between the LED light source and the elastomeric layer and / or eliminating any noticeable gaps between the LED light source and the elastomeric layer. In some examples, the backing plate 204 can be elastomeric or can include an elastomeric portion.

[0165] 2 shows an armband 300 including five sensor modules 101, 103, 105, 107, 109. In the illustrated embodiment, the sensor modules 101, 103, 105, 107, 109 are evenly distributed around the circumference of the armband 300. Compliant straps 102, 104, 106, 108, 110 extend between adjacent sensor modules 101, 103, 105, 107, 109 and hold the sensor modules 101, 103, 105, 107, 109 together within the band. The lengths of the compliant straps 102, 104, 106, 108, 110 define the distribution of the sensor modules 101, 103, 105, 107, 109 around the circumference of the band and the spacing between the sensors. A lateral end of each sensor module 101, 103, 105, 107, 109 is secured to a corresponding strap 102, 104, 106, 108, 110 using a pin.

[0166] The illustrated armband 300 is configured to be worn near the middle of a user's forearm. In at least some examples, the circumference of the armband in an unstrained state is configured to be less than the circumference of the user's forearm, such that tension in the armband maintains the elastomeric layers of the housings of each of the sensor modules 101, 103, 105, 107, and 109 in contact with the user's skin when the armband is worn. In the illustrated embodiment, the compliant straps 102, 104, 106, 108, 110 stretch, increasing the inner circumference of the armband 300 to accommodate the user's forearm. This creates an axial strain in the straps 102, 104, 106, 108, 110, which causes the armband 300 to fit tightly around the user's forearm and maintains the sensor modules 101, 103, 105, 107, 109 in contact with the skin. The elastomer layer on the skin surface of the sensor modules 101, 103, 105, 107, 109 may be configured to distribute forces and / or reduce localized pressure concentrations applied to the skin by the armband 300. The elastomer layer on the skin surface may be compressed by the pressure of the elastomer layer against the user's arm.

[0167] The action of the compliant straps 102, 104, 106, 108, and 110 being stretched to fit against the user's body can function to provide biasing of the sensor modules 101, 103, 105, 107, and 109 against the user's skin surface.

[0168] The number, size, distribution, and / or configuration of sensor modules can be tailored for various applications. For example, sensors with a larger surface area facing the skin, more light sources, and / or more photodetectors can be used for lower extremity applications. Similarly, sensor modules can be concentrated near specific muscles rather than being evenly distributed around a limb or other body part. For example, a heart rate monitor can include one or more sensor modules clustered near the sternum.

[0169] A cross-sectional schematic diagram of a two-layer sensor module 400 disposed on a user's soft tissue is shown in Figures 3A and 3B. As shown in Figures 3A and 3B, the soft tissue beneath the sensor module includes a skin surface 11, an epidermis layer 12, a dermis layer 13, a subcutaneous layer 14, and a muscle layer 15. The sensor module 400 includes two light sources 411 and 412, each disposed on opposite sides of a photodiode 420. A red or infrared LED light source 412 is disposed next to (i.e., adjacent to) the skin to the right of the photodiode 420. A green LED light source 411 is disposed to the left of the photodiode 420. A silicon layer 430 is disposed between the green LED light source 411 and the photodiode 420 and the skin surface 11. In this example, the silicon layer 430 is shown disposed on the skin surface 11, and the photodiode 420 and the light source 411 are shown disposed on the silicon layer 430. Thus, the photodiode 420 is located on the same layer as the green LED light source 411, and the photodiode 420 and the light source 411 have the same spacing from the skin surface 11. The lateral spacing between each LED light source 411 and 412 and the photodiode 420 is determined, at least in part, by the respective paths traversed by the light emitted from each light source 411 and 412 to reach the photodiode 420, with the light path of the light source 411 traversing the silicon layer 430. In some examples, the lateral spacing between each LED light source and the photodiode can be determined entirely from the expected light path. In other examples, the light sources can be evenly spaced around the photodiode.

[0170] FIG. 3A shows a sensor module 400 emitting light from a green LED light source 411 and a red or infrared LED light source 412 that is inactive. Emitted light having a wavelength in the green spectrum (having wavelengths between approximately 500 nm and 565 nm) is shown passing through a silicon layer 430, penetrating the surface 11 of the skin, and being reflected from (or by) the epidermal layer 12. In some examples, the surface 11 of the skin can reflect a portion of the incident light. The reflected light passes through the silicon layer 430 and is detected by a photodiode 420. The intensity of the green light captured by the photodiode 420 can depend, at least in part, on the distance the light travels. For example, movement of the underlying muscle and / or connective tissue can cause the silicon layer 430 to compress, shortening the optical path between the green LED light source 411 and the skin surface 11. The optical properties of the skin surface adjacent to the sensor module 400 can also change due to movement of the underlying muscle and / or connective tissue. For example, tissue density, mix, and shape may change with deformation, altering the intensity of light captured by photodiode 420. Thus, light source 411 may be seen to be configured to emit light toward the skin (e.g., skin site) through silicon layer 430 for reflection by epidermal layer 12 (or epidermal skin portion) and, in some examples, skin surface 11. Further, in this example, silicon layer 430 is adapted to be placed on skin surface 11 and configured to space light source 411 typically 5 mm from skin surface 11. This 5 mm spacing increases the amount of reflection of light emitted from light source 411 that is detected by photodiode 420, thereby improving the intensity of the detected light.

[0171] In FIG. 3B, the sensor module 400 is shown emitting light from the red or infrared LED light source 412 while the green LED light source 411 is inactive. Emitted light having a wavelength in the red or infrared spectrum (approximately 625 nm to 1400 nm, or longer) is shown passing through the surface 11 of the skin and being reflected from (or by) a subcutaneous fat layer, such as the subcutaneous tissue layer 14. In at least some examples, the light in the red or infrared spectrum may be reflected from the dermis layer 13 of tissue, the subcutaneous fat (also known as the subcutaneous tissue layer, such as the subcutaneous tissue layer 14), muscle 15, and / or connective tissue. The reflected light passes through overlapping layers of tissue and the silicon layer 430 to reach the photodiode 420. The intensity of the red or infrared light captured by the photodiode 420 may depend, at least in part, on the distance the light travels. For example, movement of the underlying muscle and / or connective tissue may cause the silicon layer 430 to compress, shortening the light path between the skin and the photodiode 420. The optical properties of the soft tissue below the skin surface may also change due to movement of the underlying muscle and / or connective tissue. For example, the density, mix, and shape of the tissue may change due to deformation, altering the intensity of the light captured by the photodiode. In this example, the light source 412 is placed in contact with the skin to achieve deeper light penetration. It will be appreciated that the light source 412 is configured to emit light towards the skin to be reflected by the subcutaneous tissue layer 14 (or non-epidermal skin portion), and in some other examples, by other layers 13 and 15. The infrared light of the light source 412 may interact with other layers, such as the dermis, as it passes through those layers, and the detected reflection of the infrared light may indicate movement occurring in those other layers.

[0172] Thus, it can be seen that silicon layer 430 is elastic, optically transmissive, and has optical properties that change in response to deformation of silicon layer 430. It can also be seen that silicon layer 430 is elastically deformable, causing a corresponding change in the optical properties of light traversing silicon layer 430. For example, silicon layer 430 can have optical properties that change in response to elastic deformation of silicon layer 430, causing a corresponding change in the optical properties of light traversing silicon layer 430.

[0173] In some examples where the photodiode 420 is placed in contact with the skin surface 11 , the reflected light is detected directly by the photodiode 420 without traversing the silicon layer 430 .

[0174] A series of exemplary graphs showing tested outputs from the sensor module of FIG. 3A and FIG. 3B for three different types of gestures are shown in FIG. 4. The top graph represents the intensity of infrared light captured by the photodiode. The bottom graph represents the intensity of green light captured by the photodiode. The sensor module was held against the skin of a user's forearm while the user moved the corresponding finger of the hand. The left graph represents the photodiode output when the ring finger and the little finger were flexed simultaneously. The center graph represents the photodiode output when only the little finger was flexed. The right graph represents the photodiode output when only the ring finger was flexed. From the graphs in FIG. 4, it is clear that the two LED light sources 411 and 412 of the sensor module 400 generate complementary information indicative of the movement and / or deformation of the soft tissue of the forearm.

[0175] In at least some examples, soft tissues at different layers and / or depths below the skin surface respond differently to muscle and / or connective tissue movement and / or deformation. For example, subcutaneous fat may move and / or reorganize below the skin surface as underlying muscles contract. These changes are not always apparent from the skin surface. In at least some examples, combined measurements collected from light reflected off different layers of soft tissue may produce a more accurate and / or robust estimate of muscle and / or connective tissue movement and / or deformation than a single measurement. For example, combining measurements taken from the skin surface and / or epidermal layer with measurements taken at the dermal and / or subcutaneous layer may produce a more accurate and / or robust estimate than a single measurement from either location. In at least some examples, independent component analysis is used to correlate the intensity of light captured by the photodiode at different wavelengths with the user's intent. For example, independent component analysis can be used to correlate the outputs of sensor modules 101, 103, 105, 107, and 109 shown in FIG. 2 with hand gestures and / or grip types.

[0176] A method for estimating motion of subcutaneous tissue includes obtaining a first deformation measurement representative of deformation of an epidermal layer of the skin, obtaining a second deformation measurement representative of deformation of a dermal layer of the skin adjacent the epidermal layer represented by the first deformation measurement, and combining the first deformation measurement and the second deformation measurement to generate an estimate of motion of the subcutaneous tissue.

[0177] In some examples, the method can include estimating a first deformation measurement from the intensity of light reflected from the skin at a first wavelength and estimating a second deformation measurement from the intensity of light reflected from the skin at a second wavelength. For example, the method can include interleaving light from a first light source emitting light at a first wavelength with light from a second light source emitting light at a second wavelength and measuring the intensity of the light from the first light source and the intensity of the light from the second light source with a single photodetector.

[0178] In at least some examples, the composite light measurements from the different soft tissue layers can be obtained by a sensor device including a first light source disposed on or adjacent to the recipient's skin and configured to direct incident light at a first wavelength through at least an upper layer of the recipient's skin and a second light source configured to direct incident light at a second wavelength onto the recipient's skin proximate to the first light source. A compliant pad made of a transparent or translucent elastomer can be configured to be disposed between the second light source and the recipient's skin. Also, at least one light detector can be configured to receive light reflected from the user's skin and / or subcutaneous tissue and measure the intensity of the light at the first wavelength and the intensity of the light at the second wavelength.

[0179] The sensor may include a band that holds the first light source, the second light source, the compliant pad (either a silicone layer or a light-transmitting layer) and at least one light detector in close proximity to the skin. The band may be configured to preload (e.g., bias) the compliant pad against the recipient's skin by applying pressure to the recipient's skin. That is, the band or the like may be configured to apply a biasing force to the compliant pad against the skin. Such an arrangement may prevent the formation of gaps between the compliant pad and the skin, which may, in some cases, adversely affect reflection detection by the sensor device, for example, during arm movement. In some examples, the band is configured to encircle the recipient's limb. The compliant pad may be configured to direct light from the second light source to the recipient's skin. The compliant pad may also be configured to absorb and / or scatter and / or reflect a variable amount of light at a second wavelength when the pad is compressed, the amount of light that the compliant pad absorbs and / or scatters and / or reflects depending on the deformation of the elastomer. In some examples, the wavelength of the second light source is greater than 650 nm and the wavelength of the first light source is less than 550 nm.

[0180] In at least some examples, one or more sensors can be combined with a strap, band, and / or belt configured to hold the sensor next to the skin (e.g., adjacent to and / or in contact with the skin). For example, a device including a strap with a first optical sensor disposed adjacent to an inner surface of the strap and a second optical sensor circumferentially spaced from the first optical sensor can be used to obtain composite optical measurements from different layers of soft tissue. In at least some examples, the second optical sensor is offset outwardly from the inner surface of the strap compared to the first optical sensor, and the strap includes a compressible elastomeric pad disposed between the second optical sensor and the inner surface of the strap. In some examples, the second optical sensor is offset outwardly from the inner surface of the strap by more than about 2 mm, and the elastomeric pad occupies a space between the second light source and the inner surface of the strap. The first optical sensor can include a first light source having a first wavelength and a first optical detector, and the second optical sensor can include a second light source having a second wavelength and a second optical detector. The strap can be configured to apply a radial compressive force to the recipient's limb such that the first optical sensor is in contact with or immediately adjacent to the recipient's skin.

[0181] In some examples, a belt including a first optical sensor configured to detect displacement of the skin below the band and a second optical sensor configured to detect displacement of the soft tissue below the skin surface can be used to obtain composite optical measurements from different layers of the soft tissue. The first optical sensor is held above the surface of the skin and can operate at a wavelength between about 625 nm and about 1 mm. The second optical sensor is held adjacent to the skin and can operate at a wavelength between about 10 nm and about 565 nm.

[0182] In some examples, a band including at least two optical sensors configured to detect displacement of soft tissue proximate to the band can be used to obtain composite light measurements from different layers of the soft tissue. The at least two optical sensors can operate in different frequency ranges of the optical spectrum, and the band can be configured to position the at least two optical sensors at different distances from the skin surface. In some examples, the band includes a compliant elastomeric material disposed between a first optical sensor of the at least two optical sensors and an inner circumference of the band, the compliant elastomeric material configured to contact the skin and deform in response to movement of the soft tissue immediately beneath the contacted skin. The compliant elastomeric material can include a dopant, and the dopant can be configured to modify a transmission rate of light from the first optical sensor of the at least two optical sensors through the compliant elastomeric material in response to deformation of the compliant elastomeric material. In at least some examples, the first sensor of the at least two sensors can be configured to measure deformation of the compliant elastomeric material as a proxy for displacement of the soft tissue. The second sensor of the at least two sensors can be positioned closer to the skin than the first sensor of the at least two sensors. The second sensor of the at least two sensors can be configured to measure displacement of soft tissue beneath the skin. In some examples, the first optical sensor of the at least two optical sensors includes a red or infrared LED light source and the second optical sensor of the at least two optical sensors includes a green, blue, violet, or ultraviolet LED light source.

[0183] The optical sensors or sensor devices of the present disclosure can function as an interface that allows a user to intuitively control and / or interact with machines and / or environments. For example, estimates of soft tissue movement and / or deformation can allow a user to interact with a virtual or augmented reality environment and / or control a robot. An exoskeleton glove is shown as an example herein. In this example, the output from the sensor is converted into a control signal that is used to control the electric motors (e.g., motor speed and / or torque) of the exoskeleton glove. The motors actuate the fingers of the glove to initiate various types and / or strengths of grip. The control platform of the exoskeleton glove can be easily transferred to other physical systems such as prosthetics and / or humanoid robotics, as well as artificial environments such as AR / VR.

[0184] FIG. 5 illustrates a schematic of an exemplary grip enhancement system 500. The system includes an actuated glove 150 that can be worn by a recipient to enhance grip. Torque is transmitted from an electric motor 140 to the fingers 100 of the glove 150 by a network of artificial tendons. FIG. 5 illustrates a single artificial tendon 120. The illustrated tendon includes a sheathed cable 121 that extends from an actuator, e.g., electric motor 140, to the tip of the index finger 100a. The distal end of the cable 121 terminates at the tip of the index finger 100a in a finger cap 155. The finger cap 155 secures the tendon 120 to the glove and distributes forces from the tendon to the recipient's index finger 100a. The illustrated finger cap 155 also includes a sensor 160. The sensor 160 can be an optical sensor or a sensor device, as described herein. Output from the sensor 160 is fed back to a controller that controls the operation of the glove. For example, the controller may use the sensors 160 for touch detection, grip adjustment, and / or performance tracking (eg, monitoring the distribution of force to each finger 100).

[0185] The proximal end of cable 121 is wrapped around the drum of pulley 125. Pulley 125 is driven by electric motor 140 to tension cable 121. Cable 121 transmits force from electric motor 140 to finger caps 155 of glove 150 as cable 121 is gradually retracted and wrapped around the drum of pulley 125. Tension in cable 121 causes the recipient's fingers to contract and fold inward toward the palm of the glove, enhancing the recipient's natural grip. Sheath 122 extends from electric motor 140 to the base of glove 150. Glove 150 and motor housing (not shown) have ferrules that position and secure sheath 122. Sheath 122 is sufficiently compression resistant to keep the length of the cable path between electric motor 140 and glove 150 approximately constant (e.g., prevent contraction of the cable path between electric motor 140 and glove 150). In some examples, the sheath may incorporate a low-friction coating that reduces the sliding friction experienced by the cable.

[0186] FIG. 6 illustrates an exemplary glove cable guide 124. The cable guide 124 constrains the cable 121 to a defined path within the glove 150. The illustrated cable guide 124 includes a stitched portion that extends along the inside of the index finger 100a. Another stitched portion (not shown in FIG. 6) extends from the proximal end of the glove (e.g., adjacent the wrist or forearm) to the base of the palm. In the embodiment illustrated in FIG. 6, the cable 121 is unconstrained across the entire palm of the glove 150. In other examples, the stitching can extend unimpeded between the base of the glove and the finger cap 155, or in separate sections of different lengths / configurations. The cable guide 124 can incorporate a flexible liner that reduces cable friction within the glove 150. For example, a PTFE-coated elastomeric tube can be used to route the cable 121 through the material of the glove 150 without restricting the recipient's mobility / flexibility. 6, the liner can be sewn into the fabric of the glove 150 at the index finger 100a and extend unconstrained across the palm. The glove 150 can also incorporate a rigid or semi-rigid (e.g., thermoset plastic) palm guide to prevent or reduce cable chafing due to pressure (caused by clamping forces from some forms of grips).

[0187] In the example terminating at the fingertip shown in FIGS. 5 and 6, the index finger 100a is flexed. In some examples, flexion can be replaced or supplemented with other forms of anatomical motion. For example, an actuator such as an electric motor 140 can adduce the thumb by tensioning a cable 121 that terminates at the base of the thumb. The glove 150 can be reinforced at the base of the thumb (e.g., with a thermoplastic insert and / or a reinforcing loop around the metacarpophalangeal joint of the thumb) to secure the cable 121 and transfer force to the thumb and / or induce adduction motion of the thumb. In some examples, the glove 150 can be configured to support multiple forms of anatomical motion. For example, in some forms of grip, thumb adduction can be used in combination with thumb flexion. Independent adduction and flexion can be achieved with separate cables 121 that terminate at the base and tip of the thumb, respectively.

[0188] A grip-enhancing glove 150 with five artificial tendons 120a, 120b, 120c, 120d, and 120e is shown diagrammatically in FIG. 7. The tendons 120 extend from a sheath 122 at the base of the glove 150 and flare outward to each of the five fingers 100. The fingers 100 of the glove have cable guides 124 that route the tendons 120 to a termination point (e.g., finger cap 155) of each finger 100. The tendons 120 are actuated by one or more motors that apply and release tension as needed to achieve a proper grip. The sheath 122 routes the tendon cables 121 from the motors to the base of the glove 150. For prosthetic hands and grip-enhancing gloves, the motors are typically housed in a wearable module carried by the recipient. For example, the motor module can be carried in a backpack, hung from a belt around the waist, or attached to the recipient's arm with an armband. The portable mode of a wearable system is typically influenced by the weight and form factor of the motor module. In some examples, the motor module also houses the control electronics. The control electronics adjust the output of the motor to regulate the gripping force applied by the glove 150. For example, the control electronics can incorporate one or more sensors (e.g., EMG and / or force sensors) that infer the recipient's force regulation and / or grip initiation intent.

[0189] FIG. 8 shows a grip-enhancing glove 150 being used by a recipient to grasp an object 170. In the illustrated embodiment, artificial tendons 120 apply force (e.g., via finger caps 155 of corresponding fingers) to at least three of the fingers 100a, 100b, 100c, 100d, and 100e. A combination of natural flexion and tendon tension causes the fingers 100 of the glove 150 to bend inward toward the recipient's palm, generating the cylindrical grip shown in FIG. 8. The force between the fingers and the palm represents the grip force. In some examples (e.g., rehabilitation), the glove 150 can be configured to replicate the grip force of a healthy adult. In other examples (e.g., certain tasks in the workplace), the glove 150 can be configured to generate forces that exceed natural human grip forces.

[0190] The distribution of forces to the fingers 100 of the glove 150 can affect the effectiveness of the grasp the recipient forms. The stability of the grip is closely related to the contact area formed with the object. For gripping applications with well-defined constraints (e.g., robotic grippers for repetitive tasks), the distribution of forces to the fingers can be optimized for a particular grip type (e.g., cylindrical, spherical, or pinch grip). Adaptive grippers that conform to the shape of the object can be used in a variety of applications and are particularly useful when gripping irregularly shaped objects. Some examples of adaptive grips can be influenced by distributing the forces to each actuated finger in a way that does not impede the freedom of movement of each actuated finger (i.e., actuating each finger independently to conform to the surface of the object). For underactuated systems, this involves splitting the output from the actuators to multiple fingers when one of the fingers is constrained (e.g., when one of the fingers is in contact with the surface of the object and motion is stopped), without impeding the freedom of movement of the fingers.

[0191] Figure 9A shows five example hand gestures used in experiments to demonstrate the accuracy achievable with an armband implemented in a configuration similar to that of Figure 2, incorporating five sensor modules. Shown in this figure are a rest gesture 910, a pinch gesture 920, a tripod gesture 930, a fist (clenched fist) gesture 940, and an extension gesture 950.

[0192] FIG. 9B shows three charts 960, 970, and 980 of signal measurements. The first chart 960 shows signal measurements taken for five gestures 910-950 on an armband worn by a subject. The vertical axis represents a scale of normalized activity values ​​indicating light intensity, and the horizontal axis represents time in seconds. For each gesture 910-950, the subject begins with 15 seconds of rest, alternating between 15 seconds of rest and 15 seconds of performing the gesture. The first chart 960 shows five measurements, each in a different shade of grey, representing the five gestures.

[0193] A second chart 970 shows raw EMG activation signal measurements acquired simultaneously using conventional bipolar EMG techniques (g.tec bio-amplifier USBamp). A third chart 980 shows a root mean square (RMS) representation of the measurements in the second chart 970.

[0194] The data acquired in relation to FIG. 9B is processed using various processes. In one process, features are extracted using a sliding window of 200 ms and a stride of 20 ms. A sample size (or period) of more than 125 ms and less than 300 ms reduces bias and variation due to the real-time constraints of typical prosthetic control systems. In another process, the data is conditioned so that the same number of samples are used for each gesture, thereby reducing bias towards certain classes. It is noted that unlike EMG technology, the data acquired with the armband can be directly processed without any filtering operations. The data acquired with the EMG technology is filtered using a Butterworth bandpass configured in the range of 5 Hz to 500 Hz. In addition to the RMS operation, the processing of the raw EMG data in the second chart 1020 also includes waveform length, zero crossing, mean absolute value, integrated EMG, Willison amplitude (WAMP), variance of the EMG signal, and log detector value. Using a deep learning model with a batch normalization layer increases the training speed and eliminates the need for normalization in the pre-processing step.

[0195] We develop three machine learning classification models and use them to compare the performance of the armband with that of traditional EMG techniques, namely, Random Forest (RF), Convolutional Neural Network (CNN), and Temporal Multi-Channel Vision Transformer (TMC-Vit). The RF model is an ensemble classification method based on the combination of multiple decision trees. In the RF model, the output is the most popular class among the decisions of the individual trees. The CNN model contains three convolutional blocks, four fully connected layers, and a final softmax layer to predict hand gestures, and each convolutional block is composed of a convolutional layer, a batch normalization layer, and a dropout layer. The TMC-ViT model is a Transformer-based model that adapts the Vision Transformer to handle temporal data with multiple channels as input, such as LMG signals, by using convolutional and max pooling layers to reduce the input dimensionality and extract its embedding. In TMC-ViT mode, two convolutional layers are used before the data is fed into the ViT, which extracts 2 × 2 patches and provides the output to a Transformer encoder consisting of four multi-head attention layers41 with four heads each.

[0196] Models acting as classifiers are trained and validated using 5-fold cross-validation with sparse categorical cross-entropy as the loss function, with one test per fold repeated in isolation. Trained models are optimized using Adam and evaluated based on accuracy. Each model is trained and optimized for each gesture and each subject.

[0197] In another experiment, the gripping force is estimated using muscle activity measurements obtained using the same armband worn on the forearm. The same regression model is used and the same model configuration is employed. In this experiment, each subject wearing the armband is instructed to perform a first grip with maximum force and a second grip with half the maximum force. Each grip lasts for 15 seconds, with a 15 second rest period before the next grip. In this experiment, a sliding window of 200 milliseconds and a stride of 20 milliseconds are employed to adjust the data obtained using the armband. In other words, in this experiment, the same regression model is trained and tested using only data obtained during the period when force was detected by the sensor device. However, in this experiment, the model has a dense layer with one neuron and a linear activation function serving as the last layer, and the model is trained and validated using 10-fold cross-validation with one test per fold repeated in isolation.

[0198] During model training, we use the mean squared error (MSE) loss function. The efficiency of the trained regression model is evaluated using the Pearson correlation coefficient. The comparison of the accuracy of the actual and estimated forces is expressed as a normalized mean squared error (NMSE) percentage. An NMSE value of 0% indicates a poor fit, while an NMSE value of 100% indicates that the two trajectories are identical. NMSE can be calculated as follows:

[0199]

number

[0200] FIG. 10 shows, on the left, a first radar chart 1010 showing the decoding accuracy of each regression model in percentage for the armband, and on the right, a second radar chart 1020 showing the decoding accuracy of each regression model in percentage for the conventional EMG technique. The first radar chart 1010 shows first, second, and third lines 1011, 1012, and 1013 corresponding to the TMC-ViT, CNN, and RF models, respectively. The second radar chart 1020 shows first, second, and third lines 1021, 1022, and 1023 corresponding to the TMC-ViT, CNN, and RF models, respectively. As can be seen from the figure, the TMC-ViT model achieves the highest accuracy for all subjects, followed by the CNN model, and then the RF model. Based on the comparison of the model results, the armband implementation achieves a higher and more consistent signal performance than that achieved with the conventional EMG technique. According to the TMC-ViT results, the armband can achieve a classification accuracy of up to 99.11%.

[0201] FIG. 11 shows a table of the classification accuracy of each model for the armband and conventional EMG techniques, respectively.

[0202] It can be seen that by using measurements acquired with the armband implementation, better decoding accuracy can be achieved with a relatively small standard deviation for each regression model. Furthermore, measurements acquired using the armband are directly provided to the model, which is more efficient in terms of time and processing resources. In contrast, in traditional EMG techniques, the acquired measurements are processed through additional steps before being provided to the model. This technical advantage is particularly important for real-time applications, where minimizing sample processing time is of utmost importance. Furthermore, the size and weight of the bio-amplifier makes feature-extracted EMG a less suitable solution for portable applications. In contrast, the armband implementation is advantageous in terms of simpler components, smaller size, lighter weight, and lower cost.

[0203] FIG. 12 shows a table of correlation and accuracy for each model in the context of the gripping force experiment, where the columns marked with "C" represent correlation and the columns marked with "A" represent accuracy. The TMC-ViT results show the highest correlation and accuracy as well as the lowest standard deviation, followed by the CNN and RF results. From the regression results, it can be seen that the gripping force can be directly decoded from the measurements acquired by the armband without processing the raw data. By using the proposed armband, the achievable accuracy and correlation of the force estimation can reach 92% and 96%, respectively. In FIG. 13, a first line graph 1310 shows a first line 1311 representing the decoded force for one subject and a second line 1312 representing the true force (actual force), and a second line graph 1320 shows a first line 1321 representing the decoded force for another subject and a second line 1322 representing the true force (actual force).

[0204] In summary, the RF, CNN, and TMC-ViT models show that measurements taken using the armband can achieve improved average accuracy of 96.64%, 97.18%, and 97.86%, respectively, in estimating the wearer's intent. Furthermore, these measurements can also be used to achieve an average accuracy of 86.05%, with a high correlation of 93.55%, in estimating the grip force. The sensor device and the armband incorporating it are advantageous in terms of component complexity, size, weight, portability, and cost.

[0205] Where the foregoing description refers to elements or integers that have known equivalents, such equivalents are intended to be included as if individually set forth.

[0206] Although the embodiments have been described with reference to a number of exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Accordingly, the preferred embodiments are to be considered in an illustrative sense only, and not for the purposes of limitation, and the scope of the invention is not limited to the embodiments. Moreover, the present invention is defined by the appended claims, not the detailed description of the invention, and all differences within the scope thereof are to be construed as being included in the present disclosure.

[0207] Many modifications will be apparent to those skilled in the art without departing from the scope of the invention as herein described with reference to the accompanying drawings.

Claims

1. A sensor device for detecting movement of subcutaneous tissue from a user's skin surface, comprising: an elastic light-transmitting layer having optical properties that change in response to deformation of the light-transmitting layer; a first light source configured to emit a first incident light through the light-transmitting layer toward the skin surface; a second light source configured to emit a second incident light toward the skin surface; a photodetector configured to detect first reflected light representative of reflection of the first incident light by a skin tissue layer or at or near the skin surface, and second reflected light representative of reflection of the second incident light by a subcutaneous tissue layer; A sensor device comprising:

2. The sensor device according to claim 1 , wherein the photodetector is provided on the light-transmitting layer and detects the first reflected light and the second reflected light through the light-transmitting layer.

3. 3. The sensor device of claim 2, wherein at least a portion of a side of the light-transmitting layer configured to contact the skin surface of the user is reflective and configured to reflect light of the first incident light.

4. The sensor device of claim 3 , wherein the first light source is disposed on the light-transmitting layer and emits the first incident light through the light-transmitting layer.

5. 5. The sensor device of claim 4, wherein the light-transmitting layer has a thickness of approximately 5 mm between the side of the light-transmitting layer configured to contact the skin surface of the user and the opposing side on which the first light source is provided, thereby increasing the intensity of the detected first reflected light.

6. The sensor device of claim 2 , wherein the second light source is provided on the skin surface of the user and configured to emit the second incident light directly through the skin surface of the user.

7. 10. The sensor device of claim 1, wherein the light-transmitting layer, the first light source, the second light source, and the light detector define a sensor module, and wherein the sensor device includes a plurality of sensor modules, each sensor module being used to detect movement of subcutaneous tissue associated with a different muscle group of the user.

8. The sensor device of claim 1 , wherein the first incident light has a first wavelength and the second incident light has a second wavelength, the first wavelength being shorter than the second wavelength.

9. 9. The sensor device of claim 8, wherein the first wavelength is in the range of about 500 nm to about 565 nm, and the second wavelength is in the range of about 625 nm to about 1,400 nm.

10. The sensor device of claim 1 , wherein the light-transmitting layer is configured to elastically deform, and the deformation of the light-transmitting layer affects one of a path and intensity of light traversing the light-transmitting layer.

11. The sensor device of claim 1 , wherein the first and second light sources are configured to non-simultaneously emit the first and second incident light beams, respectively, toward the skin surface of the user.

12. Furthermore, a band configured to apply a biasing force to the light-transmitting layer against the skin surface of the user; a processor configured to estimate a movement of subcutaneous tissue based on the detected first reflected light and second reflected light; The sensor device of claim 1 , comprising at least one of:

13. 13. The sensor device of claim 12, wherein the processor is configured to provide values ​​of the detected first reflected light and second reflected light as inputs to a model and to determine a gesture-state of the user's body part from an output of the model.

14. The sensor device of claim 1 , wherein the first reflected light represents a reflection of the first incident light by a skin tissue layer of tissue, the skin layer being an epidermis layer.

15. A method for estimating a muscle contraction state of a target muscle using the sensor device according to any one of claims 1 to 14, comprising: receiving, using a processor, sensed values ​​of each of the first reflected light and the second reflected light at both a first time point and a second time point; using the processor to estimate a deformation of a skin area adjacent to the skin of the user based on the change in the sensed value of the first reflected light; using the processor to estimate deformation of a subcutaneous area adjacent the skin area of ​​the user based on changes in the sensed value of the second reflected light; using the processor to estimate a muscle contraction state of the target muscle based on the estimated skin deformation and the estimated subcutaneous deformation; A method comprising:

16. 16. The method of claim 15, wherein the steps of claim 15 are repeated at different times to provide multiple temporal estimates of the muscle contraction state of the target muscle, and wherein the multiple temporal estimates of the muscle contraction state are used to infer a gestural movement of the user's body part associated with the target muscle.

17. A sensor device, a light-transmitting layer that is elastically deformable to cause a corresponding change in the optical properties of light traversing the light-transmitting layer; a first light emitting component configured to emit light through the light transmissive layer toward the skin site for reflection by a corresponding epidermal skin portion; a second light emitting component configured to emit light toward the skin site for reflection by a corresponding non-epidermal skin portion; a photodetector configured to detect light reflected by the epidermal and non-epidermal skin portions; A sensor device comprising:

18. The sensor device of claim 17 , wherein the non-epidermal skin portion is a subcutaneous tissue portion.

19. 18. The sensor device of claim 17, wherein the light-transmitting layer is adapted to be placed on the skin site and configured to position the first light-emitting component 5 mm from the skin site, thereby increasing the intensity of light reflected by the epidermal skin portion and detected by the photodetector.

20. The sensor device according to any one of claims 17 to 19, wherein the photodetector is configured to detect the reflected light through the light-transmitting layer.

Images