Augmented reality prosthetic training tool for amputees

Abstract

A system for training an individual in the use of a prosthetic Limb. A sleeve is positionable on a residual limb. The sleeve includes at least one sensor, receiving signals representative of activation of at least one muscle within the residual Limb, and at least one haptic feedback transducer. An augmented reality headset is provided with at least one of visual image presentation and audio presentation. A control processor communicates with the sleeve and the headset. The control processor receives at least one signal from the at least one sensor in the sleeve representing activity of muscles within the residual limb.

Description

TITLE OF THE INVENTIONAugmented Reality Prosthetic Training Tool for AmputeesCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit, under 35 USC §119(e)(1), of United States provisional application 63 / 647,593, filed 14 May 2024, expressly incorporated in its entirety herein by reference.TECHNICAL FIELD

[0002] The technical field of this disclosure encompasses apparatus, system and methods for training and rehabilitation of amputees vis-a-vis the acclimation to, and use of, prosthetic limbs.BACKGROUND OF THE INVENTION

[0003] As of the time of this writing, each day, approximately 400 Lower Limb amputations are performed in the United States alone. It is estimated that over 1.7 million US residents have undergone Lower Limb amputation. Such individuals face predominantly two particular complications: muscle memory Loss and the phenomenon referred to as “phantom limb” pain.

[0004] It has been observed in clinical studies that post-amputation, an individual’s motor neurons in the affected limb undergo extensive remapping. This remapping can increase the difficulties encountered by the individual in Learning howto control the use of a prosthetic Limb going forward. Such difficulties may be lessened, however, the sooner after the amputation procedure that the affected individual is fitted with, and undergoes training in the use of, a prosthetic limb.

[0005] Similarly, it has been observed that as much as 80 percent of lower limb amputees may experience some level of phantom limb pain. It is believed that prompt fitting of, and training in the use of, prosthetic Limbs may lessen the occurrence of, and / or degree of, such phantom Limb pain.

[0006] While the benefits of early fitting of and training with prosthetic limbs can improve the patient’s outcome and lessen the severity and / or occurrence of such adverse complications, there is often a considerable gap in time between the amputation procedure and the actual delivery of a suitable prosthetic Limb, such that the patient may begin training and treatment. Such a gap may be anywhere from 5 months to a year or more.

[0007] It would be desirable, therefore, to provide a device, system, and / or method of treatment that may be provided and implemented, in this gap between the amputationprocedure and the delivery of a suitable prosthetic limb, that will address and alleviate the adverse complications of muscle memory loss and phantom limb pain.SUMMARY OFTHE INVENTION

[0008] In an embodiment of the invention, a system is provided for training an individual in the use of a prosthetic limb. The system comprises...a sleeve positionable on a residual limb of the individual, the sleeve including at least one sensorthat receives signals representative of activation of at Least one muscle within the residual Limb, the sleeve further including at least one haptic feedback transducer...an augmented reality headset with at least one of visual image presentation and audio presentation...a control processor coupled in communication with the sleeve and the headset...the control processor receiving at least one signal from the at Least one sensor in the sleeve representing activity of muscles within the residual Limb...the control processor further generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual...the control processor further generating signals to at least one of the at Least one haptic feedback transducer in the sleeve and the augmented reality headset for generation of at least one of a feedback physical stimulus and a feedback auditory stimulus, in response to an attempted activation of muscles within the residual limb corresponding to an intended movement of a virtual prosthetic in the augmented reality image presented to the individual.

[0009] In an embodiment of the invention, the system further comprises that the at least one sensor is an electromyography sensor.

[0010] In an embodiment of the invention, the system further comprises that the at least one haptic feedback transducer generates, in response to receiving signals from the control processor, at Least one of a vibratory output, an audible output.

[0011] In an embodiment of the invention, the sleeve of the system comprises...a rectangular structure, including at least one layer of material, and having a nominal inner surface and a nominal outer surface...the at least one sensor disposed on the nominal inner surface of the rectangular structure such that when the rectangular structure is wrapped around the residual Limb of the individual, the at least one sensor is juxtaposed to and in physical contact with, a skin surface of the residual limb...a closure mechanism coupled to the rectangular structure, such that when the rectangular structure is wrapped around the residual limb of the individual, the closure mechanism releasably secures the rectangular structure to the residual Limb.

[0012] In an embodiment of the invention, the closure mechanism of the system comprises complementary hook-and-loop fastener components affixed to the nominal inner and outer surfaces of the rectangular structure at opposite ends thereof.

[0013] In an embodiment of the invention, the control processor of the system comprises...at least one pre-amplifier coupled to, and receiving signals from, the at least one sensor...at least one operational amplifier coupled to, and receiving signals from, the at least one pre-amplifier...an analog-to-digital converter (ADC) coupled to, and receiving signals from, the at Least one operational amplifier...and a central processing unit.

[0014] In an embodiment of the invention, the control processor of the system further comprises...an array of darlington bipolar transistors coupled in communication between the central processing unit and the at Least one haptic feedback transducer.

[0015] An embodiment of the invention comprises a method fortraining an individual in the use of a prosthetic Limb. The method comprises the steps of...positioning a sleeve positionable on a residual Limb of the individual, the sleeve including at least one sensorthat receives signals representative of activation of at least one muscle within the residual Limb, the sleeve further including at Least one haptic feedback transducer...providing an augmented reality headset...providing a control processor coupled in communication with the sleeve and the headset...the control processor receiving at least one signal from the at least one sensor in the sleeve representing activity of muscles within the residual limb...the control processor further generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual...the control processor further generating signals to at least one of the at least one haptic feedback transducer in the sleeve and the augmented reality headset for generation of at least one of a feedback physical stimulus and a feedback auditory stimulus, in response to an attempted activation of muscles within the residual limb corresponding to an intended movement of a virtual prosthetic in the augmented reality image presented to the individual.

[0016] In an embodiment of the method, the at Least one sensor is an electromyography sensor.

[0017] In an embodiment of the method, the at Least one haptic feedback transducer generates, in response to receiving signals from the control processor, at least one of a vibratory output, an audible output.

[0018] In an embodiment of the method, the sleeve comprises...a rectangular structure, including at least one layer of material, and having a nominal inner surface and a nominal outer surface...the at least one sensor disposed on the nominal inner surface of the rectangular structure such that when the rectangular structure is wrapped around the residual limb of the individual, the at least one sensor is juxtaposed to and in physicalcontact with, a skin surface of the residual limb...a closure mechanism coupled to the rectangular structure, such that when the rectangular structure is wrapped around the residual limb of the individual, the closure mechanism releasably secures the rectangular structure to the residual Limb.

[0019] In an embodiment of the method, the closure mechanism comprises complementary hook-and-loop fastener components affixed to the nominal inner and outer surfaces of the rectangular structure at opposite ends thereof.

[0020] In an embodiment of the method, the control processor comprises...at least one pre-amplifier coupled to, and receiving signals from, the at least one sensor...at least one operational amplifier coupled to, and receiving signals from, the at Least one preamplifier...an analog-to-digital converter (ADC) coupled to, and receiving signals from, the at Least one operational amplifier...and a central processing unit.

[0021] In an embodiment of the method, the control processor further comprises...an array of darlington bipolar transistors coupled in communication between the central processing unit and the at Least one haptic feedback transducer.

[0022] An embodiment of the invention further comprises a further method for training an individual in the use of a prosthetic limb, comprising the steps of...providing a sleeve positionable on a residual Limb of the individual, the sleeve including at Least one sensor that receives signals representative of activation of at Least one muscle within the residual limb, the sleeve further including at least one haptic feedback transducer...fitting the sleeve to the residual Limb of the individual to ensure effective transmission of information from a skin surface of the residual Limb of the individual to the at least one sensor...fitting an augmented reality headset to the individual...coupling the at least one sensor, the at least one haptic feedback transducer and the augmented reality headset in communication with a control processor...generating signals with the control processor corresponding to an augmented reality image...displaying the augmented reality image in the augmented reality headset, wherein the augmented reality image includes an image of a virtual prosthesis positioned at the end of an image of the residual Limb...causing the individual to attempt to activate muscles in the residual limb in an effort to perform a function requiring the movement of a prosthesis... monitoring muscular activity of muscles within the residual Limb of the individual... causing movement within the augmented reality image of the virtual prosthesis...providing at least one feedback stimulus to the individual.

[0023] In an embodiment of the method, the step of providing at least one feedback stimulus to the individual further comprises generating a signal by the control processor and transmitting same to the at least one haptic feedback transducer.

[0024] In an embodiment of the method, the at Least one haptic feedback transducer produces, in response to the signal generated by the control processor, at least one of a vibratory output, an audible output.

[0025] In an embodiment of the invention, wherein the method further comprises the step of generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual Limb of the individual, the method further comprises varying a characteristic of the at least one feedback stimulus depending upon a type of action being attempted by the individual.

[0026] In an embodiment of the invention, wherein the method further comprises the step of generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual Limb of the individual, the method further comprises the step of processing signals received from the at least one sensor, employing impedance control, using a mass-spring-damper analogy, to emulate joint behavior to determine a degree of bending of a joint to produce an augmented reality image of a virtual prosthetic movement in response to the attempted activation of muscles within the residual limb of the individual.BRIEF DESCRIPTION OFTHE DRAWINGS

[0027] Fig. 1 is a partially schematic illustration of the components comprising the system of the present invention.

[0028] Fig. 2 is an enlarged, inner-facing side plan view of the sensory input / output sleeve illustrated in Fig. 1.

[0029] Fig. 3 is a schematic illustration of the control circuitry of the EMG sensors according to an embodiment of the invention.

[0030] Fig. 4 is a schematic illustration of the haptic feedback circuitry according to an embodiment of the invention.

[0031] Fig. 5 is a flo chart of a method accordingto an embodiment of the invention.

[0032] Fig. 6 illustrates a representative scenario of a patient operating the system of the present invention, and in particular demonstrating a representative image presented to the patient.

[0033] Fig. 7 is a flow chart illustrating the conceptual basis of the signal processing performed on signals received from sensors in the sleeve.

[0034] Fig. 8 illustrates the conceptual mechanical underpinnings behind the relationship between the movements of a Limb, signals produced by sensors on the limb, and the conversion of same into signals informing the creation of a virtual image of a prosthesis.

[0035] Fig. 9 illustrates a fundamental equation applicable to the principles embodied in Fig. 8.DETAILED DESCRIPTION

[0036] While this invention is susceptible of embodiment in many different forms, there are shown in the drawings and described in detail herein, specific embodiments, with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to Limit the invention to the embodiment(s) illustrated.

[0037] The invention and accompanying drawings will now be discussed in reference to the numerals provided therein so as to enable one skilled in the art to practice the present invention. The drawings and descriptions are exemplary of various aspects of the invention and are not intended to narrow the scope of the appended claims. Unless specifically noted, it is intended that the words and phrases in the specification and the claims be given their plain, ordinary, and accustomed meaning to those of ordinary skill in the applicable arts. It is noted that the inventors can be their own lexicographers. The inventors expressly elect, as their own Lexicographers, to use only the plain and ordinary meaning of terms in the specification and claims unless they clearly state otherwise and then further, expressly set forth the "special" definition of that term and explain how it differs from the plain and ordinary meaning. Absent such clear statements of intent to apply a "special" definition, it is the inventors' intent and desire that the simple, plain and ordinary meaning to the terms be applied to the interpretation of the specification and claims.

[0038] The inventors are also aware of the normal precepts of English grammar. Thus, if a noun, term, or phrase is intended to be further characterized, specified, or narrowed in some way, then such noun, term, or phrase will expressly include additional adjectives, descriptive terms, or other modifiers in accordance with the normal precepts of English grammar. Absent the use of such adjectives, descriptive terms, or modifiers, it is the intent that such nouns, terms, or phrases be given their plain, and ordinary English meaning to those skilled in the applicable arts as set forth above.

[0039] Further, the inventors are fully informed of the standards and application of the special provisions of 35 U.S.C. § 112(f) or pre-AIA 35 U.S.C. § 112 ~ 6. Thus, the use of the words "function," "means" or "step" in the Detailed Description of the Invention or claims is not intended to somehow indicate a desire to invoke the special provisions of 35 U.S.C. § 112(f) or pre-AIA 35 U.S.C. § 112 ~ 6 to define the invention. To the contrary, if the provisions of 35 U.S.C. § 112(f) or pre-AIA 35 U.S.C. § 112 ~ 6 are sought to be invoked to define the inventions, the claims will specifically and expressly state the exact phrases "means for" or "step for" and the specific function (e.g., "means for roasting"), without also reciting in such phrases any structure, material or act in support of the function. Thus, even when the claims recite a "means for ... "or "step for ... "if the claims also recite any structure, material or actsin support of that means or step, or that perform the recited function, then it is the clear intention of the inventor not to invoke the provisions of 35 U.S.C. § 112(f) or pre-AIA 35 U.S.C. § 112 ~ 6. Moreover, even if the provisions of 35 U.S.C. § 112(f) or pre-AIA 35 U.S.C. § 112 ~ 6 are invoked to define the claimed inventions, it is intended that the inventions not be limited only to the specific structure, material or acts that are described in the illustrated embodiments, but in addition, include any and all structures, materials or acts that perform the claimed function as described in alternative embodiments or forms of the invention, or that are well known present or Later-developed, equivalent structures, material or acts for performing the claimed function.

[0040] In the following description, and for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various aspects of the invention. It will be understood, however, by those skilled in the relevant arts, that the present invention may be practiced without these specific details. In other instances, known structures and apparatus are shown or discussed more generally in order to avoid obscuring the invention. In many cases, a description of the operation is sufficient to enable one to implement the various forms of the invention, particularly when the operation is to be implemented in software. It should be noted that there are many different and alternative configurations, apparatus and technologies to which the disclosed inventions may be applied. Thus, the full scope of the inventions is not Limited to the examples that are described below.

[0041] Various aspects of the present invention may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of hardware or software components configured to perform the specified functions and achieve the various results.

[0042] Implementation of the present invention may require the use of several computer architectures, programming protocols, languages, databases and steps. A computer-implemented device equipped with machine-readable memory, a central processing unit, and several software capabilities. Essential components would include a server to host the application, allowing web-based access, and a database to manage the citations and claims data. The user interface would be graphical, designed for seamless user interaction, with client-side scripting in languages like JavaScript to manage dynamic elements such as the 'Convert to Positive' or 'Finalize Report' functions. On the server side, a combination of databases, runtime environments and programming languages, for example Node.js, JavaScript, Python or C# could be employed alone or in combination to manage intricate data processing tasks, ensuring that user inputs accurately correspond with the appropriate claims and references. Data could be stored and retrieved from thedatabase using a language like SQL, with secure network protocols in place to ensure the integrity and confidentiality of the data. The overall architecture would support scalability and real-time data processing, ensuring a responsive and efficient user experience.THE SYSTEM

[0043] Electromyography (EMG) is a technique for evaluating and recording the electrical activity produced by skeletal muscles. EMG is performed using an instrument called an electro myograph to produce a record called an electromyogram. An electromyograph detects the electric potential generated by muscle cells when these cells are electrically or neurologically activated. The signals can be analyzed to detect abnormalities, activation level, or recruitment order, or to analyze the biomechanics of human or animal movement. Needle EMG is an electrodiagnostic medicine technique commonly used by neurologists. Surface EMG (the approach employed in the instant invention), is a non-medical procedure used to assess muscle activation by several professionals, including physiotherapists, kinesiologists and biomedical engineers. In computer science, EMG is also used as middleware in gesture recognition towards allowing the input of physical action to a computer as a form of human-computer interaction.

[0044] In an embodiment of the system, illustrated in part in Fig. 1, the invention comprises system 10, which, in turn, comprises a “Virtual Reality” or “Augmented Reality” (hereinafter “AR”) headset 12, a sleeve 14 having embedded on an inner surface thereof (when in place on a wearer’s limb (not shown)) a plurality of electromyography (EMG) sensors 16, and a control processor 18. In an embodiment of the invention, sensors 16 are coupled in electrical communication (indicated by arrow A) with control processor 18 via a plurality of hardwire electrical connections 20. Likewise, in an embodiment of the invention, headset 12 may be electronically coupled in one-way communication (indicated by arrow B) with control processor 18 via hardwire electrical connection 22. Sleeve 14 comprises body 28, which, in a preferred embodiment is fabricated from two or more Layers of silicone and / or polyurethane (materials commonly used in prosthetic Liners), although any material or combination of materials appropriate for extended periods of direct contact with a patient’s skin, particularly areas which may be sensitized to direct contact, may be employed.

[0045] In an embodiment of the invention, headset 12 may be an off-the-shelf component, e.g., the Meta Quest 3 sold by Meta Corporation, model number S3A. Likewise, EMG sensors 16 may be off-the-shelf components, e.g., the product sold commercially as “EMG amplifier with Flying Wires,” sold by Biometrics, Ltd., with product code SX230FW. In alternative embodiments of the invention, either or both of these components may be substituted for by similar commercially available or purpose-build components having similar performance characteristics, without departing from the scope of the invention.

[0046] While in the illustrated embodiment of Fig. 1 , hardwire connections are employed to convey signals between control processor 18, and headset 12 and sleeve 14, respectively, in an alternative embodiment of the invention, communications between control processor 18, and headset 12 and sleeve 14 may be carried out wirelessly, e.g., via known Bluetooth technology. In addition, a controller from the headset (e.g., the Meta Quest 3 headset controller, not shown, but a known component having known functionalities) is attached to sleeve 14 during use of the system. The headset controller is used to track gross movements of the patient’s residual Limb, such that upon generation of the augmented / virtual image by control processor 18, the headset controller causes the virtual limb to move in concert with the headset controller, to provide the illusion that the virtual prosthesis is attached to the residual Limb. In an alternative embodiment of the invention, instead of employing the headset controller, inertial measurement units may be embedded within sleeve 14 to provide the same information to the system as the headset controller.

[0047] In the embodiment shown in Fig. 1 , for simplicity, only three EMG sensors 16 are illustrated. In alternative embodiments (discussed below), as few as two sensors 16 may be employed, or as many as desired or necessary to meet the requirements of a particular implementation. For example, in an embodiment of the invention (e.g., Fig. 2), eight (8) or more sensors 16 may be employed, with each sensor being suitably positioned and tuned, to detect, receive and transmit information relating to a selected muscle group, such that, for example, eight sensors 16 will detect, receive and transmit signals for eight separate muscle groups or portions of muscle groups. In an embodiment of the invention, sensors 16 are situated on sleeve 14 so as to acquire signals from the quadriceps and hamstring muscles, though in alternative embodiments, other muscles / muscle groups may be implicated.

[0048] Fig. 2 illustrates sleeve 14, comprising body 28, the inside-facing (i.e., patientfacing) surface of which, in particular, showing EMG sensors 16 (only a representative number of which are called out specifically by reference numeral), coupled to electrical connectors 20 (similarly only a representative number of which are called out specifically by reference numeral). In the embodiment of Fig. 2, EMG sensors 16 are provided in eight pairs, representing eight muscle groups. The placement of sensors 16 shown is merely representative, and may be varied in other embodiments by one having skill in the art without departing from the scope of the invention.

[0049] In an embodiment of the invention, sleeve 14 is removably affixed to the residual limb of a patient, e.g., via mating hook-and-loop fastener components 24, 26, located on opposing sides of and opposite ends of, body 28, to couple overlapping portionsof body 28. In an alternative embodiment of the invention (not shown), a strap-and-buckle arrangement may be employed, as may other means of affixation.

[0050] Sleeve 14 is also provided with a plurality of haptic transducers 30 (Fig. 2) which, in an embodiment of the invention, may be disposed within body 28, e.g., between juxtaposed layers of material from which body 28 may be fabricated. In an embodiment of the invention, haptic transducers 30 are coupled in communication with control processor 18 via hardwire connections 22 (only some of which are illustrated and called out by number in Fig. 2. It is to be noted that number and Location of any of the sensors 16 and / or transducers 30 as shown and described herein with respect to any of the described embodiments is merely byway of example and for purposes of illustrating the invention, and that the number and placement of the sensors 16 and / or transducers 30 may be varied by one having ordinary skill in the art, as desired or according to the needs of a particular implementation, without departing from the scope of the invention.

[0051] As described herein, any components (including but not Limited to headset 12, sleeve 14, sensors 16, control processor 18, and / or transducers 30), which are described as having hardwire communications connections, such as connections 20, 22, may, in alternative embodiments, be connected via wireless means, such as via Bluetooth communications protocols.

[0052] In a simplified version of the invention, sleeve 14 may be omitted, and sensors 16, haptic feedback transducers 30, and a holder for the controller for headset 12, may instead be placed directly on a patient’s residual limb via gel affixation.

[0053] Fig. 3 is a schematic illustration of the control processor 18, in which, merely by way of example, the signals from two sets of sensors 16 are illustrated, indicated by two signal paths (upper and lower) on the left of Fig. 3. In an embodiment of the invention, as described above, sensors 16 are positioned in proximity to the skin on the residual limb (“RL”) of the patient, coupled, e.g., via wiring 20, to control processor 18. Control processor 18 includes EMG signal amplifiers 40 (effectively pre-amplifiers, which provide an incremental gain in the signals), which receive the signals from sensors 16, and which, in turn, pass the amplified signals into one or more operational amplifier(s) 42 (which provides, in the illustrated embodiment, a further 2.5V of DC voltage (or 2.5V DC offset) to the amplified EMG signal through a non-inverting summing op-amp 44 to shift the signal into analog-to-digital converter 44’s input range). Op-amp(s) 42 likewise may, in an embodiment, comprise and off-the-shelf component, such as the op-amp sold commercially under the name “Microchip Technology Operational Amplifier, product reference MCP6002I / P.” The numerical values of the components of op-amp 42 may be selected and / or varied as necessary to meet the requirements of any particular implementation by one skilled in theart. EMG signal amplifiers 40 may typically be included in the EMG hardware supplied by the vendor. The signals from operational-amplifier 42 are in turn transmitted to analog-to-digital converter 44, which digitizes the signals for processing by central processing unit (“CPU”) 46. In an embodiment of the invention, CPU 46 may be a single-board computer, such as those sold under the commercial name “Raspberry Pi” may be employed. However, any commercially-available processor of similar capabilities may be employed without departing from the scope of the invention.

[0054] Fig. 4 is a schematic illustration of the haptic feedback circuitry according to an embodiment of the invention. In an embodiment, CPU 46 is connected via intermediary components, to eight (8) haptic feedback transducers 30. In an embodiment of the invention, transducers 30 provide a vibratory output upon excitation. The signals from CPU 46 to transducers 30 are passed through an array 48 of darlington bipolar transistors which significantly increase the output voltages supplied to transducers 30 with a relatively low required input power level.THE NEURAL NETWORK ALGORITHM

[0055] Control processor 18 may be an off-the shelf programmable computer. While processor 18 is symbolically illustrated as a Laptop, it is to be understood that much simpler devices, such as a single-board computer, such as that sold under the name Raspberry Pi, may be preferred. Alternatively, control processor 18 may be a purpose-built processor. Suitable programming may be provided using known programming techniques. In an embodiment of the invention, processor 18 is provided with appropriate programming and / or hardware to filter out noise in the signals transmitted from sensors 16. The provided software then maps the intended movements of the patient, as represented by the signals from the targeted muscle groups during a simulated activity, onto the virtual prosthesis in the augmented reality presented to the patient in headset 12, giving the patient the perception of controlling an actual human limb, thus simulating movement once a real prosthetic is attached to their residual limb. This simulated proprioception provides an enhanced immersive experience during physical therapy.

[0056] In an embodiment of the invention, the signals (and corresponding data) acquired by the sensors are converted by control processor 18 into a two-dimensional (2D) virtual model of the prosthesis. In an alternative embodiment of the invention, potentially employing a greater number of sensors 16 and / or a control processor 18 having enhanced processing capability and memory, using known computational techniques and hardware, can generate a three-dimensional (3D) representation of the prosthesis.

[0057] In an alternative embodiment of the invention, software will be provided, using known programming techniques, to provide the system with machine-learning (or reinforcement learning) capabilities

[0058] Fig. 7 is a flow chart illustrating the conceptual basis of the signal processing performed on signals received from sensors in the sleeve. In particular, Fig. 7 illustrates the signal processing cycle beginning with the raw signals from sensors 16 and ending with a final signal which determines a real-time output joint angle for the generated virtual image of the prosthetic. It is to be recognized, of course, that “real-time” is a relative term; there is, in the real world, a delay. In an embodiment of the invention, the window of time is preferably on the order of 100 milliseconds or less, though that may vary with the equipment, requirements, etc., of a particular implementation, without departing from the scope of the invention. Further, while embodiments in the instant disclosure focus on a determination of an angle of bending of the knee, it is to be understood that the invention may be implemented by one skilled in the art to address other joints in the leg and / or arm, as needed, without departing from the scope of the present invention.

[0059] Fig. 8 illustrates the conceptual mechanical underpinnings behind the relationship between the movements of a limb, signals produced by sensors on the limb, and the conversion of same into signals informing the creation of a virtual image of a prosthesis. In particular, the upper portion of Fig. 8 illustrates how a mass-spring-damper system may be used to conceptualize how joints behave under applied forces. The lower portion of Fig. 8 illustrates how human joints may be modeled after a mass-spring-damper system.

[0060] Fig. 9 illustrates a fundamental equation applicable to the principles embodied in Fig. 8. In particular, impedance laws may be applied in the overall control system, including control processor 18, as well as the extant software in headset 12 and hand-held headset controller 70, to emulate normative joint behavior.THE METHOD

[0061] Fig. 5 illustrates an exemplary method 50 for implementation of an embodiment of the present invention. Method 50 begins with the identification of a suitable candidate patient having a lower limb amputation. An appropriately sized sleeve, such as sleeve 14 described with respect to Fig. 1 is provided (step 52). The sleeve 14 is then fitted to the patient (step 54). This step further includes a calibration step to match the responses of the sleeve 14 and control processor 18 to the individual patient, as each amputee has a unique location of amputation, and thus a unique situation regarding the placement and functionality of the muscle groups within the residual limb. In an embodiment of theinvention, a “hand-held” headset controller (not shown) is provided to the patient, to enable the patient to transmit commands to control processor 18 pertaining to the prosthesis exercise / training program(s) within control processor 18. Such commands may include “START”, “STOP”, “RESET,” or similar instructions, as well as potentially commands to modify or select the particular exercise to be performed, the speed or difficulty thereof, etc. Giving the patient agency to control the training exercise via their own headset controller may further enhance patient confidence in the system and improve the speed and efficiency of the training. An appropriate exercise / training program is initiated via control processor 18 (Fig. 1), and an AR headset 12 (Fig. 1) is applied to the client. Once wearing AR headset 12, the patient will see the image of a virtual prosthetic applied to the residual limb (step 56).

[0062] In a preferred embodiment of the invention, the patient is able to see everything around them as if they were not wearing headset 12, but now also have a virtual prosthetic leg (in an embodiment, rendered as solid black) that they can see attached to the residual limb. The virtual prosthetic leg looks like a traditional prosthetic in solid black; however, in an embodiment of the invention, user-selectable variations in appearance may be enabled, which the user may select based on their preference; e.g., the rendered virtual prosthetic could have the appearance of a human leg, a prosthetic with artwork or logos (provided that any images protected by intellectual property are properly licensed or otherwise appropriately addressed), etc.

[0063] The patient will then attempt a movement of the virtual prosthetic (step 58), which causes the targeted muscle groups in the residual limb to move or otherwise be activated. At step 50, sensors 16 in sleeve 14 will detect these movements or other activations of the muscles in the patient’s residual limb, which are transmitted to control processor 18. Programming in control processor 18 will make a determination (step 60) of the “intended” movement of the virtual prosthetic and then create and present to the patient via headset 12 a virtual image of the movement of the prosthetic limb as determined by control processor 18. Further, control processor 18 will generate and transmit a haptic response signal (step 62) to haptic feedback transducers 30 in sleeve 14 to provide a stimulation in the form of vibration felt by the patient.

[0064] In an embodiment of the invention, even more than just receiving feedback when the user activates a muscle in the residual limb, the LOCATION and INTENSITY of the haptic feedback may be varied to correspond to the type of movement and what part of the virtual leg interacted with a virtual object (ex: a soccer ball). For example, if the toe of the virtual leg hit a soccer ball vs. the shin of the virtual leg, the haptic feedback would be sent to different ones of transducers 30, correspondin to different locations on the residual limb. As another example, the programming would be established such that a patient’s tappingthe ball with their toe would result in a different intensity and / or Location of the haptic feedback, as compared to a patient’s attempting to kick the ball using full effort.

[0065] In a still further embodiment of the invention, control processor 18 may be suitably programmed to transmit signals to the headset to generate audible feedback; this would necessitate that headset 12 be provided with one or more speakers to generate sounds audible to the wearer. For example, audible feedback signals simulating the sound of the ball being kicked may be generated in response to the attempted movements by the patient (with the audible sound varying with the intensity of the force employed). Another feedback sound might be the sound of a heel striking the floor.

[0066] Accordingly, embodiment(s) of the present invention may present an entire suite of feedback stimulation, including auditory (various sound effects), tactile (vibration), and visual aspects (visible prosthetic leg). The purpose ofthis feedback is to create the most realistic experience possible, allowing the amputee to feel like the leg is physically a part of their body.

[0067] Fig. 6 illustrates a representative scenario of a patient operating the system of the present invention, and in particular demonstrating a representative image presented to the patient. Patient P dons headset 12 and sleeve 14, and is provided headset hand-held controller 70. Once the program is initiated, headset 12 displays image I which shows whatever patient P is looking at (or would be looking at, absent headset 12), with the virtual prosthetic PR mapped onto the residual limb. In alternative embodiments, headset handheld controller 70 may be positioned on the residual Limb and held in place, e.g., via a suitably configured holder, holster, etc. (not shown in Fig. 6, but understood to be present in some embodiments), the shape of which will be dictated by the shape and configuration of the controller 70 itself, which may vary from product-to-product in commercially-available AR headsets. In other alternative embodiments, multiple “hand-held” controllers may be provided, one held by the patient and one worn on the residual limb, so that the “worn” controller can provide location information about the residual limb to the headset, while the other truly hand-held controller allows for more ease of use and freedom of movement for the patient.

[0068] In an embodiment of the invention, the image of the virtual prosthetic that is presented to the patient in headset 12 is that of an all-black prosthesis. In alternative embodiments of the invention, programming may be provided to allow patients to select a customized version of the virtual prosthesis, in which a more highly detailed image may be provided, more closely representing a “real-life” prosthesis, be it a more mechanized- appearing prosthetic or a human-simulative version.

[0069] In an embodiment of the invention, there are two calibrations: one for the position of the virtual prosthetic leg and another for the maximum flexion and extension of the joint(s). First, for residual limb positioning calibration, the user (in this case, the clinician) rotates and presses the potentiometer knob and buttons on headset hand-held controller 70 to position the virtual prosthetic leg onto the residual Limb. The user can also adjust the length and width of various components of the virtual prosthetic to more accurately match what their physical prosthetic will Look like (this is the first step when the user opens the software). Second, for maximum flexion / extension of joint(s), calibration with EMGs (before doing exercises) may be done by measuring the MVCs (maximum voluntary contractions) of the patient with EMGs as well as recording them attempting to do certain movements with the virtual prosthetic. This information may be used as a baseline for the amputee’s maximum contraction of muscle groups recorded by the EMGs, and to determine what their typical muscle contractions look like from various movements.

[0070] In an embodiment of the invention, information is not passed from hand-held controller 70 of headset 12 through control processor 18. Instead, the positional data of controller 70 is received by headset 12 and interpreted by the application software. This requires the user to calibrate the virtual limb within the app. In alternative embodiments, IMUs embedded in sleeve 14 pass information through control processor 18. Headset controller 70, when affixed to sleeve 14, provides the position of the residual limb while sensors 16 interpret the muscle activation to determine the movement of the part of the limb that is missing (if the amputee is an above-the-knee amputation, then headset controller 70 provides information regarding Location of the residual limb and sensors 16 enable determination of appropriate joint angle, as described herein).

[0071] Once the patient has become comfortable with the operation of the training system, as an exemplar activity, a training exercise may involve the patient simulating the action of kicking a virtual soccer ball presented to the patient as a virtual image in headset 12. Virtual activities may include kicking a ball in a forward direction, stacking and knocking down balls sequentially, and striking the ball with various parts of the virtual prosthetic leg or foot. Sleeve 14 acquires signals from the targeted muscle groups, which are received by control processor 18 and translates those signals into data representing muscle activity in the residual limb of the patient. When the patient moves the residual limb in an attempt to kick the virtual soccer ball, control processor 18 translates that movement into a visual representation of the patient moving a virtually-presented prosthetic. Simultaneously, control processor 18 generates signals sent to haptic feedback generators 30 in sleeve 14 to generate physical stimuli representative of the movement of the residual limb with attached prosthetic. Thus, a closed-loop feedback system is established which is understood toincrease neuroplasticity, which, in turn, is understood to reduce loss of muscle memory and decrease phantom limb pain.

[0072] In an embodiment of the invention, information is not passed from hand-held controller 70 of headset 12 through control processor 18. Instead, the positional data of controller 70 is received by headset 12 and interpreted by the application software. This requires the user to calibrate the virtual limb within the app. In alternative embodiments, IMUs embedded in sleeve 14 pass information through control processor 18. Headset controller 70, when affixed to sleeve 14, provides the position of the residual Limb while sensors 16 interpret the muscle activation to determine the movement of the part of the limb that is missing (if the amputee is an above-the-knee amputation, then headset controller 70 provides information regarding location of the residual limb and sensors 16 enable determination of appropriate joint angle, as described herein).

[0073] Those skilled in the art will appreciate that the elements of the methods described in association with FIG. 5 can be stored as computer readable code on a non- transitory computer readable medium as desired for a particular application. In an embodiment of the invention, the programming is executed in the Python coding language, although other programming languages may be employed as desired. As defined herein, "non-transitory computer readable medium" comprises all computer readable medium, with the sole exception being a transitory, propagating signal. The non-transitory computer readable medium can include volatile and / or non-volatile memory. Volatile memory can include memory that depends upon power to store information, e.g., various types of dynamic random access memory (DRAM), and the like. Non-volatile memory can include memory that does not depend upon power to store information, e.g., solid state media such as flash memory, EEPROM, phase change random access memory (PCRAM), and the like. Other exemplary non-transitory computer readable medium include optical discs such as digital video discs (DVD), high definition digital versatile discs (HD DVD), compact discs (CD), and Laser discs; magnetic media such as magnetic tapes, tape drives, floppy discs, and magnetic hard drives; solid state media such as flash memory, memory cards, solid- state drives, USBflash drives, random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), magnetic random access memory (MRAM), phase change random access memory (PCRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM); and other types of media from which a computer, a processor or other electronic device can read.

[0074] Those skilled in the art will further appreciate that the elements of the methods described in association with FIG. 5 can be expressed as signals betweencomponents of the system when information is transferred between the components as desired for a particular application. Those skilled in the art will appreciate that programming code for a step of the method initiated in one component can require complementary programming code for a step of the method completed in another component.

[0075] It is important to note that FIGS. 1-5 illustrate specific applications and embodiments of the invention, and are not intended to Limit the scope of the present disclosure or claims to that which is presented therein. For example, other entities, such as a mobile device manufacturer, mobile device supplier, mobile device distributor, a third party, or the Like, can take the place of the payment system operator. Upon reading the specification and reviewing the drawings hereof, it will become immediately obvious to those skilled in the art that myriad other embodiments of the invention are possible, and that such embodiments are contemplated and fall within the scope of the presently claimed invention.

[0076] Those skilled in the art will appreciate that the elements of the methods described in association with FIGS. 1-5 can be beneficially combined as desired for a particular application. While the embodiments of the invention disclosed herein are presently considered to be preferred, various changes and modifications can be made without departing from the spirit and scope of the invention. The scope of the invention is indicated in the appended claims, and all changes that come within the meaning and range of equivalents are intended to be embraced therein.

[0077] The present disclosure encompasses an invention comprising a variety of aspects as enumerated in the following clauses:

[0078] Clause 1. A system for training an individual in the use of a prosthetic limb, comprising: a sleeve positionable on a residual limb of the individual, the sleeve including at least one sensor that receives signals representative of activation of at least one muscle within the residual Limb, the sleeve further including at Least one haptic feedback transducer; an augmented reality headset with at least one of visual image presentation and audio presentation; a control processor coupled in communication with the sleeve and the headset; the control processor receiving at least one signalfrom the at least one sensor in the sleeve representing activity of muscles within the residual Limb; the control processor further generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, theaugmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual; the control processor further generating signals to the at least one haptic feedback transducer in the sleeve as an indication to the individual of appropriateness of an attempted activation of muscles within the residual limb corresponding to an intended movement of a virtual prosthetic in the augmented reality image presented to the individual, the at least one haptic feedback transducer providing a physical tactile sensory indication to the individual.

[0079] Clause 2. The system according to clause 1, wherein the at least one sensor is an electromyography sensor.

[0080] Clause 3. The system according to clause 1, wherein the at least one haptic feedback transducer generates a vibration in response to receiving signals from the control processor.

[0081] Clause 4. The system according to clause 1 , wherein the sleeve comprises: a rectangular structure, including at Least one layer of material, and having a nominal inner surface and a nominal outer surface; the at least one sensor disposed on the nominal inner surface of the rectangular structure such that when the rectangular structure is wrapped around the residual limb of the individual, the at least one sensor is juxtaposed to and in physical contact with, a skin surface of the residual Limb; a closure mechanism coupled to the rectangular structure, such that when the rectangular structure is wrapped around the residual Limb of the individual, the closure mechanism releasably secures the rectangular structure to the residual Limb.

[0082] Clause s. The system according to clause 4, wherein the closure mechanism comprises complementary hook-and-loop fastener components affixed to the nominal inner and outer surfaces of the rectangular structure at opposite ends thereof.

[0083] Clause 6. The system according to clause 1, wherein the control processor comprises: at Least one pre-amplifier coupled to, and receiving signals from, the at least one sensor; at least one operational amplifier coupled to, and receiving signals from, the at Least one pre-amplifier;an analog-to-digital converter (ADC) coupled to, and receiving signals from, the at least one operational amplifier; and a central processing unit.

[0084] Clause ?. The system according to clause 6, wherein the control processor further comprises: an array of darlington bipolar transistors coupled in communication between the central processing unit and the at least one haptic feedback transducer.

[0085] Clause 8. A method for training an individual in the use of a prosthetic limb, comprising: positioning a sleeve positionable on a residual limb of the individual, the sleeve including at least one sensor that receives signals representative of activation of at least one muscle within the residual limb, the sleeve further including at least one haptic feedback transducer; providing an augmented reality headset; providing a control processor coupled in communication with the sleeve and the headset; the control processor receiving at least one signalfrom the at least one sensor in the sleeve representing activity of muscles within the residual limb; the control processor further generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual; the control processor further generating signals to the at least one haptic feedback transducer in the sleeve as an indication to the individual of appropriateness of an attempted activation of muscles within the residual limb corresponding to an intended movement of a virtual prosthetic in the augmented reality image presented to the individual, the at Least one haptic feedback transducer providing a physical tactile sensory indication to the individual.

[0086] Clause 9. The method according to clause 8, wherein the at least one sensor is an electromyography sensor.

[0087] Clause 10. The method according to clause 8, wherein the at least one haptic feedback transducer generates a vibration in response to receiving signals from the control processor.

[0088] Clause n . The method according to clause 8, wherein the sleeve comprises: a rectangular structure, including at Least one layer of material, and having a nominal inner surface and a nominal outer surface; the at Least one sensor disposed on the nominal inner surface of the rectangular structure such that when the rectangular structure is wrapped around the residual limb of the individual, the at least one sensor is juxtaposed to and in physical contact with, a skin surface of the residual Limb; a closure mechanism coupled to the rectangular structure, such that when the rectangular structure is wrapped around the residual Limb of the individual, the closure mechanism releasably secures the rectangular structure to the residual limb.

[0089] Clause 12. The method according to clause 11 , wherein the closure mechanism comprises complementary hook-and-loop fastener components affixed to the nominal inner and outer surfaces of the rectangular structure at opposite ends thereof.

[0090] Clause 13. The method to clause 8, wherein the control processor comprises: at Least one pre-amplifier coupled to, and receiving signals from, the at least one sensor; at least one operational amplifier coupled to, and receiving signals from, the at Least one pre-amplifier; an analog-to-digital converter (ADC) coupled to, and receiving signals from, the at least one operational amplifier; and a central processing unit.

[0091] Clause 14. The method according to clause 13, wherein the control processor further comprises: an array of darlington bipolar transistors coupled in communication between the central processing unit and the at least one haptic feedback transducer.

[0092] Clause 15. A method for training an individual in the use of a prosthetic limb, comprising the steps of:providing a sleeve positionable on a residual limb of the individual, the sleeve including at least one sensor that receives signals representative of activation of at least one muscle within the residual limb, the sleeve further including at least one haptic feedback transducer; fittingthe sleeve to the residual limb of the individualto ensure effective transmission of information from a skin surface of the residual limb of the individual to the at least one sensor; fitting an augmented reality headset to the individual; couplingthe at least one sensor, the at Least one haptic feedback transducer and the augmented reality headset in communication with a control processor; generating signals with the control processor corresponding to an augmented reality image; displaying the augmented reality image in the augmented reality headset, wherein the augmented reality image includes an image of a virtual prosthesis positioned at the end of an image of the residual limb; causing the individual to attempt to activate muscles in the residual limb in an effort to perform a function requiring the movement of a prosthesis; monitoring muscular activity of muscles within the residual limb of the individual; causing movement within the augmented reality image of the virtual prosthesis; providing at least one feedback stimulus to the individual.

[0093] Clause 16. The method according to clause 15, wherein the step of providing at least one feedback stimulus to the individual further comprises: generating a signal by the control processor and transmitting same to the at least one haptic feedback transducer.

[0094] Clause 17. The method according to clause 16, wherein the at least one haptic feedback transducer produces a vibratory response to the signal generated by the control processor.

[0095] Clause 18. The method according to clause 8, wherein the step of generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic Limb positioned on the residual limb of the individual, further comprisesvarying a characteristic of the at Least one feedback stimulus depending upon a type of action being attempted by the individual.

[0096] Clause 19. The method according to clause 8, wherein the step of generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic Limb positioned on the residual limb of the individual, further comprises the step of processing signals received from the at least one sensor, employing impedance control, using a mass-spring-damper analogy, to emulate joint behavior to determine a degree of bending of a joint to produce an augmented reality image of a virtual prosthetic movement in response to the attempted activation of muscles within the residual Limb of the individual.

Claims

WHAT IS CLAIMED IS:

1. A system for training an individual in the use of a prosthetic limb, comprising: a sleeve positionable on a residual limb of the individual, the sleeve including at least one sensor that receives signals representative of activation of at least one muscle within the residual limb, the sleeve further including at Least one haptic feedback transducer; an augmented reality headset with at least one of visual image presentation and audio presentation; a control processor coupled in communication with the sleeve and the headset; the control processor receiving at least one signalfrom the at least one sensor in the sleeve representing activity of muscles within the residual Limb; the control processor further generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual; the control processor further generating signals to at Least one of the at Least one haptic feedback transducer in the sleeve and the augmented reality headset for generation of at Least one of a feedback physical stimulus and a feedback auditory stimulus, in response to an attempted activation of muscles within the residual limb correspondingto an intended movement of a virtual prosthetic in the augmented reality image presented to the individual.2 The system according to claim 1, wherein the at least one sensor is an electromyography sensor.

3. The system accordingto claim 1 , wherein the at least one hapticfeedback transducer generates a vibration in response to receiving signals from the control processor.

4. The system according to claim 1 , wherein the sleeve comprises: a rectangular structure, including at Least one layer of material, and having a nominal inner surface and a nominal outer surface; the at least one sensor disposed on the nominal inner surface of the rectangular structure such that when the rectangular structure is wrapped around the residual limb of the individual, the at least one sensor is juxtaposed to and in physical contact with, a skin surface of the residual Limb;a closure mechanism coupled to the rectangular structure, such that when the rectangular structure is wrapped around the residual limb of the individual, the closure mechanism releasably secures the rectangular structure to the residual limb.

5. The system according to claim 4, wherein the closure mechanism comprises complementary hook-and-loop fastener components affixed to the nominal inner and outer surfaces of the rectangular structure at opposite ends thereof.

6. The system according to claim 1 , wherein the control processor comprises: at Least one pre-amplifier coupled to, and receiving signals from, the at Least one sensor; at least one operational amplifier coupled to, and receiving signals from, the at Least one pre-amplifier; an analog-to-digital converter (ADC) coupled to, and receiving signals from, the at least one operational amplifier; and a central processing unit.

7. The system according to claim 6, wherein the control processor further comprises: an array of darlington bipolar transistors coupled in communication between the central processing unit and the at least one haptic feedback transducer.

8. A method for training an individual in the use of a prosthetic limb, comprising: positioning a sleeve positionable on a residual Limb of the individual, the sleeve including at least one sensor that receives signals representative of activation of at Least one muscle within the residual limb, the sleeve further including at Least one haptic feedback transducer; providing an augmented reality headset with at Least one of visual image presentation and audio presentation; providing a control processor coupled in communication with the sleeve and the headset; the control processor receiving at least one signalfrom the at least one sensor in the sleeve representing activity of muscles within the residual limb; the control processor further generating signals representing an augmented reality image presented in a viewing surface of the augmented reality headset, theaugmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual; the control processor further generating signals to at least one of the at least one haptic feedback transducer in the sleeve and the augmented reality headset for generation of at Least one of a feedback physical stimulus and a feedback auditory stimulus, in response to an attempted activation of muscles within the residual limb correspondingto an intended movement of a virtual prosthetic in the augmented reality image presented to the individual.

9. The method according to claim 8, wherein the at Least one sensor is an electromyography sensor.

10. The method according to claim 8, wherein the at least one haptic feedback transducer generates a vibration in response to receiving signals from the control processor.

11. The method according to claim 8, wherein the sleeve comprises: a rectangular structure, including at least one layer of material, and having a nominal inner surface and a nominal outer surface; the at least one sensor disposed on the nominal inner surface of the rectangular structure such that when the rectangular structure is wrapped around the residual limb of the individual, the at least one sensor is juxtaposed to and in physical contact with, a skin surface of the residual Limb; a closure mechanism coupled to the rectangular structure, such that when the rectangular structure is wrapped around the residual limb of the individual, the closure mechanism releasably secures the rectangular structure to the residual Limb.

12. The method according to claim 11, wherein the closure mechanism comprises complementary hook-and-loop fastener components affixed to the nominal inner and outer surfaces of the rectangular structure at opposite ends thereof.

13. The method to claim 8, wherein the control processor comprises: at Least one pre-amplifier coupled to, and receiving signals from, the at Least one sensor; at Least one operational amplifier coupled to, and receiving signals from, the at Least one pre-amplifier; an analog-to-digital converter (ADC) coupled to, and receiving signals from, the at least one operational amplifier; anda central processing unit.

14. The method according to claim 13, wherein the control processorfurther comprises: an array of darlington bipolar transistors coupled in communication between the central processing unit and the at least one haptic feedback transducer.

15. A method for training an individual in the use of a prosthetic limb, comprising the steps of: providing a sleeve positionable on a residual limb of the individual, the sleeve including at least one sensor that receives signals representative of activation of at least one muscle within the residual limb, the sleeve further including at least one haptic feedback transducer; fittingthe sleeve to the residual limb of the individualto ensure effective transmission of information from a skin surface of the residual limb of the individual to the at least one sensor; fitting an augmented reality headset to the individual, the augmented reality headset having at Least one of visual image presentation and audio presentation; couplingthe at least one sensor, the at Least one haptic feedbacktransducer and the augmented reality headset in communication with a control processor; generating signals with the control processor corresponding to an augmented reality image; displaying the augmented reality image in the augmented reality headset, wherein the augmented reality image includes an image of a virtual prosthesis positioned at the end of an image of the residual limb; causing the individual to attempt to activate muscles in the residual limb in an effort to perform a function requiring the movement of a prosthesis; monitoring muscular activity of muscles within the residual limb of the individual; causing movement within the augmented reality image of the virtual prosthesis; providing at least one feedback stimulus to the individual.

16. The method according to claim 15, wherein the step of providing at least one feedback stimulus to the individual further comprises:generating a signal by the control processor and transmitting same to at least one of the at least one haptic feedback transducer and the augmented reality headset.

17. The method according to claim 16, wherein the at least one haptic feedback transducer produces a vibratory response to the signal generated by the control processor.

18. The method accordingto claim 8, wherein the step of generating signals representing an augmented reality image presented in aviewingsurface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual, further comprises varying a characteristic of the at least one feedback stimulus depending upon a type of action being attempted by the individual.

19. The method accordingto claim 8, wherein the step of generating signals representing an augmented reality image presented in aviewingsurface of the augmented reality headset, the augmented reality image including an image of a virtual prosthetic limb positioned on the residual limb of the individual, further comprises the step of processing signals received from the at least one sensor, employing impedance control, using a mass-spring-damper analogy, to emulate joint behavior to determine a degree of bending of a joint to produce an augmented reality image of a virtual prosthetic movement in response to the attempted activation of muscles within the residual limb of the individual.

Images