Robotic system for rehabilitation of a body part using virtual reality

Abstract

The present invention relates to a rehabilitation assistance system (100), comprising at least one robotic device, a virtual reality system, control electronics and a software application, said control electronics communicating with said robotic device and said application communicating with said control electronics and said virtual reality system so as to obtain a first data item representative of an ideal trajectory and a second data item representative of a movement of a user, determine first instructions for generating virtual feedback, compare said movement with said trajectory, determine second instructions for applying force feedback to said robotic device and transmit said first instructions to said virtual reality system and said second instructions to said control electronics. Figure for abstract: Figure 4

Description

Title of the invention: Robotic system for the virtual reality rehabilitation of a body part technical field

[0001] The present invention relates to the field of medical devices and systems.

[0002] The present invention relates more particularly to a rehabilitation system for a user's body part, stimulating a user's sensory abilities through the use of haptic robotic devices.

[0003] Rehabilitation in the context of the present invention means, throughout the following description, methods of facilitating the execution of tasks, in particular the performance of correct movements, in users suffering from a physical disability.

[0004] The present invention will thus find many advantageous applications in the fields of physiotherapy and occupational therapy, particularly in assisting the development of muscle functions. State of the art

[0005] The loosening of neural connections and transmissions between the brain and different parts of the body has serious long-term consequences. This can significantly impact the level of disability. Motor impairments affecting the upper and lower limbs are common in these individuals. These disabilities arise due to the loss of communication between the brain and the affected side of the body, for example, as a result of a stroke.

[0006] One year after the onset of the first stroke, physical independence (for 66% of stroke survivors) and occupation (for 75% of stroke survivors) are the areas of disability most affected.

[0007] This necessitates multidisciplinary rehabilitation of stroke patients, which aims to reduce the impact of the disease on daily functioning. Occupational therapy aims to facilitate task performance by improving relevant execution skills or by developing and teaching compensatory strategies to overcome lost performance skills.

[0008] Training in self-care activities, training in leisure activities and advice and instructions concerning assistive devices are the three most frequently chosen interventions for stroke patients.

[0009] Various interventions (for example, helping the upper limbs to perform correct movements such as reaching for touch) are carried out by therapists in hospitals or their clinics. This occupational therapy generally consists of The repetitive nature of physiotherapy exercises and the therapists' schedules can interfere with intensive rehabilitation programs for patients. The high cost of this type of therapy limits the number of patients who can benefit and reduces the potential for success. The repetitive nature of physiotherapy exercises limits the overall duration. Therefore, offering exercises with increased interest and challenge can capture the patient's attention and increase the time spent in therapy.

[0010] Virtual reality therapy is one of the methods used for rehabilitation. The main objective of using virtual reality in therapy is to increase patients' level of engagement in the rehabilitation process, as well as to motivate patients to do more and increase the time spent in rehabilitation. Virtual reality in therapy is classified as repetitive and objective therapy. Virtual reality scenes have been used in therapy primarily as stimulating and motivating environments for therapy; they are designed to facilitate the training of repetitive movements for individuals.

[0011] Robotic therapy is another known method in rehabilitation. Robots can be used both in assistive therapy, to accompany a movement, and in resistance therapy, to force the user to make an additional effort.

[0012] The Applicant submits, however, that rehabilitation systems combining robotic applications with virtual reality scenes are currently limited to providing users with a sense of touch in combination with visual and audio feedback. This combination aims, in particular, to increase user immersion, that is, interest and attention during therapy, as well as, more generally, the effectiveness of virtual reality therapy. However, simply providing tactile sensation offers only very limited information to the user and does not significantly enhance the development of abilities during rehabilitation.

[0013] The Applicant therefore submits that there is currently no satisfactory alternative solution to assist as much as possible the rehabilitation of a user, so as to perform physiotherapy exercises correctly and to derive all the benefits from them.

[0014] Summary of the invention,

[0015] The present invention aims to improve the current situation described above.

[0016] The present invention is more particularly aimed at overcoming the above limitations by offering a rehabilitation assistance system capable of enabling the user to develop new movement patterns.

[0017] To this end, the object of the present invention relates, in a first aspect, to a robotic system for assisting in the rehabilitation of at least one part of the body of a user, the system including: - at least one robotic device including a handle suitable for being attached to the user's body part; - a virtual reality system; - control electronics; and - a software application, the control electronics being configured to control at least one robotic device, the application being configured to communicate via wired or wireless means with the control electronics and the virtual reality system in order to: - obtain a first data representative of an ideal trajectory of the user, the ideal trajectory corresponding to a movement of the handle; - receive a second piece of data representing a movement of the user; - determine initial instructions for generating virtual feedback from the virtual reality system based on the first and / or second data point; - compare the movement with the ideal trajectory; - determine second instructions for applying force feedback to the robotic device, based on a comparison result; - transmit the first instructions to the virtual reality system and the second instructions to the control electronics.

[0018] It is understood here that the handle corresponds to any part or element of the robotic device with which the user's body part is connected. In other words, the handle serves as an interface between the user and the robotic rehabilitation assistance system. Thus, the user's movement displaces the handle, and the handle transmits force feedback from the robotic device. Depending on the body part involved, the user grasps the handle, places their body part on the handle, or attaches the body part to the handle. The handle may, for example, have a specific shape designed for its connection to a specific body part.

[0019] In other words, the system application receives, on the one hand, a trajectory to be followed by the user in connection with rehabilitation, and on the other hand, a movement performed by the user during their rehabilitation. The first and second pieces of data are received, for example, during the performance of a specific rehabilitation exercise. In a particular example, the first piece of data corresponds to the selection of a rehabilitation exercise from among a plurality of exercises, each rehabilitation exercise being associated with an ideal trajectory. The ideal trajectory corresponds, for example, to a given movement of a specific joint, or of an upper or lower limb of the user. The application determines initial instructions for generating virtual feedback, that is, feedback implemented by the virtual reality system. Virtual feedback includes, for example, visual and / or audio feedback, as known to a person skilled in the art.

[0020] The application compares the user's movement with the ideal trajectory, i.e., it matches the data or detects divergent movements, for example, specific movements corresponding to compensatory movements to be avoided during rehabilitation. The force feedback is thus determined by the application for implementation by the haptic device via the control electronics, in order to assist and guide the user during their rehabilitation, particularly when the user deviates, for example, from a given threshold, relative to the ideal trajectory.

[0021] In a particular example, the ideal trajectory corresponds to a range of positions, and comparing the movement with the ideal trajectory determines whether the user's movement falls within the ideal trajectory. When the user attempts a prohibited movement, i.e., when the user's movement deviates from the ideal trajectory, the application determines the force feedback to be applied, via the robotic device, to force the user back onto the ideal trajectory.

[0022] The second data corresponds, for example, to the evolution over time of the position of the user, of points representative of the user, for example of his joints, or of the position of the handle of the robotic device.

[0023] The control electronics correspond, for example, to an electronic board associated with the robotic device and communicating with the software application. In one particular example, the application is implemented at least partially by a microprocessor integrated into the control electronics or the virtual reality system, with instructions transmitted via a wired connection. In another example, the application is implemented remotely on a remote electronic device, for example, a central electronic device of the rehabilitation system, with instructions transmitted wirelessly.

[0024] It is further understood that the control electronics can be configured to control a plurality of robotic devices, for example associated with different parts of the user's body, in order to generate a variety of force feedback depending on the difference between the ideal trajectory and the user's movement, or to simultaneously track and control the movement of several body parts, for example a complex movement.

[0025] The Applicant submits that the human brain has the capacity to interpret sensory information provided by haptics, even if it is not real or presented in a natural way. In particular, blind people can correctly describe textures and shapes presented in virtual reality by receiving a haptic stimulation on their skin. This concept is known as sensory substitution, or as a tactile vision substitution system (TVSS), and takes advantage of the user's brain plasticity. In particular, sensory substitution allows information to be provided to the user that could not be transmitted through sight or hearing alone; however, visual, auditory, and tactile input must be maintained, for example, in such a way as to simulate a virtual obstacle.

[0026] The Applicant further submits that the method according to the invention performs a centralized determination of, on the one hand, virtual feedback, and on the other hand, force feedback. Such centralized determination ensures consistency between tactile, visual, and auditory stimuli, for example, in order to synchronize stimuli or, more generally, to represent an obstacle via several simultaneous and complementary stimuli. This consistency allows the user's brain, beyond simply experiencing a tactile sensation, to use the information provided to recreate new movement patterns. In other words, force feedback is used to improve the effectiveness of rehabilitation.

[0027] Thanks to the present invention, the rehabilitation assistance system provides an additional source of sensory data for the user, which is synchronized with all the sensory input transmitted by virtual reality. This source of sensory data can be integrated into a control strategy for the rehabilitation system, that is, a strategy comprising an ideal trajectory associated with a specific movement of the user's body. This control strategy invokes the healthy part of the user's brain to create new movement patterns in order to control the affected parts of the body, during rehabilitation exercises and then in daily life activities.

[0028] In one embodiment, the second instructions are further determined according to the first instructions.

[0029] In other words, the force feedback of the robotic device is determined based on the virtual feedback. Since the virtual feedback generally corresponds to a representation of the user's movement evolution during a rehabilitation exercise, for example in a rehabilitation scenario such as the one described below, the force feedback can be determined within the context of the virtual feedback, so as to ensure not only that the force feedback is consistent with the ideal trajectory, but also that it is derived from the content of the virtual feedback. This design also makes it possible to combine a robotic device with a virtual reality therapy system familiar to those skilled in the art, ensuring seamless integration of the robotic device.

[0030] In an advantageous embodiment of the invention, the application is further configured to obtain information representative of a rehabilitation scenario, and Obtaining the first piece of data corresponds to determining the ideal trajectory based on the rehabilitation scenario.

[0031] It is understood here that each rehabilitation scenario is associated with a specific rehabilitation exercise, a type of movement, and / or a degree of freedom of an upper or lower limb of the user. Each rehabilitation scenario is, for example, determined beforehand by a specialist, such as an occupational therapist. Obtaining information representative of a rehabilitation scenario corresponds, for example, to the application receiving information representing the selection of a scenario from among a plurality of scenarios stored in the memory of the rehabilitation assistance system, the memory being in communication with the application. According to another example, obtaining information representative of the rehabilitation scenario corresponds to receiving data, for example, by manual input via a human-machine interface, including specific characteristics of the rehabilitation scenario.

[0032] A person skilled in the art understands that each rehabilitation scenario is associated with an ideal trajectory, that is, a desired movement encouraged by the rehabilitation scenario. For example, if the rehabilitation scenario is developed to target the flexion-extension movement of the shoulder, the rehabilitation exercise is designed to ensure that the user performs the desired movement to move from an initial position to a final position. The rehabilitation scenario includes, for example, a plurality of virtual objects to collect or interact with, arranged in space according to the ideal trajectory. All other movements, that is, compensatory movements, correspond to a deviation from the ideal trajectory. The system then generates feedback based on such a deviation.

[0033] Based on information representing the rehabilitation scenario, the application then determines the ideal trajectory, that is, the trajectory that ensures that patients perform the desired movement associated with the rehabilitation scenario. The ideal trajectory thus corresponds to a trajectory allowing for correct muscle activation or the absence of compensatory movements.

[0034] According to another embodiment, a plurality of ideal trajectories are stored in a memory of the rehabilitation assistance system, with the first data point corresponding to the selection of the ideal trajectory based on the rehabilitation scenario. The rehabilitation scenarios, for example, belong to a set of rehabilitation scenarios that address the 14 degrees of freedom of the user's upper and lower limbs, i.e., the 7 degrees of freedom of the upper limbs and the 7 degrees of freedom of the lower limbs.

[0035] In a particular embodiment, the virtual reality system further comprises a display device, the first instructions comprising instructions displaying graphic content representative of the first data point and / or the second data point.

[0036] It is understood here that the integration of the display device corresponds to the addition of visual feedback to the rehabilitation assistance system, which is integrated with the virtual feedback. Such visual feedback is known to those skilled in the art, particularly in the context of virtual reality therapy. The display device corresponds, for example, to a screen placed in front of the user or to a virtual reality headset. The Applicant further submits that the use of a screen or other device located away from the user is preferable to a headset worn directly by the user, so that the user can see their upper and lower limbs during rehabilitation and control them more precisely.

[0037] The graphic content is, for example, representative of the ideal trajectory and / or the user's movement and / or a rehabilitation scenario as defined above. As another example, the initial instructions include instructions to display a visual alert based on a comparison result. Thus, force feedback is supplemented by a visual alert to better guide the user during their rehabilitation.

[0038] Preferably, the application is further configured to: - generate a virtual scene from the first piece of data; - determine the position of a representative avatar of the user in the virtual scene, based on the second piece of information. and in which the graphic content is representative of the virtual scene and the position of said avatar.

[0039] It is understood here that the virtual scene corresponds to a visual representation associated with a given rehabilitation exercise, or with the rehabilitation scenario described above, the virtual scene implicitly or explicitly including the ideal trajectory to be followed by the user. The virtual scene and the avatar thus allow the user to better perceive their movement during rehabilitation, in relation to the objectives set for them. The objective associated with the virtual scene corresponds, for example, to the manipulation of a given virtual object or to reaching a plurality of virtual objects to be collected.

[0040] Preferably, the comparison of the movement with the trajectory is carried out from the position in the virtual scene.

[0041] In other words, the determination of deviations between the user's movement and the ideal trajectory is performed remotely, by comparing the movement of the user's avatar with the ideal trajectory of the virtual scene. The force feedback applied by the robotic device, or any other additional feedback, is thus determined in such a way as to enhance consistency with the content The graphic is generated by the display device, i.e., with virtual feedback. In a particular design, the force feedback of the robotic device, determined via the application, is added to the determination of visual or auditory feedback, as described below, performed according to methods known to those skilled in the art. In this example, the application extracts real-time information about the position of the user, represented by an avatar, in the virtual scene, to determine the force feedback.

[0042] In an additional embodiment, the virtual reality system further includes a sound device, the first instructions including instructions for generating audio feedback representative of the first data and / or the second data.

[0043] It is understood here that, like the display device, the use of sound devices within a rehabilitation assistance system, particularly during virtual reality therapy, is known to those skilled in the art. The virtual reality system includes, for example, both a display device and a sound device, or a single device configured to generate graphic content and audio feedback. The audio feedback is, for example, also representative of the virtual scene described above, so as to provide a complete set of sensory information integrating tactile feedback with the existing visual and audio feedback. A combined design of all the feedback thus makes it possible to maximize the potential for producing and reproducing precise movements.

[0044] It is further understood that it is possible to integrate or not the use of visual and / or audio feedback depending on the needs and / or pathology of the user, according to the knowledge of the person skilled in the art.

[0045] In a specific embodiment, the second data is received from at least one robotic device.

[0046] It is understood here that the user's movement corresponds to the movement as perceived by the robotic device, in particular the movement of the handle. This simple design therefore corresponds to tracking the user's movement relative to the ideal trajectory, converting the user's movement into a specific displacement of the handle and vice versa. Rehabilitation exercises, for example the rehabilitation scenarios described above, are designed with respect to a specific movement of the robotic device to ensure the desired movement for the user.

[0047] According to one embodiment, the robotic rehabilitation assistance system includes a capture device communicating with the virtual application, the second data being received from the capture device. Such a capture device corresponds, for example, to any type of sensor known to a person skilled in the art and se selected according to a variety of criteria, for example ease of use, accuracy of capturing user movements or interoperability with other elements of the system.

[0048] In yet another embodiment, the robotic device includes a base, the handle being articulated relative to the base to allow three translational movements and one rotational movement of the handle relative to the base.

[0049] It is understood here that the movements by which the handle is articulated relative to the base correspond to the movements by which the handle movement is possible, the movements by which the robotic device can apply force feedback, or the movements by which the robotic device can obtain the second input. Such movements correspond to a haptic robot architecture known to those skilled in the art. Obviously, it is also possible to integrate a wide variety of robotic devices known to those skilled in the art into the system, which, for example, have various configurations and articulations, or are specifically designed for rehabilitation.

[0050] In an additional embodiment, the force feedback corresponds, depending on the result of the comparison, alternately to a blocking or a release of the user's movement.

[0051] In other words, the rehabilitation assistance system is configured to reproduce a contact stiffness representing an obstacle to the user, i.e., a virtual wall preventing the user from deviating from the ideal trajectory. The ideal trajectory thus corresponds to a free trajectory, with other directions of movement generating force feedback.

[0052] According to other variants, the force feedback is weaker and corresponds, for example, to a simple vibration when the user's movement is incorrect. It is thus possible to design a variety of feedback intensities suitable for encouraging the user to return to the ideal trajectory, for example, a simple vibration when the user's movement deviates from the ideal trajectory, or a gradation in vibration intensity depending on the user's rehabilitation progress.

[0053] In another embodiment, the force feedback corresponds, depending on the result of the comparison, alternately to a blocking or a resistance to the movement of the user.

[0054] In other words, when the user's movement corresponds to the ideal trajectory, the rehabilitation assistance system continues to apply force feedback against the user's movement, for example, a weaker force feedback than when the user deviates from the ideal trajectory. Thus, the user must use their muscles more to follow the ideal trajectory, while being guided. in this ideal trajectory. This design allows for participation in the user's muscle rehabilitation, particularly in a second phase of rehabilitation once the user has created new movement patterns. The system then trains the user's muscles to reproduce the instructions given by the brain.

[0055] It is still possible to imagine other variations, for example, force feedback configured to accompany the user's movement along the ideal trajectory, assisting correct movements in addition to blocking deviations. Those skilled in the art understand that the level of assistance or resistance depends primarily on the user's profile and rehabilitation history.

[0056] In an additional embodiment, the application is further configured to receive a third piece of data representative of a level of user rehabilitation, and the second instructions are further determined based on the third piece of data.

[0057] Preferably, the third data includes information representative of a muscle rehabilitation force and the second instructions include a force feedback determined according to the third data.

[0058] In other words, this design allows the force feedback to be adapted, as suggested in the variants described above, according to user-specific information, particularly their needs and physical abilities. The third piece of data is, for example, combined with the first piece of data, or included in information representative of a rehabilitation scenario as described above. The application receives information, for example, that characterizes both the type of movement to be performed by the user, and therefore the ideal trajectory, and the level of force to be applied when the user performs this movement, whether to assist, resist, or block the user's movement.

[0059] According to a second aspect, the present invention relates to a computer program comprising instructions for implementing the following steps: - obtaining a first data representative of an ideal trajectory of a user; - reception of a second piece of data representing a movement of the user; - determination of initial instructions for generating virtual feedback from a virtual reality system based on the first and / or second data point; - comparison of the movement with the ideal trajectory; - determining second instructions for applying force feedback to a robotic device, based on a comparison result; and - transmission of the first instructions to the virtual reality system and of the second instructions to a control electronics of the robotic device.

[0060] It is understood here that the computer program includes instructions adapted for the execution of the steps implemented by the software application according to the first aspect of the present invention.

[0061] According to one variant, the computer program also includes instructions adapted for the execution of the variants described above in connection with the first aspect of the present invention.

[0062] According to a third aspect, the present invention relates to a computer-readable recording medium on which a computer program according to the second aspect of the invention is recorded.

[0063] On the one hand, the recording medium can be any entity or device capable of storing the program. For example, the medium can include a storage means, such as a ROM, a CD-ROM or a microelectronic circuit-type ROM, or a magnetic recording means or a hard disk drive.

[0064] On the other hand, this recording medium can also be a transmissible medium such as an electrical or optical signal, such a signal being able to be transmitted via an electrical or optical cable, by conventional or radio frequency, by self-directing laser beam, or by other means. The computer program according to the present invention can, in particular, be downloaded from an Internet-type network.

[0065] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to execute or to be used in the execution of the process in question.

[0066] Thus, through the various functional and structural technical characteristics above, the Applicant proposes a system and a program for assisting rehabilitation allowing haptic sensory feedback, offering a greater amount of information to the user and being combined with visual and auditory feedback from virtual reality therapy so as to provide a coherent set of sensory data allowing the user to create new movement patterns. Brief description of the figures

[0067] Other features and advantages of the present invention will become apparent from the description of the particular and non-limiting embodiments of the present invention below, with reference to the attached Figures 1 and 4, in which:

[0068] [Fig.1]

[0069] Fig. 1 schematically illustrates a rehabilitation assistance system for at least one part of a user's body, according to a particular and non-limiting embodiment of the present invention;

[0070] [Fig.2]

[0071] [Fig.2] illustrates a flowchart of the different stages of a rehabilitation assistance process implemented by a system conforming to [Fig.1];

[0072] [Fig.3]

[0073] Fig. 3 illustrates a system conforming to Fig. 1 used for assisting in the rehabilitation of a user's lower limb;

[0074] [Fig.4]

[0075] Fig. 4 illustrates a system conforming to Fig. 1 used for assisting in the rehabilitation of an upper limb of a user. Detailed description

[0076] A rehabilitation assistance system for at least one part of a user's body will now be described in the following text with joint reference to Figures 1 and 4 attached to the description. The same elements are identified with the same reference symbols throughout the following description.

[0077] As stated in the preamble to the description, current virtual reality therapy solutions are limited to visual and / or audio feedback, which offers only limited user engagement and does not allow for precise guidance of movements during rehabilitation exercises. On the other hand, current rehabilitation system solutions using robotics and haptics are limited to providing a tactile sensation to increase user engagement and the time spent in rehabilitation.

[0078] One of the objectives of the present invention is to enable the use, by a person in rehabilitation, for example in a situation of disability, of a system maximizing the recovery of physical skills and sensory stimulation.

[0079] This is made possible in the example described below.

[0080] According to the example in Figures 1, 3 and 4, a rehabilitation assistance system 100 is used by a user 3, for example a person who has suffered a stroke or is suffering from some kind of disability, for whom rehabilitation is likely to increase motor skills.

[0081] In this same example, the system 100 comprises at least one robotic device 110 equipped with a handle 111 capable of being associated with a part of the user's body 3, i.e., worn, for example in direct contact with the body, during its use. According to the example in [Fig. 3], the handle 111 is thus attached to a foot of the user 3, while according to the example in [Fig. 4], the handle 111 is associated with a hand of the user 3, for example held by the user 3.

[0082] Such a robotic device 110 is configured to provide haptic feedback, or in other words, force feedback. The robotic device 110 includes, for exampleat least one vibration motor, that is, a device configured to generate vibrations of fixed or variable intensity according to a command. In another example, the robotic device 110 includes at least one motor configured to apply a force along one or more degrees of freedom of the robotic device 110. More generally, the robotic device 110 includes one or more electronic components known to those skilled in the art and configured to apply a force, vibrations, or pulsations in a controlled manner. It is understood here that, depending on the variants considered, the needs and objectives of those skilled in the art, a plurality of robotic devices 110 can be used, for example, depending on the desired tactile feedback or force feedback.The robotic device 110 can in particular be selected, in accordance with the description below, according to its ability to mimic a wall or a virtual barrier, i.e. to block the movement of the user 3 when the robotic device 110 is activated.

[0083] It is understood here that the force applied by the robotic device 110 depends on its architecture, in particular on the articulation of the handle 111. The robotic device 110 includes, for example, a base about which the handle 111 is articulated, the robotic device 110 being able to apply force feedback for each movement along which the handle 111 is articulated. In a specific example, the handle 111 is articulated about the base to allow three translational movements and one rotational movement, for example as illustrated with respect to the coordinate system of [Fig. 4], three translational movements along the X, Y, and Z axes, as well as one rotational movement about the Y axis.

[0084] According to a particular variant, the system 100 comprises a plurality of robotic devices 110 associated with the same user 3, for example associated with different parts of the user 3's body. For example, a pair of robotic devices 110 are provided, associated with two hands or two feet of the user 3, or any other combination of robotic devices 110 allowing for addressing more complex or more precise movements of the user 3.

[0085] Optionally, the system 100 also includes at least one motion capture device, which is also intended to be associated with the user 3. It is understood here that the motion capture device is configured to track the movement of the user 3 over time and can be implemented in a variety of ways known to those skilled in the art. In a preferred embodiment, the capture device includes at least one Azure Kinect® sensor, i.e., a device comprising a depth sensor, a spatial microphone array with a video camera, and an orientation sensor, and enabling precise tracking of the user 3's movements. In another embodiment, the system 100 includes or is coupled with a virtual reality therapy system 150, as described below, including a virtual reality headset. The capture device, in this example, comprises a plurality of sensors connected to the virtual reality headset. Such sensors are known to those skilled in the art for controlling an avatar of user 3 in a virtual scene, for example, the virtual scene generated by the application as described below, or more generally, any virtual scene created in the field of virtual reality. Obviously, the capture device may advantageously include a combination of the aforementioned sensors. According to another variant, the robotic device 110 includes integrated sensors for tracking its movement, the movement of the handle 111 being linked to the movement of user 3.

[0086] In combination with the robotic device 110, the system 100 includes control electronics 130 in communication with the haptic device 110, so as to control it. The control electronics 130 is, for example, integrated into the robotic device 110 or configured to be connected to a plurality of robotic devices 110 associated with the user 3. In order to allow control of the electronic elements of the system 100, in particular the robotic devices 110, the control electronics 130 includes, for example, a power supply and at least one processor. In a preferred embodiment, the control electronics 130 includes an electronic board, for example of the Arduino® type, programmed to control the robotic device 110.

[0087] In addition, the system 100 also includes a virtual reality system 150, corresponding, for example, to a virtual reality therapy system known in the prior art. The virtual reality system 150 includes, for example, a display device 151 and / or a sound device 152, respectively configured for rendering visual and / or audio feedback. According to a first design, the assembly formed by the haptic device 110 and the control electronics 130 is separate from the virtual therapy system 150. According to another design in accordance with [Fig. 1], the virtual reality therapy system 150 and the other elements described above are grouped within the system 100 and jointly controlled, for example, by the virtual reality therapy system 150 or by a remote device 140 as described below.

[0088] It is understood here that the visual feedback from the display device 151 corresponds to graphic content, for example, graphic content representing the data processed below during the process in [Fig. 2]. The visual feedback, the audio feedback, and the force feedback from the haptic device 110 advantageously correspond to complementary renderings of the same data, allowing all of the user's senses 3 to be stimulated during their rehabilitation.

[0089] Finally, the system 100 also includes a software application configured for The system communicates with the virtual reality system 150 and the control electronics 130 via wired or wireless connections. In one design, the software application is directly implemented by the processor of the control electronics 130, which communicates with the other components of the control electronics 130 via a wired connection. In a second design, the software application is implemented by a remote device 140 that communicates with the control electronics 130, specifically with its processor. This design allows, in particular, for a reduction in the processing power required by the control electronics 130, freeing up resources for communication with the remote device 140.The remote device 140 corresponds, for example, to an electronic device associated with a therapist, for example equipped with a human-machine interface 160 with which the therapist can interact to guide the user's rehabilitation 3. According to yet another variant, the remote device 140 is integrated into the virtual reality therapy system 150. The software application and the control electronics 130 thus make it possible to control the robotic device 10 and the virtual reality system 150. In particular, a combined control of the robotic device 110 and the virtual reality system 150 makes it possible to ensure consistency between force feedback and visual and / or audio feedback, by integrating the robotic device 110 and the virtual reality system 150 into the same system 100.

[0090] The system 100 described above is thus configured to implement the steps of a rehabilitation assistance process according to [Fig.2]. Such a process is, for example, recorded according to instructions in a computer program, for example stored on a dedicated storage medium, for example in a memory of the control electronics 130, the remote device 140 or the virtual reality therapy system 150.

[0091] In a first step 21, the application obtains initial data representing an ideal trajectory for the user 3. In other words, this initial data depends on the rehabilitation being sought and represents a movement target for the user 3. The ideal trajectory advantageously corresponds to a movement of the handle 111, i.e., a specific interaction between the user 3 and the robotic device 110, which can be influenced by force feedback from the robotic device 110. The initial data is thus, for example, received through communication with the virtual reality therapy system 150; that is, the application receives information on the rehabilitation exercise performed via the virtual reality therapy system in order to integrate control of the robotic device 110.In another example, the first piece of data is received via communication with the human-machine interface 160 and corresponds to an instruction given by the therapist.

[0092] According to one embodiment, the application is configured to obtain information representative of a rehabilitation scenario. Obtaining the first data point then corresponds to determining the ideal trajectory based on the rehabilitation scenario. Here, a rehabilitation scenario corresponds to a given rehabilitation exercise, corresponding to a specific type of movement, a joint, or a given degree of freedom of the user 3. In other words, the software application determines, based on the rehabilitation exercise to be performed or parameters associated with such an exercise, the ideal trajectory that the user 3 must maintain. The ideal trajectory therefore corresponds to the trajectory that ensures that the user 3 performs exactly the desired movement for a given rehabilitation scenario.

[0093] Determining the ideal trajectory corresponds, for example, to selecting the ideal trajectory from a set of predefined trajectories, for example, based on manual selection information from the therapist, or based on a selected rehabilitation scenario, the correspondence between the ideal trajectory and the rehabilitation scenario being stored in a memory communicating with the software application. For example, a set of rehabilitation scenarios, created manually and stored in a memory of system 100, is provided, allowing all movements that can be rehabilitated by system 100 to be addressed, for example, the 14 degrees of freedom of the user's limbs 3.

[0094] In a second step 22, the application receives a second data representative of a movement of the user 3. In accordance with the description above, the second data is thus transmitted by the robotic device 110 and / or the capture device to the control electronics 130, the remote device 140 or the virtual reality therapy system 150, depending on the component implementing the software application and the device enabling the acquisition of the second data.

[0095] It is understood here that the information contained in the second data point depends on its acquisition. In one embodiment, the second data point comprises information representing a movement of the handle 111, or a set of information representing a movement of a plurality of joints of the robotic device 110. The movement of the user 3 thus results from their interaction with the handle 111, which can be tracked without difficulty. In another embodiment, the second data point comprises a set of information representing the distance between the capture device and the user 3. The set of information corresponds, for example, to the distances between the capture device and the joints of the user 3, as perceived by the capture device. The application then determines the movement of user 3 from the set of representative distance information, for example based on their evolution over time or by positioning user 3 relative to the reference position of the capture device.

[0096] In a third step 23, the application determines initial instructions for generating virtual feedback based on the first and / or second data point. It is understood here that virtual feedback corresponds to the visual and / or audio feedback implemented by the virtual reality system 150, for example, any form of feedback known to a person skilled in the art within the context of virtual reality therapy.

[0097] According to a particular embodiment, in which the rehabilitation assistance system 100 includes the display device 151, the application is configured to generate a virtual scene from the first data point. The application then determines the position of a representative avatar of the user 3 within the virtual scene from the second data point. The graphic content rendered by the display device 151 is then representative of the virtual scene and the position of the avatar. Optionally, the system 100 also includes the sound device 152, the audio feedback also being representative of the virtual scene.

[0098] In a fourth step 24, the movement of the user 3 is compared with the ideal trajectory, that is to say the application matches the movement with the ideal trajectory or identifies and characterizes any possible deviation of the movement from the ideal trajectory.

[0099] In a variant combined with the variant described above, the comparison between the movement and the ideal trajectory is performed based on the position of the user's avatar 3 in the virtual scene. In other words, the avatar's position in the virtual scene determines whether the user's movement 3, for example, the position of a given limb, is within the ideal trajectory. This design ensures consistency between the virtual feedback and the rest of the method according to the invention.

[0100] In a fifth step 25, the application determines second instructions for applying force feedback to the robotic device 110 based on a comparison result. In a fifth step 26, the first instructions are then transmitted to the virtual reality system 150 and the second instructions to the control electronics 130. The virtual reality system 150 then implements the first instructions, for example by controlling the display device 151 and the sound device 152 to generate the virtual scene described above. In parallel, the control electronics 130 implements the second instructions by controlling the robotic device 110. The control of the robotic devices 110 corresponds, for example, to starting or stopping one or more integrated motors, for example vibration motors.

[0101] Thus, when the movement of user 3 corresponds to the ideal trajectory, it is understood that the robotic device 110 does not provide any force feedback. Similarly, any visual and audio feedback is adapted so as not to alert user 3; for example, a simple audio feedback representing the evolution of user 3's movement is provided.

[0102] Conversely, when the movement of user 3 deviates from the ideal trajectory, the control of the robotic device 110 is adapted to indicate this deviation to user 3 and force them to return to the ideal trajectory. The visual and audio feedback also includes, for example, an alert or an indication of the correction to be made to return to the ideal trajectory.

[0103] It is understood here that the instructions, or their implementation via the control electronics 130, are adapted according to the number and / or position of the robotic devices 110. The control electronics 130 triggers, for example, a force feedback from a specific robotic device 110 or in a specific direction in order to indicate more precisely to the user 3 which joint to reposition or in which direction to correct the movement.

[0104] According to an advantageous embodiment, the second instructions are further determined based on the first instructions. In other words, and similar to the embodiment described above in which the comparison in the fourth step 24 is performed using the virtual scene, the force feedback is determined based on the virtual feedback, so as to ensure consistency among all the feedback and to provide usable information for the user's brain 3 in its construction of movement models. This determination of the second instructions based on the first instructions corresponds, for example, to a temporal and spatial synchronization of the feedback, for example, so that when the user reaches a virtual obstacle, it is represented at the same time and in the same direction via haptic, visual, and optionally audio signals.

[0105] In parallel, the instructions or their implementation can also be adapted in intensity, according to the type or operating interval of the robotic devices 110, so as to provide adaptive force feedback according to the user 3 or the amplitude of the difference between the movement and the ideal trajectory.

[0106] According to a specific design, the force feedback corresponds to a blocking or release of the movement of user 3. The robotic device 110 is thus controlled to prevent user 3 from deviating further from the ideal trajectory, or more generally to provide significant resistance, depending on the ability of the robotic device 100 to mimic a virtual wall. Obviously, such a blocking of movement is associated with a specific direction, the return of user 3 in the Ideal trajectory remaining free.

[0107] According to another embodiment, the force feedback corresponds alternatively to a blocking or resistance to the movement of user 3. In other words, when user 3 follows the ideal trajectory, the robotic device 110 continues to apply force feedback, for example, a weaker force feedback than when user 3 deviates from the ideal trajectory. User 3 must therefore exert additional effort to perform the desired movement, which contributes to muscle rehabilitation.

[0108] It is therefore understood that the force feedback can be adapted in a variety of ways, in particular according to the user's rehabilitation path. In yet another design, the application also receives a third piece of data representing a level of user 3's rehabilitation, or more generally, representing a user 3 profile. The determination of the second instructions according to the fifth step 25 is then further carried out based on the third piece of data, that is to say, in order to adapt the force feedback according to user 3. The application obtains, for example, the first and third pieces of data together in a specific rehabilitation scenario for a specific movement and rehabilitation path, that is to say, a scenario for which the ideal trajectory and the force feedback are predetermined.

[0109] According to a specific example, this third input comprises information representing a muscle retraining force, the force feedback being determined based on this muscle retraining force. This force feedback adaptation applies, for example, when user 3 follows the ideal trajectory, in order to apply resistance as described above, or conversely, to guide user 3 by accompanying their movement with a force in the same direction. The force feedback adaptation can also apply when user 3 deviates from the ideal trajectory, in order to be more or less permissive towards deviations and compensatory movements, and to balance user engagement with the correction of their errors.

[0110] Thus, it will be understood that the present invention provides a rehabilitation assistance system allowing, in a simple way, the combination of haptic feedback implemented by a robotic device with already known virtual reality therapy systems in order to carry out a variety of learning scenarios created by therapists, ensuring consistency between all feedback so that the haptic feedback facilitates the learning of movements by the user.

[0111] It should be noted that this detailed description relates to a particular embodiment of the present invention, but that in no way does this description limit the scope of the invention; on the contrary, its purpose is to eliminate any possible imprecision or misinterpretation of the The following demands

[0112] It should also be noted that the reference signs in parentheses in the following claims are in no way intended to be limiting; these signs are solely intended to improve the intelligibility and understanding of the following claims and the scope of the protection sought.

Claims

Claims

1. Robotic system (100) for assisting in the rehabilitation of at least one body part of a user (3), said system comprising: - at least one robotic device (110) comprising a handle capable of being associated with said body part of said user (3); - a virtual reality system (150); - control electronics (130); and - a software application, said control electronics (130) being configured to control said at least one robotic device (110), said application being configured to communicate by wire or wireless means with said control electronics (130) and said virtual reality system (150) so as to: - obtain (21) a first data item representative of an ideal trajectory of said user (3), said ideal trajectory corresponding to a movement of said handle 111; - receive (22) a second data item representative of a movement of said user (3);- determining (23) first instructions for generating a virtual feedback of said virtual reality system (150) as a function of said first data and / or said second data; - comparing (24) said movement with said ideal trajectory; - determining (25) second instructions for applying a force feedback of said robotic device (110), as a function of a result of said comparison; - transmitting (26) said first instructions to said virtual reality system (150) and said second instructions to said control electronics (130).;

2. A robotic rehabilitation assistance system (100) according to claim 1, wherein said second instructions are further determined (25) based on said first instructions.

3. Robotic rehabilitation assistance system (100) according to claim 1 or 2, wherein said application is further configured to obtain information representative of a rehabilitation scenario, and wherein said obtaining (21) of said first data corresponds to a determination of said ideal trajectory as a function of said rehabilitation scenario.

4. Robotic rehabilitation assistance system (100) according to one of claims 1 to 3, wherein said virtual reality system (150) further comprises a display device (151), said first instructions comprising instructions for displaying graphic content representative of said first data and / or said second data.

5. Robotic rehabilitation assistance system (100) according to claim 4, wherein said application is further configured to: - generate a virtual scene from said first data; - determine a position of an avatar representative of said user (3), in said virtual scene, from said second data, and wherein said graphic content is representative of said virtual scene and said position of said avatar.

6. A robotic rehabilitation assistance system (100) according to claim 5, wherein said comparison (24) of said movement with said trajectory is performed from said position in said virtual scene.

7. A robotic rehabilitation assistance system (100) according to one of claims 1 to 6, wherein said virtual reality system (150) further comprises a sound device (152), said first instructions comprising instructions for generating audio feedback representative of said first data and / or said second data.

8. A robotic rehabilitation assistance system (100) according to one of claims 1 to 7, wherein said second data is received from said at least one robotic device (110).

9. A robotic rehabilitation assistance system (100) according to claim 8, wherein said robotic device 110 comprises a base, said handle 111 being articulated relative to said base to allow three translational movements and one rotational movement of said handle 111 relative to said base.

10. Robotic rehabilitation assistance system (100) according to one of claims 1 to 9, wherein said force feedback corresponds, depending on said result of said comparison, alternately to a blocking or a release of said movement of said user (3).

11. Robotic rehabilitation assistance system (100) according to one of claims 1 to 9, wherein said force feedback corresponds, depending on said result of said comparison, alternatively to a blockage or to a resistance towards said movement of said user (3).

12. Robotic rehabilitation assistance system (100) according to one of claims 1 to 11, wherein said application is further configured to receive a third piece of data representative of a level of rehabilitation of said user (3), and wherein said second instructions are further determined as a function of said third piece of data.

13. A robotic rehabilitation assistance system (100) according to claim 12, wherein said third data item comprises information representative of a muscular rehabilitation force, and wherein said second instructions comprise force feedback determined as a function of said third data item.

14. Computer program product comprising instructions for implementing the following steps: - obtaining (21) a first data item representative of an ideal trajectory of a user (3); - receiving (22) a second data item representative of a movement of said user (3); - determining (23) first instructions for generating a virtual feedback of a virtual reality system (150) as a function of said first data item and / or said second data item; - comparing (24) said movement with said ideal trajectory; - determining (25) second instructions for applying force feedback of a robotic device (110), as a function of a result of said comparison; - transmitting (26) said first instructions to said virtual reality system (150) and said second instructions to control electronics (130) of said robotic device (110), when these instructions are executed by a processor.

15. A computer-readable recording medium having recorded thereon a computer program according to claim 14.