Intelligent adaptive control system for prosthetic, orthotic or artificial limb

Abstract

A method for the self-development of a control agent for a prosthesis, orthosis or an artificial limb, said control agent being configured for its control by its control means, comprising the steps of: obtaining a motion model defining a mechanical relationship between elements of the prosthesis or the like; acquiring a training response model defining a predefined motion response of the prosthetic limb and the like; and determining the current motion state of the artificial limb and the like. And additionally comprising repeating the steps of: determining a candidate mechanical action based on a current motion state associated with an element of an artificial limb or the like; outputting a next candidate motion state based on the current motion state and the candidate mechanical action; outputting a next predefined motion state based on the current motion state; obtaining a reward value based on the next candidate motion state and the next predefined motion state; and setting the next candidate motion state as the current motion state.

Description

Technical Field

[0001] The field of this invention generally relates to systems and methods for prostheses, orthoses, or artificial limbs, which enable control of the prostheses, orthoses, or artificial limbs using specially developed models based on machine learning and reinforcement learning. Specific embodiments relate to matters concerning prostheses or orthoses for functionally assisting, enhancing, and / or replacing limbs of human or animal subjects, or for enhancing the body or part of the body of human or animal subjects, and to matters concerning artificial limbs for functionally serving as limbs of humanoid robots or animal-like robots. Background Technology

[0002] In existing prostheses, orthoses, or artificial limbs, it is known that data collected using sensors associated with the operation of the prosthesis, orthosis, or artificial limb can provide feedback during use. The collected data can be taken into account to enable more accurate control of the operation of the prosthesis, orthosis, or artificial limb. This is especially true for prostheses, orthoses, or artificial limbs that include actuators (e.g., motors or dampers). Body movement (in humans or animals) involves the coordination of various interrelated and complex translations and rotations of different body parts. Monitoring and data collection are necessary to provide a more comfortable experience for the user of the prosthesis or orthosis, or to enable the artificial limb to reproduce organic body movements that more closely resemble natural body movements. After data collection, processing the data to actively or passively control different elements of the prosthesis, orthosis, or artificial limb allows for the simulation of movements of the corresponding body parts. However, the active or passive control of the prosthesis, orthosis, or artificial limb still relies on a predetermined configuration and / or behavioral dataset associated with the elements of the prosthesis, orthosis, or artificial limb, and its adaptability is limited relative to actual possible environmental or behavioral changes.

[0003] In existing technological solutions, various pre-defined configurations and / or behavioral datasets associated with a fixed number of environmental settings and / or behaviors are used to address the aforementioned problems. However, this method of controlling prostheses, orthoses, or artificial limbs has limitations in terms of adaptability, particularly during transitions between different environments or behaviors, where precise control of the prostheses, orthoses, or artificial limbs to approximate natural body movements remains lacking. Summary of the Invention

[0004] The purpose of this invention is to provide, improve and / or develop a control agent, preferably based on machine learning, such as deep learning, probabilistic models, Gaussian processes and reinforcement learning, for controlling prostheses, orthotics or artificial limbs so that they can more easily and accurately adapt to environmental disturbances (e.g., speed changes and terrain changes) and changes in activity (or behavior).

[0005] According to a first aspect of the invention, a system including a prosthesis, orthosis, or artificial limb is provided. The system includes at least one sensing device and a state determination device. The at least one sensing device is configured to sense data related to movement performed by a user involving a prosthesis, orthosis, or artificial limb worn or equipped by the user. The state determination device is configured to repeatedly perform the following steps: determining a state associated with the movement based on sensing data acquired from the at least one sensing device; and acquiring an uncertainty value representing the level of uncertainty of the user being in the determined state.

[0006] By determining the state associated with movement and the uncertainty value, the control agent used to control the prosthesis, orthosis, or prosthetic limb can base its output on this state and uncertainty value to more accurately allow the prosthesis, orthosis, or prosthetic limb to mimic equivalent natural body movements. Unlike methods based on a fixed number of predetermined configurations and / or behavioral datasets and depending on whether the movement indicates that the user wearing the prosthesis, orthosis, or prosthetic limb has switched from one activity to the next, the control of the prosthesis, orthosis, or prosthetic limb can be continuously and adaptively adjusted through iterative determination of the state, thereby providing a smoother adaptation for activity switching or transitions from one activity to the next. Furthermore, the acquired uncertainty value can be used as a safety mechanism to prevent incorrect control of the prosthesis, orthosis, or prosthetic limb due to incorrect (or erroneous) transition triggers.

[0007] In the context of this invention, the term "orthosis" is used to refer to a device used to assist a person with a limb disorder, as well as a device to enhance the physical function of an able-bodied wearer. The latter is also referred to as an "exoskeleton." Therefore, when an orthosis is mentioned in the context of this application, it can refer to an orthosis used to assist a person with a limb disorder, as well as an exoskeleton.

[0008] Furthermore, in the context of this invention, uncertainty should be understood as the range of error within a confidence interval, typically calculated using the standard error of a point estimate, which is a measure of the variability of the sampled distribution of that estimate. Uncertainty is a concept describing the level of confidence or doubt people have about a particular event or measurement result. In this invention, the state determination device can use a model to output a state associated with the movement of a prosthesis, orthosis, or artificial limb. The uncertainty value is related to the model output regarding the state.

[0009] On the other hand, a probability distribution is a mathematical function that describes the probability of different outcomes in a stochastic process. For example, in a specific embodiment concerning leg prostheses, the model used may attempt to approximate a probability function of gait events (defined as a series of continuous or discrete events). Since the model used may only have data from a sample of amputees rather than the entire population, the estimate of the probability function will contain a degree of uncertainty.

[0010] According to a preferred embodiment, the system further includes a control device configured to control the prosthesis, orthosis, or artificial limb based on the determined state and the acquired uncertainty value.

[0011] By cognizing the state associated with movement and modulating that cognition incorporating uncertainty, predictions of the next movement are improved, and components of prostheses, orthotics, or artificial limbs can be adjusted to follow that prediction.

[0012] According to embodiments, the determination of the state associated with motion can be achieved by determining the current event of the motion or by determining the activity associated with the motion. More specifically, according to exemplary embodiments, the determination of the state associated with motion includes: determining an activity within an activity list and / or determining a motion event within a motion event list. Additionally or alternatively, the determination of the current event of the activity and / or the motion can be performed from a continuum of activities and / or a continuum of motion events, rather than from a list of discrete activities or a list of discrete motion events.

[0013] The following explanations relating to movement in the gait cycle are provided to provide a clearer understanding of the events and activities associated with gait movement. However, those skilled in the art will understand that these explanations are merely illustrative, and the invention can be similarly implemented when considering other types of body movement.

[0014] Gait cycles describe the periodic movement patterns that occur during walking. A single gait cycle begins with the heel of one foot striking the ground and ends with the same heel striking the ground again. Gait cycles can be divided into two main gait phases: the stance phase and the swing phase. The stance phase is the stage in the gait cycle where the foot touches the ground and bears the body weight. More specifically, it can be described as the period from heel strike (heel strike) to toe liftoff. The swing phase is the second phase of the gait, during which the foot moves freely forward. It is described as the period between toe liftoff and heel strike. Furthermore, each of these phases has sub-phases.

[0015] The support phase consists of three sub-phases: the initial load-bearing phase, the middle standing phase, and the push-off phase. The oscillation phase also consists of three sub-phases: the early oscillation phase, the middle oscillation phase, and the final oscillation phase.

[0016] Please note that the above definition of gait periodicity during walking is for illustrative purposes only. Those skilled in the art will understand that different gait patterns can be defined for different types of activities (e.g., climbing / descending stairs, cycling, going up / down hills, etc.). It should also be noted that some gait patterns do not exhibit temporal periodicity.

[0017] Therefore, in embodiments, force analysis during motion (e.g., during gait) can be performed based on sensing data, and determining the current event of motion can include determining the main phase of motion and / or determining the sub-phase of motion within a list of main phases and / or sub-phases of motion.

[0018] Additionally or alternatively, motion activity analysis, such as gait activity analysis, can be performed based on the sensing data, and motion-related activities can be determined based on the motion activity analysis.

[0019] Here, "activity" refers to the type of movement achieved by or through a user's prosthesis, orthosis, or artificial limb. Activity can be defined based on the trajectory of the movement (e.g., gait pattern), the duration of the movement, and / or based on various data sensed during the execution of the movement. Activity can also be associated with activity level, i.e., how active the user is. Activity level can be determined, for example, based on sensed movement. For example, regarding gait movement, the list of activities includes any of the following: standing, jumping, walking (at different speed levels), running (at different speed levels), sitting, driving, using stairs, walking on a slope (uphill or downhill), cycling, lying down, etc.

[0020] Preferably, the state determination device uses one model for each activity.

[0021] According to an exemplary embodiment, the state determination apparatus is further configured to determine a plurality of state candidates, each of which is determined using a different state determination model. Uncertainty values ​​are obtained based on the plurality of state candidates.

[0022] A state determination device can use multiple models—instead of just one—to determine multiple state candidates. Each of these models can be trained differently and can have different performance and accuracy depending on the given state. By considering multiple state candidates, the state associated with the motion can be predicted more accurately.

[0023] According to a preferred embodiment, the status determination device is integrated or attached to a prosthesis, orthosis, or artificial limb, and / or included in a wearable or mobile device.

[0024] According to a particular embodiment, at least one sensing device is integrated or attached to a prosthesis, orthosis, or artificial limb, and / or included in a wearable or mobile device.

[0025] According to an exemplary embodiment, the state determination device is also configured to estimate a motion quality value of a movement performed by the user, the motion quality value being based on optimal biomechanical standards. In this way, abnormal or suboptimal body movements can be detected, potentially enabling the early detection of pathologies.

[0026] Taking gait cycles as an example, it's important to note that walking requires the healthy functioning of several bodily systems, including the musculoskeletal, nervous, cardiovascular, and respiratory systems. Loss of healthy gait function can lead to falls, injuries, loss of mobility and freedom of movement, and a significant decline in quality of life. Therefore, estimating quality of movement is crucial for improving the lives of users wearing or equipped with prostheses, orthotics, or artificial limbs.

[0027] Sensory data related to gait movement can be used for gait analysis to assess spatial, temporal, and sequential variables of gait movement. These variables can include limb movements and positions, joint angles, trajectories, velocities, forces generated, and muscle activity at specific body segments during various phases of the gait cycle. Kinematic and biomechanical equations can then be calculated to determine deviations from known standards and to establish gait patterns for users wearing prostheses, orthotics, or equipped with artificial limbs. It is known that each individual has a unique gait pattern. This can depend on multiple individual variables such as age, height, weight, sex, walking speed, strength, flexibility, type of surgery, amputation shape, and aerobic capacity. Gait patterns can be assessed through gait analysis.

[0028] Alterations in normal gait can be caused by various deformities, injuries, weaknesses, diseases, or pain in any part of the body. For example, loss of dorsiflexion may indicate nerve root compression, common peroneal nerve compression, stroke, or neurological disorders such as multiple sclerosis. Deviations from normal gait can be detected by estimating the quality of movement performed by the subject; high quality of movement is associated with normal gait.

[0029] By sensing data related to movements performed by a user involving a prosthesis, orthosis, or artificial limb worn or equipped by the user, relevant data can be collected and used to improve the use of the prosthesis, orthosis, or artificial limb. The sensed data can be raw data acquired directly (e.g., via at least one sensing device), preprocessed data based on the directly acquired data, or post-processed data.

[0030] Another type of data related to movement and involving prostheses, orthoses, or artificial limbs can be, for example: logs of control data used during the use of prostheses, orthoses, or artificial limbs; inputs to the processor of the prosthesis, orthose, or artificial limb; physical characteristics of the different components that make up the prosthesis, orthose, or artificial limb; algorithms used to operate the prosthesis, orthose, or artificial limb; software data for the processor; user data (feedback) related to sensory performance; and structural data of the model's input.

[0031] Additional sensing data may originate from sensing devices that may be included in the prosthesis, orthosis, or artificial limb, relating to the internal operation of the prosthesis, orthosis, or artificial limb. Environmental data relating to the environment of the prosthesis, orthosis, or artificial limb may be sensing data from sensing devices included in the prosthesis, orthosis, or artificial limb, relating to the environment of the prosthesis, orthosis, or artificial limb. Environment-related data may relate to portions of the environment in which the user wearing the orthosis, prosthesis, or artificial limb is located, or may relate to portions of the wearer or subject (i.e., the user), such as data about a part of the wearer's or subject's body. In one embodiment, environment-related data may relate to signals from neural sensors in the wearer of the implanted prosthesis, orthosis, or artificial limb. Such signals may include movement commands for the prosthesis, orthosis, or artificial limb.

[0032] The sensing data can be stored in a data storage device connected to the prosthesis, orthosis, or artificial limb.

[0033] Then, at least a portion of the sensing data can be transmitted to a remote server. Alternatively or additionally, preprocessed and / or post-processed data based on the sensing data can be transmitted to the remote server. This data can be processed by the processing device of the prosthesis, orthosis, or artificial limb and / or by a processing device connected to a communication module of the prosthesis, orthosis, or artificial limb. By transmitting the data to the remote server, the data can be accumulated at the remote server level and analyzed for various purposes.

[0034] A communication module can be used to transmit data to a remote server. The communication module can be part of a prosthesis, orthosis, or artificial limb; it can be a standalone unit; and / or it can be part of another device separate from the prosthesis, orthosis, or artificial limb. The communication module can interface with a data storage device connected to the prosthesis, orthosis, or artificial limb. The interface can be wireless or wired. The communication module and the remote server can communicate wirelessly or via a wired connection.

[0035] It is important to note that data processing stored in a data storage device can be performed immediately after data storage or at a later time. Similarly, data transfer to a remote server can be performed immediately after data storage, after the stored data has been processed, or at a later time.

[0036] In the context of this invention, the term "remote server" may refer to a single server or a group of servers, such as a distributed server group, like an edge cloud.

[0037] According to a preferred embodiment, the method further includes using a communication module to receive configuration and / or behavioral data from a remote server. The configuration and / or behavioral data can be obtained based on the transmitted data.

[0038] In this way, configuration and / or behavioral data can be transmitted to prostheses, orthoses, or artificial limbs without human intervention by the operator. Configuration and / or behavioral data influence how the prosthesis, orthose, or artificial limb operates and / or behaves. Therefore, configuration and / or behavioral data can be optimized for each individual based on data transmitted to a remote server. This can further improve the time efficiency of maintenance or upkeep. Configuration data can include control data for the operation of the prosthesis, orthose, or artificial limb. Typically, configuration data controls the internal operation of the prosthesis, orthose, or artificial limb, and the configuration data can be updated by a remote server based on internal information received from the prosthesis, orthose, or artificial limb (e.g., errors, logs, etc.). Behavioral data can include parameters used by the processor during the operation of the prosthesis, orthose, or artificial limb. Typically, behavioral data will determine the external operation of the prosthesis or orthose, i.e., how the prosthesis, orthose, or artificial limb behaves relative to the actions or activities being performed by the user or in response to environmental characteristics (e.g., ground type, stairs, slope, information about the person wearing the prosthesis or orthose or equipped with an artificial limb, or their body parts, etc.). Behavioral data may include software code, such as control models that control the operation of forces, resistances, or motion-generating devices that influence the mechanical behavior and / or movement of components in orthotics, prostheses, or artificial limbs.

[0039] By being able to remotely change configuration and / or behavioral data, configuration and / or behavioral data can be updated in an improved manner.

[0040] According to an exemplary embodiment, the orthosis, prosthesis, or artificial limb includes: a force, resistance, or motion generating device (e.g., an actuator or damper) for influencing the mechanical behavior and / or motion of elements of the orthosis, prosthesis, or artificial limb; and a control module (or control device) configured to control the force, resistance, or motion generating device according to a control agent. Configuration and / or behavioral data received from a remote server can be used to modify the control agent.

[0041] In this way, the operation of a prosthesis, orthosis, or artificial limb can be remotely altered using data transmission. The actuator can be pneumatic, piezoelectric, electric, hydraulic, magnetic, or mechanical. In some embodiments, the changes can be made "online," such as as soon as the data becomes available, while in other embodiments, the changes can be made "offline," such as when the data is stored before use. Furthermore, combinations are possible; for example, some changes can be made "online" while others can be made "offline." The damper can be active or passive. In prostheses, orthosis, or artificial limbs that include a control module and force, resistance, or motion-generating devices (configured to influence the mechanical behavior and / or movement of the elements of the prosthesis, orthosis, or artificial limb), different parameters can be considered for quantifying the mechanical effects.

[0042] A control agent can receive one or more inputs and can use one or more weight parameters to generate one or more outputs based on the inputs. The control agent can define control strategies, such as those known in the field of artificial intelligence. For example, one or more inputs may include: the outputs of one or more sensing devices; and / or processed data based on the outputs of one or more sensing devices; and / or other data inputs.

[0043] The configuration and / or behavioral data sent to the communication module may include the parameters described above related to the mechanical behavior and / or movement of the components of the orthosis, prosthesis, or artificial limb. However, those skilled in the art will understand that the configuration and / or behavioral data is not limited to those parameters and may include data related to electronic components, or data related to electronic components included in the parts involved in the operation of the prosthesis, orthosis, or artificial limb, such as data related to battery management, thermal management, data storage management, clocking, etc. In one embodiment, the configuration and / or behavioral data may define one or more new input sources for the control agent, or the configuration and / or behavioral data may include a new control agent itself.

[0044] According to a preferred embodiment, at least one sensing device includes any one or any combination of the following: a neural sensor, an angle sensing device, an accelerometer, a gyroscope, a Hall sensor, a force sensor, a magnetometer, a pressure sensor, a torque sensor, a temperature sensor, an energy metering device, a current sensor, a voltage sensor, a humidity sensor, a sonar sensor, an electromyography (EMG) sensor, a barometric pressure sensor, a pressure sensor grid, an electroencephalogram (EEG) sensor, a radio frequency identification (RFID) sensor, and a geolocation sensor.

[0045] In this way, various types of data related to the operation and / or environment of prostheses, orthoses, or artificial limbs can be sensed and collected. Depending on the selected sensors, different algorithmic models can be developed for state determination devices that use the sensed data to infer the state of the prosthesis, orthosis, or artificial limb.

[0046] According to an exemplary embodiment, at least one sensing device includes a plurality of sensors. Based on configuration and / or behavioral data, one or more of the plurality of sensors are selected to provide input to a control agent.

[0047] In this way, the most suitable sensors or sensor sets can be selected to improve the mechanical behavior and / or movement of prostheses, orthotics, or artificial limbs via force, resistance, or motion generation devices. Therefore, different sensor sets can be used as inputs to a control model based on configuration and / or behavioral data. Additionally or alternatively, different sensor sets can be used as inputs to a control agent based on environmental data. Furthermore, sensing data from different sensor sets can be used as inputs to a state determination device based on environmental data.

[0048] Those skilled in the art will understand that the above-described technical considerations and advantages relating to embodiments of systems including prostheses, orthotics, or artificial limbs are also applicable, with appropriate modifications, to the corresponding control method embodiments described below.

[0049] According to a second aspect of the invention, a method is provided for controlling a prosthesis, orthosis, or artificial limb in a system, the system comprising: a prosthesis, orthosis, or artificial limb, at least one sensing device, and a state determination device. The method includes repeatedly performing the following steps: - Acquire sensing data from at least one sensing device relating to movements performed by a user involving a prosthesis, orthosis, or artificial limb worn by the user; - The state determination device determines the state associated with motion based on sensing data; - The uncertainty value representing the level of uncertainty of the user's state is obtained by the state determination device.

[0050] Those skilled in the art will understand that the above-described technical considerations and advantages relating to embodiments of methods for controlling prostheses, orthotics, or artificial limbs are also applicable, with appropriate modifications, to the corresponding self-developing control agent method embodiments described below.

[0051] According to a third aspect of the invention, a self-developing method for a control agent for a prosthesis, orthosis, or artificial limb is provided. The control agent is configured to be used by a control device of the prosthesis, orthosis, or artificial limb to control the prosthesis, orthosis, or artificial limb. The method includes the following steps: - Obtain a motion model of a prosthesis, orthosis, or artificial limb, the motion model being defined in terms of the mechanical relationships between the components of the prosthesis, orthosis, or artificial limb; - Obtain a training response model for a prosthesis, orthosis, or artificial limb, wherein the training response model defines a predefined motion response for the prosthesis, orthosis, or artificial limb; - Determine the current movement status of the prosthesis, orthosis, or artificial limb; and Repeat execution: - The control agent determines candidate mechanical actions based on the current motion state, and the mechanical actions are related to elements of a prosthesis, orthosis or artificial limb; - The motion model outputs the next candidate motion state based on the current motion state and candidate mechanical actions; - The trained response model outputs the next predefined motion state based on the current motion state; - Obtain a reward value based on the next candidate motion state and the next predefined motion state, wherein the reward value is associated with the current motion state and the candidate mechanical action; - Set the next candidate motion state as the current motion state.

[0052] Here, the motion model is a virtual model designed to mechanically reproduce the prosthesis, orthosis, or artificial limb from the sensor's perspective and taking into account the controllability of the prosthesis, orthosis, or artificial limb. In one embodiment, the motion model can be implemented using a neural network.

[0053] Using this virtual representation of a prosthesis, orthosis, or artificial limb, the predefined motion response is the desired mechanical output of the prosthesis, orthosis, or artificial limb using sensor data as input. The desired mechanical output is based on multiple natural equivalent responses from a user wearing or equipped with the prosthesis, orthosis, or artificial limb. The set of natural equivalent responses from subjects wearing or equipped with the prosthesis, orthosis, or artificial limb is collected in the training response model. The training response model can be obtained based on, for example, gait analysis of the user's healthy limb, using able-bodied subjects, or using kinematic simulations based on the physical characteristics of the user's body.

[0054] Complex relationships can be established between the input sensed data and the desired mechanical output. This collection of complex relationships is gathered in a control agent, which can also be a model. Starting from the initial control agent, its development pursues two objectives. The first objective is to compare multiple candidate mechanical actions and their outcomes with the trained response model through multiple iterations, using reward values ​​to evaluate its performance. The second objective is to use the reward values ​​accumulated through iterations to make the multiple candidate motion states obtained from the multiple candidate mechanical actions more closely approximate the corresponding parts of the trained response model.

[0055] To achieve the first objective, starting from the current motion state, the control agent determines the candidate mechanical action to follow in order to reach the next logical motion state from the control agent's perspective. This next logical motion state is referred to herein as the next candidate motion state. The next candidate motion state is obtained by considering the candidate mechanical action and its impact on the prosthesis, orthosis, or artificial limb due to the motion model. However, the next logical motion state may differ from the development of motion states as defined in the training response model. Comparing the two, a reward value is assigned to the pair consisting of the current motion state and the candidate mechanical action. The above steps are then repeated starting from the newly reached motion state, thus describing a series of motion states coupled with mechanical actions and associated with reward values.

[0056] In this way, a strategy is developed that will help guide the development of a control agent, such that a series of mechanical actions induce a series of motion states that mimic the corresponding series of motion states from the training response model.

[0057] More specifically, to check whether the modifications to the control agent are proceeding in the right direction, a reward-based strategy is used. This strategy is a method of training artificial intelligence (AI) to match a given expectation, known as reinforcement learning. The reward value can be provided by a simple algorithm that can score any output. Here, the score (i.e., the reward value) can be calculated by comparing the difference between the trained response model and a series of motion states obtained using the control agent. The reward values ​​obtained through iterative iterations can be input into a cumulative reward function over time. A score derived from the cumulative reward function can be calculated, and if the score exceeds a predetermined threshold, the control agent is modified to reinforce a series of candidate mechanical actions determined during the iterations; otherwise, a previous version of the modified control agent is used. According to an embodiment, the score can correspond to the final value of the cumulative reward function or to the integral value of the cumulative reward function. The iterations of modification and reward value allocation are repeated until motion corresponding to the trained response model can be obtained using the control agent and the motion model.

[0058] Additionally or alternatively, multiple control agents can be developed and compared based on their respective cumulative reward functions. Furthermore, if the cumulative reward function associated with one of the multiple control agents is superior to that of the others (based on a comparison of scores derived from the multiple cumulative reward functions), the control agent can be modified to reinforce a set of candidate mechanical actions determined during the iteration process.

[0059] In this way, AI is used to develop control agents for prostheses, orthotics, or artificial limbs to more closely correspond to natural body movements. According to embodiments, the control agent can be developed to correspond to a single user activity, or it can be developed to be sufficient for a continuum of movement states and activities.

[0060] According to an exemplary embodiment, the method further includes the following steps: -After the predetermined conditions are met, a cumulative reward function is determined based on one or more reward values ​​obtained; - Modify the control agent based on the score derived from the cumulative reward function.

[0061] Iteration can continue until a predetermined condition is reached. In one embodiment, the predetermined condition may be a predetermined number of iterations. In another embodiment, the predetermined condition may be the convergence of a cumulative reward function based on one or more reward values ​​as a function of the number of iterations. In yet another embodiment, the predetermined condition may be continuously obtaining reward values ​​below a predetermined threshold.

[0062] Preferably, the predetermined conditions include: a predetermined number of iterations for the repeated steps; or, a predetermined threshold for the convergence characteristics of the cumulative reward function.

[0063] Based on the cumulative reward function, the control agent can then be modified. If the score derived from the cumulative reward function is below a first score threshold, the first candidate mechanical action can be marked as a "dead end," and the control agent can restart multiple iterations from the same initial current motion state. If the score derived from the cumulative reward function is above the first or second score threshold, certain parameters of the control agent can be enhanced to make it adopt mechanical actions similar to the candidate mechanical actions when used with a control device for a prosthesis, orthosis, or artificial limb.

[0064] Furthermore, modifications to the control agent can include modifying the reward scoring algorithm used to obtain reward values ​​during one or more iterations of a repeated step. In this way, the reward scoring algorithm can be used to bias reinforcement learning, favoring certain candidate mechanical actions over others.

[0065] According to a preferred embodiment, the modification of the control agent is also based on user reward input from a user wearing or equipped with a prosthesis, orthosis, or artificial limb, which is associated with the performance of the modified control agent used by the control device of the prosthesis, orthosis, or artificial limb when controlling the worn or equipped prosthesis, orthosis, or artificial limb.

[0066] In this way, additional input sources can be considered when developing the control agent. To obtain this input, the developing control agent can be implemented in a real prosthesis, orthosis, or artificial limb corresponding to the virtually reproduced prosthesis, orthosis, or artificial limb, and the user can provide a rating based on their use of the prosthesis, orthosis, or artificial limb controlled by the developing control agent. User reward input is calculated based on this rating and is used to supplement or replace the calculations performed by the reward scoring algorithm to obtain a reward value.

[0067] According to an exemplary embodiment, the modification of the control agent is also based on extracted reward inputs associated with the performance of the modified control agent used by the control device of the prosthesis, orthosis, or artificial limb when controlling the worn or equipped prosthesis, orthosis, or artificial limb. The extracted reward inputs may be based on measurements extracted from one or more evaluation sensing devices during the user's use of the prosthesis, orthosis, or artificial limb. The one or more evaluation sensing devices may be sensing devices included in the prosthesis, orthosis, or artificial limb, or sensing devices external to the prosthesis, orthosis, or artificial limb. The extracted measurements may be related to one or more of biomechanical characteristics, physical functional characteristics, physiological characteristics, and / or psychological characteristics. Psychological characteristics may be obtained based on questionnaires, perceived fatigue ratings, and / or visual analog scales.

[0068] In specific cases involving leg-related prostheses, orthoses, or artificial limbs, the extracted measurements may be associated with one or more of the following: symmetry, cadence, center of motion, center of pressure, bipedal bearing time, ground reaction force, ankle power, hip power, knee power, ankle range of motion, hip range of motion, knee range of motion, trunk range of motion, upper limb range of motion, single-leg bearing time, bearing time, stride length, stride width, stride length, stride duration, swing time, ankle torque, hip torque, knee torque, distance, duration, hill walking assessment index, reaction time, speed, stair walking assessment index, respiratory rate, carbon dioxide production, electroencephalogram (EEG), electromyography (EMG), heart rate, metabolic rate, oxygen consumption, oxygen uptake cost, respiratory exchange rate, skin conductance, and ventilation equivalent.

[0069] According to a particular embodiment, the current motion state of a prosthesis, orthosis, or artificial limb is defined based on one or more sensing data from a list of available sensing data during the use of the prosthesis, orthosis, or artificial limb.

[0070] According to an exemplary embodiment, the motion model, the trained response model, the control agent, and the trained response model are associated with activities performed by the user of the prosthesis, orthosis, or artificial limb.

[0071] Those skilled in the art will understand that the above-mentioned technical considerations and advantages related to the gait transition determination method embodiments during the use of prostheses, orthoses or artificial limbs, or the control method embodiments for controlling prostheses, orthoses or artificial limbs, and the self-development method embodiments for control models, are also applicable to the corresponding development method embodiments described below, with appropriate modifications.

[0072] According to a fourth aspect of the invention, a method is provided for developing multiple control models for prostheses, orthotics, or artificial limbs. The method includes the following steps: - Obtain a motion model of a prosthesis, orthosis, or artificial limb, the motion model being defined in terms of the mechanical relationships between the components of the prosthesis, orthosis, or artificial limb; - Obtain multiple training response models for prostheses, orthoses, or artificial limbs, each of which defines a predefined motion response for the prosthesis, orthose, or artificial limb and is associated with biomechanically different users wearing or equipped with prostheses, orthoses, or artificial limbs. - Obtain an initial plurality of control agents, preferably similar control agents, each of which is associated with a different training response model among a plurality of training response models and is configured to be used by a control device of a prosthesis, orthosis or artificial limb to control the worn or equipped prosthesis, orthosis or artificial limb. - Select a control agent from the initial pool of control agents and define the selected control agent as the preferred control agent; and Repeat execution: - Select a challenger control agent from among multiple control agents, which is different from the preferred control agent and preferably was not selected in previous iterations; - Obtain user input indicating user preferences in the challenger control agent and preferred control agent relative to the performance of the challenger control agent and preferred control agent during the process of controlling the prosthesis, orthosis or artificial limb being worn or equipped; - Based on user input, select a new preferred control agent between the previous preferred control agent and the challenger control agent.

[0073] In this way, multiple control agents can be scored in competition with each other, so that through one or more iterations of the above repeated steps, one or more control agents that are most suitable for the user (scored according to preferences) are selected.

[0074] Those skilled in the art will understand that the initial multiple control agents can be obtained by any known method. Alternatively or additionally, the initial multiple control agents can be obtained by the aforementioned self-evolution method.

[0075] By using multiple control agents (each associated with a different training response model from multiple training response models), this competition allows for the determination of which one or more of the initial multiple control agents are closer to the user's desired custom control agent for performing scoring. In other words, one or more better starting points are selected through the method described above for further developing the desired custom control agent.

[0076] According to a preferred embodiment, the method further includes the following step: determining the final preferred control agent after a predetermined condition is met.

[0077] Preferably, the predetermined conditions include selecting a predetermined number of times the same preferred control agent is used during the iteration of the repeated steps.

[0078] According to an exemplary embodiment, the method further includes the following steps: - Select the final preferred control agent as the first control agent; and The method also includes repeating the following steps: -Based on the first control agent, preferably by replicating the first control agent, a second control agent is generated; - A second control agent is trained based on a motion model of a prosthesis, orthosis, or artificial limb and a training response model associated with a first control agent, wherein the training is preferably performed within a predetermined number of training epochs; - Obtain further user input indicating further user preferences in the first and second control agents relative to the performance of the first and second control agents during the process of controlling the prosthesis, orthosis, or artificial limb being worn or equipped. - Based on further user input, select a new first control agent between the previous first control agent and second control agent.

[0079] In this way, the final preferred control agent can be further developed iteratively, with each generation of training using user input to obtain a new first control agent that is closer to the desired customized control agent.

[0080] According to a preferred embodiment, the initial acquisition of multiple control agents is based on one or more sensing data from a list of available sensing data during the use of a prosthesis, orthotics, or artificial limb.

[0081] Those skilled in the art will understand that the above-mentioned technical considerations and advantages related to the embodiments of the development methods of multiple control models, as well as the embodiments of the self-development methods of control models, are also applicable to the corresponding system embodiments described below with appropriate modifications.

[0082] According to a fifth aspect, a system including a prosthesis, orthosis, or artificial limb is provided. The system further includes: at least one sensing device configured to sense data related to movements performed by a user involving a prosthesis, orthosis, or artificial limb worn or equipped by the user; and a control device configured to use a control agent to control elements of the prosthesis, orthosis, or artificial limb. The control agent used is either a modified control agent developed according to the method of the third aspect, or a selected new preferred control agent developed according to the method of the fourth aspect.

[0083] According to another aspect of the present invention, a computer program comprising computer-executable instructions is provided for performing any of the methods described above when the program is run on a computer or according to any step of any of the above embodiments.

[0084] According to another aspect of the invention, a computer device or other hardware device is provided, which is programmed to perform one or more steps of any embodiment of the above-described methods. According to another aspect, a data storage device is provided, which encodes a program in a machine-readable and machine-executable form to perform one or more steps of any embodiment of the various methods described above. Attached Figure Description

[0085] This and other aspects of the invention will now be described in more detail with reference to the accompanying drawings, which illustrate the presently preferred embodiments. Throughout the drawings, the same numerals refer to the same features.

[0086] Figure 1 A schematic diagram illustrating an exemplary embodiment of a system including a prosthesis, orthosis, or artificial limb that communicates with assistive devices is shown. Figure 2 A flowchart illustrating an exemplary embodiment of a method for controlling a prosthesis, orthosis, or artificial limb in a system including a prosthesis, orthosis, or artificial limb is shown. Figure 3 A flowchart depicts an exemplary embodiment of a method for the self-evolving control agent for prostheses, orthotics, or artificial limbs; Figure 4A flowchart depicts an exemplary embodiment of a method for developing a preferred control agent for prostheses, orthotics, or artificial limbs; Figure 5 A flowchart depicts an exemplary embodiment of a method for further development of a final preferred control agent for prostheses, orthotics, or artificial limbs. Detailed Implementation

[0087] Figure 1 A schematic diagram illustrating an exemplary embodiment of a system according to the present invention comprising a prosthesis, orthosis, or artificial limb communicating with an assistive device is shown. The prosthesis 1 and orthosis 2 may be configured to functionally assist, enhance, and / or replace a limb of a human or animal subject, or to enhance the body or part of the body of a human or animal subject. The artificial limb 3 may be configured to functionally serve as a limb of a humanoid robot or animal-like robot. The term "orthosis" is used to refer to a device for assisting a person with a limb disorder, as well as a device for enhancing the physical function of an able-bodied wearer. Thus, when referring to an orthosis, it can be an orthosis for assisting a person with a limb disorder, as well as an exoskeleton. The prosthesis 1, orthosis 2, or artificial limb 3 includes at least one sensing device and a state determination device. At least one sensing device is configured to sense data related to movement performed by a user involving a prosthesis, orthosis, or artificial limb worn or equipped by the user. The state determination device is configured to repeatedly perform the following steps: determining a state associated with the movement based on sensing data acquired from at least one sensing device; and acquiring an uncertainty value representing the level of uncertainty of the user in the determined state.

[0088] The prosthesis 1, orthosis 2, or artificial limb 3 may include a data storage device and a processing device. A status determination device may be included in the processing device. A communication module connected to the data storage device of the prosthesis 1, orthosis 2, or artificial limb 3 may communicate with the assistive device 4. According to an embodiment, one or more of the prosthesis 1, orthosis 2, or artificial limb 3 may communicate with the assistive device 4. The assistive device 4 may be a remote server or a mobile device.

[0089] The prosthesis 1, orthosis 2, or artificial limb 3 may include force, resistance, or motion generating devices, such as actuators and / or dampers, for influencing the mechanical behavior and / or motion of elements of the prosthesis 1, orthosis, or artificial limb. The prosthesis 1, orthosis 2, or artificial limb 3 may also include a control device. In a prosthesis 1, orthosis 2, or artificial limb 3 including a control device, the control device may be configured to control such force, resistance, or motion generating devices, and different parameters may be considered to achieve this control. For example, parameters may be used as weights for different considered inputs (e.g., sensed data) in a feedback loop. The force, resistance, or motion generating device, such as actuators and / or dampers, can then be controlled actively or passively based on the feedback loop.

[0090] The data storage device can be configured to store data related to the operation of the prosthesis 1, orthosis 2, or artificial limb 3, and / or environmental factors related to the prosthesis 1, orthosis 2, or artificial limb 3. The communication module may include a communication interface configured to transmit at least a portion of the stored data and / or processed data based thereon to the assistive device 4. The assistive device 4 itself may be directly or indirectly (preferably via an encrypted connection) connected to the cloud network 5. At least a portion of the stored data and / or processed data transmitted to the assistive device 4 may be stored in a secure database 6 for further processing.

[0091] Data from the secure database 6 can be used to determine and / or develop different models 7, 8. Other types of data, such as firmware data or mobile client data, can also be obtained based on the data from the secure database 6. For example, configuration and / or behavioral data based on data from the secure database 6 can be obtained and sent back to the assistive device 4 via the cloud network 5. The communication module of the prosthesis 1, orthosis 2, or artificial limb 3 can then be configured to receive the configuration and / or behavioral data from the assistive device 4.

[0092] The prosthesis 1, orthosis 2, or artificial limb 3 may include: force, resistance, or motion generation devices (e.g., actuators or dampers) for influencing the mechanical behavior and / or motion of elements of the prosthesis 1, orthosis 2, or artificial limb 3; and a control device configured to control the force, resistance, or motion generation devices based on an adaptive model 8 (or control agent) and / or a detection model 7 (or state determination model). It should be noted that the control device does not need to be a separate module; it can be integrated with the processing device of the prosthesis 1, orthosis 2, or artificial limb 3. Configuration and / or behavioral data received from the assistive device 4 can be used to adapt the firmware and / or model used by the control device.

[0093] In one embodiment, configuration and / or behavioral data are related to the operation of force, resistance, or motion-generating devices (e.g., actuators or dampers) of the prosthesis 1, orthosis 2, or prosthetic limb 3. The actuator or damper can influence the movement of components of the prosthesis, orthosis, or prosthetic limb. During use, the action of the actuator or damper can be defined based on a set of parameters that influence the feedback loop used to control the actuator or damper. The configuration and / or behavioral data may include new values ​​for at least a portion of this set of parameters to obtain more natural movement of the prosthesis 1, orthosis 2, or prosthetic limb 3. Typically, the behavioral data will determine the external operation of the prosthesis 1, orthosis 2, or prosthetic limb 3, i.e., how the prosthesis 1, orthosis 2, or prosthetic limb 3 behaves relative to the actions or activities being performed by the user or in response to environmental characteristics (e.g., ground type, stairs, slope, information about the person wearing the prosthesis or orthosis or equipped with an artificial limb). Behavioral data may include software code, such as control agents for controlling the operation of forces, resistances, or motion-generating devices that influence the mechanical behavior and / or movement of components of the orthosis 1, prosthesis 2, or artificial limb 3.

[0094] The prosthesis 1, orthosis 2, or artificial limb 3 includes at least one sensing device for acquiring sensing data related to the operation of the prosthesis 1, orthosis 2, or artificial limb 3 and / or related to the environment of the prosthesis 1, orthosis 2, or artificial limb 3. A data storage device may store the sensing data and / or processed data based on the sensing data.

[0095] The control device can be configured to control a force, resistance, or motion adjustment device based on sensing data from at least one sensing device by a control agent. Typically, the at least one sensing device may include multiple sensors, and the control device can be configured to select one or more of the multiple sensors as inputs to the control agent based on configuration and / or behavioral data.

[0096] At least one sensing device may include a first set of sensors and a second set of sensors; the processing device may be configured to determine first processed data based on sensing data from the first set of sensors, and / or determine second processed data based on sensing data from the second set of sensors, according to the activity pattern of the prosthesis 1, orthosis 2, or artificial limb 3 in use. In this way, data stored in a data storage device (and optionally transmitted to the assistive device 4) may be adapted to the current activity pattern. For example, depending on whether the individual is standing, walking, or running, it may be desirable to sense different data and / or process the sensed data in different ways.

[0097] Preferably, the communication module is connected to the prosthesis 1, orthosis 2, or artificial limb 3 via a wired connection.

[0098] The control agent can receive one or more inputs and can use one or more weight parameters to generate one or more outputs based on the inputs. The control agent can define control strategies, such as those known in the field of artificial intelligence. For example, one or more inputs may include: the outputs of one or more sensors of at least one sensing device; and / or processed data based on the outputs of one or more sensors; and / or other data inputs. For example, the sensing data can also be processed according to a detection algorithm to determine one or more operational categories (e.g., activity patterns) of the prosthesis, such as whether a person is walking on flat ground or on a slope, whether a person is standing, walking, or running, etc. These one or more operational categories can then be input into the control agent. The control agent can use one or more weight parameters associated with different inputs. One or more outputs of the control agent allow the control device to control the mechanical effects on force, resistance, or motion modulation devices. The weight parameters can be used for adaptive control and can define different sets of weights based on the activity patterns of prosthesis 1, orthosis 2, or prosthetic limb 3 and based on the motion achieved by prosthesis 1, orthosis 2, or prosthetic limb 3.

[0099] Optionally, the control device may also output data to be stored in a data storage device for transmission to the auxiliary device 4.

[0100] In one embodiment, configuration and / or behavioral data may be defined as: one or more new input sources for the control agent; and / or the new control agent itself; and / or one or more parameters used by the control agent.

[0101] Figure 2 A flowchart illustrating an exemplary embodiment of a method according to the present invention for controlling a prosthesis, orthosis, or artificial limb in a system including a prosthesis, orthosis, or artificial limb is provided. The system including the prosthesis, orthosis, or artificial limb can be similar to the method described above. Figure 1 The system described.

[0102] In the first step S11, the method includes acquiring sensing data related to movements performed by a user involving a prosthesis, orthosis, or artificial limb worn or equipped by the user. The sensing data may be: raw data acquired directly (e.g., from at least one sensing device); preprocessed data based on the directly acquired data; or post-processed data. At least one sensing device may be integrated or attached to the prosthesis, orthosis, or artificial limb, and / or included in a wearable or mobile device.

[0103] At least one sensing device may include any one or any combination of the following: neural sensor, angle sensing device, accelerometer, gyroscope, Hall sensor, force sensor, magnetometer, pressure sensor, torque sensor, temperature sensor, energy metering device, current sensor, voltage sensor, humidity sensor, sonar sensor, electromyography (EMG) sensor, barometric pressure sensor, pressure sensor grid, electroencephalography (EEG) sensor, radio frequency identification (RFID) sensor, geolocation sensor.

[0104] In addition to the acquired sensing data, other types of data can also be acquired. Another type of data related to motion and involving prostheses, orthoses, or artificial limbs may include, for example, logs of control data used during the use of the prosthesis, orthose, or artificial limb; inputs to the processor of the prosthesis, orthose, or artificial limb; physical characteristics of the different components that make up the prosthesis, orthose, or artificial limb; algorithms used to operate the prosthesis, orthose, or artificial limb; software data used to process the device; user data (feedback) related to sensing performance; and structural data of the model's input.

[0105] According to embodiments, sensing data may originate from sensing devices that may be included in a prosthesis, orthosis, or artificial limb, the sensing devices relating to the internal operation of the prosthesis, orthosis, or artificial limb. Environmental data relating to the environment of the prosthesis, orthosis, or artificial limb may be sensing data from sensing devices included in the prosthesis, orthosis, or artificial limb, the sensing devices relating to the environment of the prosthesis, orthosis, or artificial limb. Environment-related data may relate to a portion of the environment of a user wearing an orthosis or prosthesis, or a user equipped with an artificial limb, or may relate to a portion of the wearer or subject (i.e., the user), such as data concerning a portion of the wearer's or subject's body. In one embodiment, environment-related data may relate to signals from neural sensors in a wearer of an implanted prosthesis or orthosis, or a subject equipped with an artificial limb. Such signals may include movement commands for the prosthesis, orthosis, or artificial limb.

[0106] The sensed data can be stored in a data storage device connected to the prosthesis, orthosis, or artificial limb. The first step S11 of acquiring the sensed data can preferably be performed during the use of the prosthesis, orthosis, or artificial limb. Supplementary processing data can also be acquired at a later time based on the sensed data.

[0107] In the second step S12, the method includes determining a motion-related state by a state determination device based on sensing data. A trained model can be used by the state determination device to determine motion-related states based on sensing data. The state determination device can be integrated or attached to a prosthesis, orthosis, or artificial limb, and / or included in a wearable or mobile device.

[0108] The states associated with motion can correspond to characteristic segments of the motion, which are characterized by considering one or more of the intensity, trajectory, rhythm, or duration of the motion. Thus, depending on the accuracy, type, and / or quantity of the sensed data, the states associated with motion can correspond to the entire motion performed by the user involving a prosthesis, orthosis, or artificial limb (e.g., a throwing motion using a hand prosthesis), or they can correspond to a more minute part of that motion (e.g., finger release when throwing an object using a hand prosthesis).

[0109] According to embodiments, the determination of a state associated with motion can be achieved by determining a current event of motion or by determining an activity associated with motion. More specifically, according to exemplary embodiments, determining a state associated with motion includes: determining an activity within an activity list and / or determining a motion event within a motion event list. Additionally or alternatively, the determination of an activity and / or a current event of motion can be performed from a continuum of activities and / or a continuum of motion events, rather than from a list of discrete activities or a list of discrete motion events. Preferably, when determining an activity associated with motion, the state determination device uses a model for each activity.

[0110] The term "activity" refers to the type of movement achieved by or through a user's prosthesis, orthosis, or artificial limb. Activities can be defined based on the trajectory of the movement (e.g., gait patterns), the duration of the movement, and / or based on various data sensed during the execution of the movement. Activities can also be associated with activity levels, i.e., how active the user is. Activity levels can be determined, for example, based on sensed movements. For example, regarding gait movements, the list of activities includes any of the following: standing, jumping, walking (at different speed levels), running (at different speed levels), sitting, driving, using stairs, walking on a slope (uphill or downhill), cycling, lying down, etc.

[0111] Additionally or alternatively, motion activity analysis, such as gait activity analysis, can be performed based on the sensing data, and motion-related activities can be determined based on the motion activity analysis.

[0112] In one embodiment, force analysis during motion (e.g., during gait) can be performed based on sensing data, and determining the current event of the motion can include: determining the principal phase of the motion; and / or determining the subphase of the motion within a list of principal phases and / or a list of subphases.

[0113] According to an exemplary embodiment, the state determination device can be further configured to determine a plurality of state candidates, each of which is determined using a different state determination model. As will be explained in more detail below with respect to step S13, the uncertainty value can be obtained based on the plurality of state candidates. Furthermore, the state determination device can use multiple models—instead of just one model—to determine the plurality of state candidates. Each of the plurality of models can be trained differently and can have different performance and accuracy depending on the given state.

[0114] Following step S12, the method includes step S13, in which the state determination device obtains an uncertainty value representing the uncertainty level of the user being in the determined state.

[0115] In the context of this invention, uncertainty should be understood as the range of error within a confidence interval, typically calculated using the standard error of a point estimate, which is a measure of the variability of the sampled distribution of that estimate. Uncertainty is a concept describing the level of confidence or doubt people have about a particular event or measurement result. In one embodiment of the invention, the state determination device can use a model to output a state associated with the movement of a prosthesis, orthosis, or artificial limb. In this case, the uncertainty value can be correlated with the model output regarding the state. Because the model used may only have data from a sample of amputees rather than the entire population, the estimate of the probability function of the state associated with the prosthesis, orthosis, or artificial limb will have a certain degree of uncertainty.

[0116] Then, the above steps S11 to S13 are iterated to continuously obtain the state and related uncertainties associated with movement by using a prosthesis, orthosis or artificial limb.

[0117] Optionally, step S14 can be performed after step S13. For this purpose, the system also includes a control device. In step S14, the method further includes the step of the control device controlling the prosthesis, orthosis, or artificial limb based on the determined state and the acquired uncertainty value.

[0118] More specifically, control over a worn or fitted prosthesis, orthosis, or artificial limb may include any one or more of the following: control of the actuators of the prosthesis, orthosis, or artificial limb by a control device using a control agent; and control of adjustments to the components of the prosthesis, orthosis, or artificial limb.

[0119] Those skilled in the art will understand that steps S11 to S14 can be iterated during the user's use of the prosthesis, orthosis, or artificial limb in order to achieve more accurate control of the prosthesis, orthosis, or artificial limb.

[0120] Optionally, step S15 may be performed after step S13. In step S15, the method further includes estimating a motion quality value of a movement performed by the user using a state determination device, the motion quality value being based on an optimal biomechanical standard. The motion quality value may correspond to a score obtained by comparing the biomechanical standard of a specific movement performed by the user involving a prosthesis, orthosis, or artificial limb with the optimal biomechanical standard for that specific movement.

[0121] More specifically, for clarity, taking the gait cycle as an example, sensory data related to gait movement can be used for gait analysis to assess spatial, temporal, and sequential variables of gait movement. These variables can include limb movements and positions, joint angles, trajectories, velocities, generated forces, and muscle activity at specific body segments during each phase of the gait cycle. Kinematic and biomechanical equations can then be calculated to determine deviations from known standards and to establish gait patterns for users wearing prostheses or orthoses or equipped with artificial limbs. It is known that each individual has a unique gait pattern. This can depend on multiple individual variables such as age, height, weight, sex, walking speed, strength, flexibility, type of surgery, amputation shape, and aerobic capacity. Gait patterns can be assessed through gait analysis. Changes in normal gait can be caused by various deformities, injuries, weakness, diseases, or pain in any part of the body. Deviations from normal gait can be detected by estimating the quality of movement performed by the subject, where high quality of movement is associated with normal gait.

[0122] Those skilled in the art will understand that similar analyses can be performed for other movements or for prostheses, orthoses, or artificial limbs corresponding to other body parts. Those skilled in the art will also understand that steps S11, S12, S13, and S15 can be iterated as the user uses the prosthesis, orthose, or artificial limb.

[0123] Figure 3 A flowchart illustrating an exemplary embodiment of a self-evolving method for a control agent for a prosthesis, orthosis, or artificial limb according to the present invention is provided. The prosthesis, orthosis, or artificial limb may include components similar to those described above. Figure 1 The system described.

[0124] In step S21, the method includes obtaining a motion model of a prosthesis, orthosis, or artificial limb, the motion model defining the mechanical relationships between the components of the prosthesis, orthosis, or artificial limb.

[0125] The term "motion model" refers to a virtual model designed to mechanically reproduce the prosthesis, orthosis, or artificial limb from a sensor perspective and taking into account the controllability of the prosthesis, orthosis, or artificial limb. In one embodiment, the motion model can be implemented using a neural network.

[0126] In step S22, the method includes obtaining a training response model of a prosthesis, orthosis, or artificial limb, the training response model defining a predefined motion response of the prosthesis, orthosis, or artificial limb.

[0127] Preferably, the training response model is associated with a user wearing or equipped with a prosthesis, orthosis, or artificial limb. Additionally or alternatively, the training response model may be associated with activities in an activity list that includes any of the following: walking, running, sitting, driving, using stairs, walking on a slope, cycling, lying down, standing, and jumping.

[0128] By utilizing a virtual representation of a prosthesis, orthosis, or artificial limb through a motion model, a predefined motion response corresponds to a desired mechanical behavior of the prosthesis, orthosis, or artificial limb, which is associated with the mechanical output of the prosthesis, orthosis, or artificial limb using sensor data as input. The desired mechanical output is based on multiple natural equivalent responses from a user wearing or equipped with the prosthesis, orthosis, or artificial limb. The set of natural equivalent responses from subjects wearing or equipped with the prosthesis, orthosis, or artificial limb is collected in a training response model. The training response model can be obtained based on, for example, gait analysis of the user's healthy limb, using able-bodied subjects, or using kinematic simulations based on the physical characteristics of the user's body. Those skilled in the art will understand that training response models for other body parts can be obtained similarly.

[0129] In step S23, the method includes determining the current motion state of the prosthesis, orthosis, or artificial limb. The current motion state can be determined based on a desired starting point for the self-evolution of the control agent. Alternatively, the current motion state can be randomly determined from a list of motion states of interest.

[0130] In step S24, the method includes the step of determining candidate mechanical actions by a control agent based on the current motion state, the mechanical actions being related to elements of a prosthesis, orthosis, or artificial limb.

[0131] Complex relationships can be established between the input sensed data and the desired mechanical output. This collection of complex relationships is gathered in a control agent, which can also be a model. Starting from the initial control agent, its development pursues two objectives. One objective is to compare multiple candidate mechanical actions and their outcomes with the trained response model through multiple iterations, using reward values ​​to evaluate its performance. To achieve this objective, starting from the current motion state, the control agent determines the candidate mechanical action to follow in order to reach the next logical motion state from the control agent's perspective.

[0132] In one embodiment, the elements of a prosthesis, orthosis, or artificial limb include an actuator, and determining candidate mechanical actions includes determining the mechanical action of the actuator.

[0133] In step S25, the method includes the step of outputting the next candidate motion state by the motion model based on the current motion state and the candidate mechanical action.

[0134] The next logical motion state reached after applying the candidate mechanical action is referred to herein as the next candidate motion state. Starting from the current motion state, the next candidate motion state is obtained by considering the candidate mechanical action and its impact on the prosthesis, orthosis, or artificial limb due to the motion model.

[0135] In step S26, the method includes the step of training the response model to output the next predefined motion state based on the current motion state.

[0136] Given a motion and an initial current motion state, a natural sequence of motion states is expected. These motion states can be derived from a trained response model. The next predefined motion state can correspond to the next expected motion state that is anticipated to be achieved through mechanical movements of a prosthesis, orthosis, or artificial limb.

[0137] In step S27, the method includes obtaining a reward value based on the next candidate motion state and the next predefined motion state, the reward value being associated with the current motion state and the candidate mechanical action.

[0138] The next logical motion state (i.e., the next candidate motion state) can differ from the development of motion states as defined in the training response model. A reward value is assigned to the pair consisting of the current motion state and the candidate mechanical action by comparing the two. In one embodiment, the reward value can be calculated by comparing the differences between a series of motion states obtained using the control agent and those obtained in the training response model.

[0139] In step S28, the method includes setting the next candidate motion state as the current motion state.

[0140] Then, starting from the new motion state achieved (set as the current motion state), the above steps S24 to S28 are repeated to describe a series of motion states derived from mechanical actions and associated with reward values.

[0141] Optionally, after step S28, steps S29 and S30 can be performed. In step S29, the method further includes determining a cumulative reward function based on one or more acquired reward values ​​after a predetermined condition is met.

[0142] Iteration can continue until a predetermined condition is reached. In one embodiment, the predetermined condition may be a predetermined number of iterations. In another embodiment, the predetermined condition may be the convergence property of the cumulative reward function obtained during one or more iterations of the repeated steps. In yet another embodiment, the predetermined condition may be continuously obtaining reward values ​​below a predetermined threshold.

[0143] In step S30, the method further includes the step of modifying the control agent based on the cumulative reward function.

[0144] Another goal of the self-evolving control agent is to use the reward values ​​accumulated through iterations to make multiple candidate motion states obtained from multiple candidate mechanical actions more closely resemble the corresponding parts of the trained response model. In this way, a strategy is developed that helps guide the development of the control agent so that a series of mechanical actions eventually elicits a series of motion states that mimic the corresponding series of motion states from the trained response model.

[0145] Based on the cumulative reward function, the control agent can then be modified. If the score derived from the cumulative reward function is below a first score threshold, the first candidate mechanical action can be marked as a "dead end," and the control agent can restart multiple iterations from the same initial current motion state. If the score derived from the cumulative reward function is above the first or second score threshold, certain parameters of the control agent can be enhanced to make it adopt mechanical actions similar to the candidate mechanical actions when used with a control device for a prosthesis, orthosis, or artificial limb.

[0146] Preferably, the modification of the control agent may include modifying the reward scoring algorithm used to obtain one or more reward values ​​during one or more iterations of the repeated steps.

[0147] According to an embodiment, the control agent can be developed to correspond to a single activity of the user, or it can be developed to be sufficient for a continuum of motion states and activities.

[0148] Those skilled in the art will understand that, in order to continue controlling the self-evolution of the agent, the steps of the method can be repeated starting from step S23 after step S30.

[0149] Figure 4 A flowchart illustrating an exemplary embodiment of a method for developing a preferred control agent for a prosthesis, orthosis, or artificial limb according to the present invention is provided. The prosthesis, orthosis, or artificial limb may be included in a manner similar to that described above. Figure 1 The system described.

[0150] In step S31, the method includes obtaining a motion model of a prosthesis, orthosis, or artificial limb, the motion model being defined in terms of the mechanical relationships between the components of the prosthesis, orthosis, or artificial limb.

[0151] In step S32, the method includes obtaining multiple training response models of a prosthesis, orthosis, or artificial limb. Each of the multiple training response models defines a predefined motor response of the prosthesis, orthosis, or artificial limb and is associated with a biomechanically different user wearing or equipped with the prosthesis, orthosis, or artificial limb. Preferably, the multiple training response models are associated with activities within an activity list that includes any one of the following: walking, running, sitting, driving, using stairs, walking on a slope, cycling, lying down, standing, and jumping.

[0152] In step S33, the method includes the following steps: obtaining an initial plurality of control agents, preferably similar control agents, each of the initial plurality of control agents being associated with a different training response model among a plurality of training response models and configured to be used by a control device of a prosthesis, orthosis or artificial limb to control the worn or equipped prosthesis, orthosis or artificial limb.

[0153] Those skilled in the art will understand that the initial multiple control agents can be obtained by any known method. Alternatively or additionally, the initial multiple control agents can be obtained through... Figure 3 The self-evolution method described in the embodiments is used to obtain [the information].

[0154] In step S34, the method includes the following steps: selecting a control agent from an initial plurality of control agents and defining the selected control agent as the preferred control agent. The control agent can be selected randomly from the initial plurality of control agents. Alternatively, the selection of the control agent can be based on a comparison of the user's physical characteristics (for which the control agent is developed) with respect to the physical characteristics of multiple biomechanically different users associated with the initial plurality of control agents.

[0155] In step S35, the method includes selecting a challenger control agent from among multiple control agents, the challenger control agent being different from the preferred control agent and preferably not selected in previous iterations. The challenger control agent can be randomly selected from among the multiple control agents.

[0156] In step S36, the method includes the following steps: acquiring user input indicating—in the process of a control device for a prosthesis, orthosis, or artificial limb controlling a worn or equipped prosthesis, orthosis, or artificial limb—user preferences in the challenger control agent and the preferred control agent in terms of performance of the challenger control agent and the preferred control agent. The acquisition of user input may be achieved via a user interface of a mobile device.

[0157] In step S37, the method includes the following steps: selecting a new preferred control agent between the previously preferred control agent and the challenger control agent based on user input. The new preferred control agent may be one that the user rated higher between the previously preferred control agent and the challenger control agent.

[0158] Then, starting from the selected new preferred control agent (set as the preferred control agent), the above steps S34 to S37 are repeated, thereby advancing the competitive selection among multiple control agents.

[0159] Optionally, step S38 may be performed after step S37. In step S38, the method further includes the step of determining a final preferred control agent after a predetermined condition is met. In one embodiment, the predetermined condition may include selecting a predetermined number of times the same preferred control agent is selected during the iteration of the repeated steps.

[0160] Figure 5 A flowchart depicts an exemplary embodiment of a method for further development of a final preferred control agent for prostheses, orthotics, or artificial limbs according to the present invention.

[0161] In step S41, the method includes the step of selecting the final preferred control agent as the first control agent.

[0162] In step S42, the method includes the step of generating a second control agent based on a first control agent—preferably by copying the first control agent.

[0163] In step S43, the method includes the following steps: training a second control agent based on a motion model of a prosthesis, orthosis, or artificial limb and a training response model associated with a first control agent, wherein the training is preferably performed over a predetermined number of training cycles.

[0164] In step S44, the method includes the following steps: obtaining further user input indicating—during the process of a control device for a prosthesis, orthosis, or artificial limb controlling a worn or equipped prosthesis, orthosis, or artificial limb—user preferences in the first and second control agents, respectively, regarding the performance of the first and second control agents. The acquisition of further user input may be achieved via a user interface of a mobile device.

[0165] In step S45, the method includes the following step: selecting a new preferred control agent between the previous first control agent and the second control agent based on further user input. The new first control agent may be one that the user rated higher among the previous first control agent and the second control agent.

[0166] Then, starting with the selected new first control agent (set as the first control agent), the above steps S42 to S45 are repeated, thereby advancing the training-based development relative to the previous replication of the first control agent. Those skilled in the art will understand that the new first control agent can be used in any iteration by the control device of the prosthesis, orthosis, or artificial limb to control the worn or equipped prosthesis, orthosis, or artificial limb.

[0167] Although the principles of the invention have been explained above with reference to specific embodiments, it should be understood that this description is by way of example only and is not intended to limit the scope of protection defined by the appended claims.

Claims

1. A method for the self-development of a control agent for a prosthesis, orthosis, or artificial limb, said control agent being configured to be used by a control device of the prosthesis, orthosis, or artificial limb to control the prosthesis, orthosis, or artificial limb, wherein, The method includes the following steps: - Obtain a motion model of the prosthesis, orthosis, or artificial limb, the motion model defining the mechanical relationships between the components of the prosthesis, orthosis, or artificial limb; - Obtain a training response model for the prosthesis, orthosis, or artificial limb, wherein the training response model defines a predefined motion response for the prosthesis, orthosis, or artificial limb; - Determine the current motion state of the prosthesis, orthosis, or artificial limb; The method further includes repeatedly performing the following steps: - The control agent determines candidate mechanical actions based on the current motion state, the mechanical actions being related to elements of the prosthesis, orthosis, or artificial limb; - The motion model outputs the next candidate motion state based on the current motion state and the candidate mechanical actions; - The trained response model outputs the next predefined motion state based on the current motion state; - Obtain a reward value based on the next candidate motion state and the next predefined motion state, the reward value being associated with the current motion state and the candidate mechanical action; - Set the next candidate motion state as the current motion state.

2. The method according to claim 1, wherein, The method further includes the following steps: -After the predetermined conditions are met, a cumulative reward function is determined based on one or more reward values ​​obtained; - Modify the control agent based on the score derived from the cumulative reward function.

3. The method according to claim 2, wherein, The modification of the control agent is also based on user reward input from a user wearing or equipped with the prosthesis, orthosis, or artificial limb, which is associated with the performance of the modified control agent used by the control device of the prosthesis, orthosis, or artificial limb when controlling the worn or equipped prosthesis, orthosis, or artificial limb.

4. The method according to any one of claims 1 to 3, wherein, The current motion state of the prosthesis, orthosis, or artificial limb is defined based on one or more sensing data from a list of available sensing data during the use of the prosthesis, orthosis, or artificial limb.

5. The method according to claim 4, wherein, The list of available sensing data includes data sensed by any one or more of the following sensing devices: neural sensors, angle sensors, accelerometers, gyroscopes, Hall sensors, force sensors, magnetometers, pressure sensors, torque sensors, temperature sensors, energy meters, current sensors, voltage sensors, humidity sensors, sonar sensors, EMG sensors, barometric pressure sensors, pressure sensor grids, EEG sensors, RFID sensors, and geolocation sensors.

6. The method according to claim 2 or 3, wherein, The predetermined conditions include: a predetermined number of iterations for the repeated steps; or, a predetermined threshold for the convergence characteristics of the cumulative reward function.

7. The method according to any one of claims 2 to 6, wherein, Modifying the control agent includes modifying the reward scoring algorithm used to obtain the reward value.

8. The method according to any one of claims 1 to 7, wherein, The components of the prosthesis, orthosis, or artificial limb include actuators, and wherein determining candidate mechanical actions includes determining the mechanical action of the actuator.

9. The method according to any one of claims 1 to 7, wherein, The training response model is associated with a user who wears or is equipped with the prosthesis, orthosis, or artificial limb.

10. The method according to any one of claims 1 to 9, wherein, The trained response model is associated with activities in an activity list, wherein the activity list includes any one of the following: walking, running, sitting, driving, using stairs, walking on a slope, cycling, lying down, standing, jumping.

11. A method for developing a preferred control agent for a prosthesis, orthosis, or artificial limb, comprising the following steps: - Obtain a motion model of the prosthesis, orthosis, or artificial limb, the motion model defining the mechanical relationships between the components of the prosthesis, orthosis, or artificial limb; - Obtain multiple training response models of the prosthesis, orthosis, or artificial limb, each of which defines a predefined motion response of the prosthesis, orthosis, or artificial limb and is associated with biomechanically different users wearing or equipped with the prosthesis, orthosis, or artificial limb. - Obtain an initial plurality of control agents, preferably similar control agents, each of the initial plurality of control agents being associated with a different training response model among the plurality of training response models and configured to be used by the control device of the prosthesis, orthosis or artificial limb to control the worn or equipped prosthesis, orthosis or artificial limb. - Select a control agent from the initial plurality of control agents and define the selected control agent as the preferred control agent; The method further includes repeatedly performing the following steps: - Select a challenger control agent from the plurality of control agents, wherein the challenger control agent is different from the preferred control agent and preferably has not been selected in previous iterations; - Obtain user input indicating user preferences in the Challenger Control Agent and the Preferred Control Agent in relation to the performance of the Challenger Control Agent and the Preferred Control Agent during the control of the prosthesis, orthosis or artificial limb control device for controlling the worn or equipped prosthesis, orthosis or artificial limb; - Based on the user input, select a new preferred control agent between the previous preferred control agent and the challenger control agent.

12. The method according to claim 11, wherein, The method further includes the following steps: -After the predetermined conditions are met, the final preferred control agent is determined.

13. The method according to claim 12, wherein, The method further includes the following steps: - Select the final preferred control agent as the first control agent; and The method further includes repeatedly performing the following steps: - Generate a second control agent based on the first control agent, preferably by copying the first control agent; - The second control agent is trained based on the motion model of the prosthesis, orthosis or artificial limb and the training response model associated with the first control agent, wherein the training is preferably performed over a predetermined number of training cycles; - Obtain further user input indicating further user preferences in the first and second control agents in relation to the performance of the first and second control agents during the process of the control device of the prosthesis, orthosis or artificial limb for controlling the worn or equipped prosthesis, orthosis or artificial limb; - Based on the further user input, a new first control agent is selected between the previous first control agent and the second control agent.

14. The method according to any one of claims 11 to 13, wherein, The initial acquisition of multiple control agents is based on one or more sensing data from a list of available sensing data during the use of the prosthesis, orthosis, or artificial limb.

15. The method according to claim 14, wherein, The list of available sensing data includes data sensed by any one or more of the following sensing devices: neural sensors, angle sensors, accelerometers, gyroscopes, Hall sensors, force sensors, magnetometers, pressure sensors, torque sensors, temperature sensors, energy meters, current sensors, voltage sensors, humidity sensors, sonar sensors, EMG sensors, barometric pressure sensors, pressure sensor grids, EEG sensors, RFID sensors, and geolocation sensors.

16. The method according to any one of claims 12 to 15, wherein, The predetermined conditions include the selection of a predetermined number of times the same preferred control agent is used during the iteration of the repeated steps.

17. The method according to any one of claims 11 to 16, wherein, The plurality of trained response models are associated with activities in an activity list, wherein the activity list includes any one of the following: walking, running, sitting, driving, using stairs, walking on a slope, cycling, lying down, standing, jumping.

18. A system comprising a prosthesis, orthosis, or artificial limb, said system further comprising: - At least one sensing device configured to sense data relating to movements performed by a user that involve a prosthesis, orthosis, or artificial limb worn or equipped by the user; - A control device configured to use a modified control agent developed according to claim 2 or a preferred control agent developed according to claim 11 in an element controlling the prosthesis, orthosis, or artificial limb.

19. The system of claim 18 further includes at least one user interface device configured to receive user input.

20. The system according to claim 18 or 19, wherein, The control device is integrated into or attached to the prosthesis, orthosis or artificial limb, and / or included in a wearable or mobile device.

21. The system according to any one of claims 18 to 20, wherein, The at least one sensing device is integrated or attached to the prosthesis, orthosis or artificial limb, and / or included in the wearable or mobile device.

22. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 10, or cause the computer to perform the steps of the method according to any one of claims 11 to 17.

Images