Walking assistance device and method and device for controlling the walking assistance device
Through sensor measurement and reinforcement learning algorithm optimization walking assist device, the problem of instability and difficulty in personalized adjustment of walking assist in the prior art is solved, and a safer and more stable walking assist effect is achieved.
Patent Information
- Application Number
- CN201910814915.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-01-04
- Filing Date
- 2019-08-30
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2039-08-30
AI Technical Summary
Existing walking assistance devices are difficult to effectively assist elderly users or users with weakened muscle strength to walk smoothly and safely, and it is difficult to personalize the adjustment to adapt to the gait characteristics of different users.
The user's gait is measured by sensors, and torque control variables are generated based on the smoothing and time delay of the state variables. Combined with the reinforcement learning algorithm, the provision of auxiliary torque is optimized to ensure that the size and degree of obstacles of auxiliary torque are suitable for user needs.
It improves the safety and stability of the walking auxiliary device, reduces the situation of walking mismatch, enhances the transmission efficiency of auxiliary torque, adapts to the gait characteristics of different users, and improves the regularity and safety of walking.
Smart Images

Figure CN111407603B_ABST
Abstract
Description
[0001] This application claims the priority benefit of Korean Patent Application No. 10-2019-0001189 filed on January 4, 2019, in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference in their entirety. Technical Field
[0002] At least one example embodiment relates to a walking assist device and / or a method of controlling a walking assist device. Background Art
[0003] Walking assistance devices can be used to assist users who experience inconvenience to walk more easily. Such walking inconvenience may be attributed to various reasons (e.g., illness or accident), and due to these reasons, the user cannot easily walk on his own. In addition, the recent aging social problem has led to an increasing number of people experiencing inconvenience and pain from reduced muscle strength or joint problems due to aging. Therefore, there is an increasing interest in walking assistance devices that enable elderly users or patients with reduced muscle strength or joint problems to walk with less effort.
[0004] A walking assist device may be worn on a user's body to provide the user with the power required for walking and assist the user in walking with a normal gait pattern. A walking assist device may directly interact with the user's body and therefore may require a high level of safety. Summary of the invention
[0005] At least one example embodiment is directed to a method of controlling a walking assist device configured to be worn by a user.
[0006] In at least one example embodiment, the method may include: determining a first state variable of a user's gait phase based on the user's gait; obtaining a second state variable by smoothing and time-delaying the first state variable; obtaining a third state variable by applying a torque control variable to the second state variable; and determining an assisting torque provided by the walking assistance device based on the third state variable.
[0007] In some example embodiments, the method may include determining the torque control variable in response to a gait phase in a gait cycle of the user corresponding to a set gait phase.
[0008] In some example embodiments, the method may include smoothing the torque control variable to generate a smoothed torque control variable, wherein obtaining the third state variable includes obtaining the third state variable by applying the smoothed torque control variable to the second state variable.
[0009] In some example embodiments, in normal mode, the step of determining the torque control variable includes determining the desired torque control variable by applying a state variable value corresponding to a current gait phase in a user's gait cycle to a torque control variable determination function based on parameters obtained from previous learning results.
[0010] In some example embodiments, in the learning mode, the step of determining the torque control variable includes determining a retrieval torque control variable generated based on a probability function, and setting the retrieval torque control variable as the torque control variable.
[0011] In some example embodiments, in the learning mode, the step of determining the torque control variable further comprises: determining a score of a previous torque control variable based on a walking assist power index determined from a previous gait of the user; and determining whether the score satisfies a condition.
[0012] In some example embodiments, the walking assistance power index includes: a first walking assistance power index indicating the magnitude of walking assistance power delivered by the walking assistance device, and a second walking assistance power index indicating the degree of hindrance to the walking user by the walking assistance device.
[0013] In some example embodiments, determining the score comprises determining the score based on a weighted sum of the first walking assist kinetic energy index and the second walking assist kinetic energy index.
[0014] In some example embodiments, the step of determining the first state variable includes determining the first state variable based on hip joint angle information of the user measured by a sensor of the walking assistance device.
[0015] In some example embodiments, the step of obtaining the second state variable includes: smoothing the first state variable to generate a smoothed first state variable; and obtaining the second state variable by time-delaying the smoothed first state variable.
[0016] In some example embodiments, the smoothing step includes obtaining a smoothed first state variable based on a first state variable determined for a current gait cycle and a first state variable determined for a previous gait cycle.
[0017] Some example embodiments relate to a non-transitory computer-readable medium comprising computer-readable instructions that, when executed by a computer, cause the computer to perform the method of controlling a walking assist device.
[0018] At least one example embodiment relates to an apparatus configured to control a walking assist device.
[0019] In at least one example embodiment, the device may include: a memory; and a controller configured to: control the assisting torque provided by the walking assistance device based on the gait of a user wearing the walking assistance device by: determining a first state variable of a gait phase of the user based on the gait, obtaining a second state variable by smoothing and time-delaying the first state variable, obtaining a third state variable by applying a torque control variable to the second state variable, and determining the assisting torque to be provided by the walking assistance device based on the third state variable.
[0020] In some example embodiments, the controller is configured to determine a gait phase in a gait cycle of the user based on the first state variable, and determine the torque control variable in response to the gait phase corresponding to a set gait phase.
[0021] In some example embodiments, the controller is configured to obtain the third state variable by smoothing the torque control variable to generate a smoothed torque control variable, and applying the smoothed torque control variable to the second state variable.
[0022] In some example embodiments, the controller is configured to: operate in a learning mode to: determine a torque control variable, determine a score of a previous torque control variable based on a walking assistance power index determined from a previous gait of the user, in response to the score not satisfying a set condition, determine a retrieval torque control variable generated based on a probability function, and set the retrieval torque control variable as the torque control variable.
[0023] In some example embodiments, the walking assistance power index includes: a first walking assistance power index indicating the magnitude of walking assistance power to be delivered by the walking assistance device, and a second walking assistance power index indicating the degree of hindrance to a walking user by the walking assistance device.
[0024] At least one example embodiment is directed to a walking assist device.
[0025] In at least one embodiment, the walking assistance device may include: a sensor configured to measure the gait of a user wearing the walking assistance device; a driver configured to operate an actuator of the walking assistance device based on a torque control signal to provide an assisting torque to the user; and a controller configured to generate a torque control signal to control the assisting torque based on the gait measured by the sensor by the following processing: determining a first state variable of a gait phase of the user based on the gait, obtaining a second state variable by smoothing and time-delaying the first state variable, obtaining a third state variable by applying the torque control variable to the second state variable, and determining the assisting torque provided by the walking assistance device based on the third state variable.
[0026] In some example embodiments, the controller is configured to determine a gait phase in a gait cycle of the user based on the first state variable, and determine the torque control variable in response to the gait phase corresponding to a set gait phase.
[0027] In some example embodiments, the controller is configured to obtain the third state variable by the following processing, smooth the torque control variable to generate a smoothed torque control variable, and apply the smoothed torque control variable to the second state variable.
[0028] Additional aspects of example embodiments will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] These and / or other aspects will become clear and more readily understood from the following description of example embodiments taken in conjunction with the accompanying drawings, in which:
[0030] Figure 1 is a diagram illustrating a walking assistance device worn on a user according to at least one example embodiment;
[0031] Figure 2 is a diagram illustrating a configuration of a walking assistance device according to at least one example embodiment;
[0032] Figure 3 and Figure 4 is a flow chart illustrating a method of controlling a walking assistance device according to at least one example embodiment;
[0033] Figure 5 is a flow chart illustrating a method of determining a torque control variable according to at least one example embodiment;
[0034] Figure 6 is a diagram illustrating a configuration of an apparatus for controlling a walking assistance device according to at least one example embodiment;
[0035] Figure 7 is a diagram illustrating a method of determining a first state variable according to at least one example embodiment;
[0036] Figure 8 is a diagram illustrating the operation of a state variable smoother according to at least one example embodiment;
[0037] Fig. 9 is a diagram illustrating the operation of a state variable delayer according to at least one example embodiment;
[0038] Fig.10is a graph illustrating a relationship between a first state variable and a gait phase according to at least one example embodiment;
[0039] Fig.11 is a diagram illustrating the operation of a torque control variable smoother according to at least one example embodiment;
[0040] Fig.12 is a diagram illustrating a configuration of a system for controlling a physical interaction device according to at least one example embodiment;
[0041] FIG. 13A to FIG. 13D are diagrams illustrating different examples of a system for controlling a physical interaction device according to at least one example embodiment. DETAILED DESCRIPTION
[0042] Hereinafter, some example embodiments will be described in detail with reference to the accompanying drawings. With regard to the reference numerals assigned to the elements in the accompanying drawings, it should be noted that the same elements will be designated by the same reference numerals as much as possible even if they are shown in different drawings. In addition, in the description of the embodiments, when it is considered that a detailed description of the well-known related structures or functions will lead to an ambiguous interpretation of the present disclosure, such description will be omitted.
[0043] However, it should be understood that the disclosure is not intended to be limited to the specific example embodiments disclosed. On the contrary, the example embodiments will cover all modifications, equivalents and substitutes that fall within the scope of the example embodiments. Throughout the description of the drawings, the same reference numerals represent the same elements.
[0044] In addition, terms such as first, second, A, B, (a), (b), etc. may be used herein to describe components. Each of these terms is not used to limit the nature, order, or sequence of the corresponding component, but is only used to distinguish the corresponding component from other components. It should be noted that if a component is described as being "connected," "coupled," or "joined" to another component in the specification, although the first component may be directly connected, coupled, or joined to the second component, a third component may be "connected," "coupled," or "joined" between the first and second components.
[0045] The terms used herein are for the purpose of describing specific embodiments only and are not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. It will also be understood that when the terms "comprise" and / or "comprising" are used herein, the characteristics, wholes, steps, operations, elements, components and / or groups thereof described are indicated, but the presence or addition of one or more other characteristics, wholes, steps, operations, elements, components and / or groups thereof are not excluded.
[0046] It should also be noted that in some optional embodiments, the functions / actions mentioned may not occur in the order mentioned in the drawings. For example, two drawings shown in succession may actually be performed substantially simultaneously according to the functions / actions involved, or may sometimes be performed in the reverse order.
[0047] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the disclosure of this application belongs. Unless explicitly defined herein, terms such as those defined in general dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and should not be interpreted as an idealized or overly formal meaning.
[0048] Furthermore, in the description of the exemplary embodiments, when it is considered that a detailed description of thus-known structures or functions will lead to obscure interpretation of the exemplary embodiments after understanding the disclosure of the present application, such description will be omitted.
[0049] Various example embodiments will now be described more fully with reference to the accompanying drawings, in which some example embodiments are shown. In the drawings, the thicknesses of layers and regions are exaggerated for clarity.
[0050] Hereinafter, examples will be described in detail with reference to the accompanying drawings, and like reference numerals in the drawings denote like elements throughout.
[0051] Figure 1 is a diagram illustrating a walking assistance device worn on a user according to at least one example embodiment.
[0052] The walking assist device 110 or gait assist device can assist the user 100 wearing the walking assist device 110 to walk easily. The walking assist device 110 can assist or support a part or the entire leg of the user 100 to help the user 100 walk more easily. The walking assist device 110 can be used as follows: Figure 1 The walking assist device 110 may be configured to assist or support the muscle strength of the user 100 when the user 100 walks to improve the walking motion or gait of the user 100 or to enable the user 100 to walk normally. For example, when an average user or an elderly user with reduced muscle strength wears the walking assist device 110, the walking assist device 110 may improve the user's walking ability by enabling the user to walk for a longer period of time compared to when the user is not wearing the walking assist device 110, and enable the user to walk independently by providing or increasing the power required for the user to walk. Figure 1 The type of walking assist device 110 shown in FIG. 1 is provided as an example, and the example embodiments described herein may therefore be applicable to other types of walking assist devices.
[0053] Reference Figure 1 , the walking assistance device 110 generates a torque or a rotational force at the left hip joint 120L and the right hip joint 120R under the control of the controller 130, and the generated torque provides power for flexion and extension to the legs of the user 100 through the transmitters 140L and 140R provided above the knees of the user 100. The walking assistance device 110 measures the walking motion or gait of the user 100 through its sensor, and estimates the gait state or gait phase in the gait cycle of the user 100 based on the measured gait. The walking assistance device 110 determines the direction along which power is to be provided to each leg and the amount of power to be provided at the current point in time based on the estimated gait phase.
[0054] The walking assistance device 110 is worn on the body of the user 100 so that the walking assistance device 110 applies power to the user 100 while receiving power applied by the user 100. Therefore, there is an interaction between the user 100 and the walking assistance device 110. In such an interaction, the reaction to the physical intervention of the walking assistance device 110 may vary from individual to individual. That is, individuals may react differently to the same physical intervention.
[0055] Therefore, one or more example embodiments may personalize the walking assistance device 110 by determining rules for adjusting the control variables of the walking assistance device 110 to be suitable for each individual user. The aforementioned term "personalized" indicates that the control strategy or rule of the walking assistance device 110 may be applied differently to individual users based on differences in individual gait characteristics, body states, muscle strength, etc. Personalization may be achieved by learning the control strategy of the walking assistance device 110 based on the measured gait states of the users.
[0056] The example embodiments described below provide safe and consistent walking assistance performance through personalization of the state variables and torque control variables of the walking assistance device 110 based on the walking motion or gait of the user. For safe and consistent walking assistance performance, a smoother and / or a delayer may be used, and reinforcement learning for improving (or, optionally, optimizing) the parameters for the control strategy may be performed. Through reinforcement learning, the auxiliary transmission power can be increased by increasing the ratio indicating how much the auxiliary torque provided by the walking assistance device 110 to the user 100 is useful to the user 100 walking, and the mismatch in walking assistance can be reduced by reducing the ratio indicating how much the auxiliary power provided by the walking assistance device 110 to the user 100 hinders the user 100 walking. In addition, walking assistance customized for individual users (or, optionally, optimized) can be provided, reducing the amount of energy required for walking, and improving the level of walking regularity, walking safety, and walking stability. Hereinafter, the example embodiments will be described in detail with reference to the accompanying drawings.
[0057] Figure 2 is a diagram illustrating a configuration of a walking assist device according to at least one example embodiment.
[0058] Reference Figure 2 The walking assistance device 200 includes a sensor 240, a control device 210, and a driver 250. The control device 210 may be a device for controlling the walking assistance device 200. According to an example, the walking assistance device 200 may further include a support member for supporting the body of a user wearing the walking assistance device 200 and a fixing member fixed to the body of the user. For example, the walking assistance device 200 may be Figure 1 The walking assistance device 110 is configured such that the control device 210 corresponds to Figure 1 Controller 130.
[0059] The sensor 240 may include various sensors. The sensor 240 may include a sensor configured to sense information associated with the user's walking motion or gait, and a sensor configured to sense information required to control the operation of the walking assistance device 200. For example, the sensor 240 may include a sensor configured to measure the user's gait (e.g., an acceleration sensor, an inertial sensor, and a gyroscope sensor), and a torque sensor configured to measure the assist torque transmitted by the driver 250, a current / voltage sensor, etc.
[0060] In one example, the user's gait can be measured by a sensor that senses angle information or motion information of two hip joints corresponding to the positions of the hip joints of the user's two legs. The angle information of the hip joints may include at least one set of information associated with the angle of the hip joints, the difference between the angles of the hip joints, the movement direction of the hip joints, and the angular velocity of the hip joints.
[0061] The control device 210 configured to control the walking assistance device 200 includes a controller 220 and a memory 230. The memory 230 is connected to the controller 220 and is configured to store instructions to be executed by the controller 220, and data to be processed by the controller 220 and / or data that has been processed by the controller 220. For example, the memory 230 stores parameters corresponding to the assist torque control signal output by the controller 220. The memory 230 may include a non-transitory computer-readable storage medium (e.g., a high-speed random access memory (RAM)) and / or a non-volatile computer-readable storage medium (e.g., at least one disk storage device, a flash memory device, or other non-volatile solid-state memory device).
[0062] The controller 220 may be implemented using a processing circuit (such as hardware including a logic circuit), a hardware / software combination (such as a processor executing software), or a combination thereof. For example, the processing circuit may include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a programmable logic unit, a microprocessor, or an application specific integrated circuit (ASIC), etc.
[0063] The controller 220 generates a control signal to control the walking assist device 200. For example, the controller 220 generates a torque control signal based on the gait measured by the sensor 240 to control the assist torque to be provided by the walking assist device 200. The controller 220 controls the driver 250, which is configured to generate the assist torque in the walking assist device 200 based on the generated control signal.
[0064] As discussed in more detail below, the controller 220 can be programmed as a dedicated processor for performing reinforcement learning, so that the walking assistance device 200 can improve the assisting torque transmission power by increasing the ratio of the assisting torque provided to the user by the walking assistance device 200 that is useful to the user when walking, and reduce the number of mismatches in walking assistance by reducing the ratio of the assisting torque provided to the user by the walking assistance device 200 that hinders the user from walking.
[0065] The driver 250 operates the actuator of the walking assistance device 200 based on the torque control signal generated by the controller 220. The driver 250 provides the assist torque to the position of the user's hip joint through the actuator. The actuator converts electrical energy into kinetic energy and applies the kinetic energy to the user's body to provide the user with the power required for the user to walk. The actuator is set on a portion corresponding to the position of the user's hip joint, and generates an assist torque for bending and extending the user's legs to assist the user in walking.
[0066] The controller 220 determines a state variable indicating a gait state of the user based on the gait of the user, and controls the walking assistance device 200 based on the determined state variable. The controller 220 sets a parameter for controlling the assist torque based on the state variable, and outputs a periodic assist torque control signal based on the set parameter to assist the user in walking.
[0067] In one example, the controller 220 controls the assisting torque to be provided by the walking assist device 200 based on the state variable, and determines whether to personalize the control signal for controlling the assisting torque based on the state variable. The controller 220 sets a gain for adjusting the intensity of the assisting torque and sets a time delay for adjusting the output time of the assisting torque, and defines the state variable based on the set gain and the set time delay. By adjusting the gain and the time delay, the walking assist device 200 can respond more stably to the user's sudden movement, the user's sudden stop, or the change of the environment, and improve the level of walking regularity and safety. The first state variable, the second state variable, and the third state variable to be described below can be used to define the assisting torque control signal for controlling the operation of the walking assist device 200.
[0068] In one example, the controller 220 determines a first state variable of a gait state of a user based on a walking motion or gait of the user. The controller 220 smoothes the first state variable and time-delays the smoothed first state variable to obtain a second state variable that is smoothed and time-delayed from the first state variable. In this example, the term "time-delaying..." may indicate a process performed using a time delay that is a time value set by a user or based on a control design rule (or, alternatively, preset). For example, when the walking speed of the user is greater than a first reference value, the controller 220 sets the time delay to be relatively small. On the contrary, when the walking speed of the user is less than the first reference value, the controller 220 sets the time delay to be relatively large. For another example, the controller 220 determines a delay based on the walking acceleration of the user. In this example, when the walking acceleration of the user is greater than a second reference value, the controller 220 sets the time delay to be small. On the contrary, when the walking acceleration of the user is less than the second reference value, the controller 220 sets the time delay to be large. In the aforementioned example, the walking speed and walking acceleration of the user may be determined based on the sensing information obtained from the sensor 240.
[0069] The controller 220 obtains the third state variable by applying the torque control variable to the second state variable. The controller 220 determines the gait phase in the gait cycle of the user based on the first state variable, and determines the torque control variable when the determined gait phase corresponds to a defined (or, optionally, predefined) gait phase.
[0070] The torque control variable is determined differently in the learning mode and in the normal mode. The user can select the learning mode or the normal mode. The user can select the learning mode to optimize or personalize the walking assistance device 200. Through the learning mode, the parameters associated with controlling the assist torque to be provided by the walking assistance device 200 can be adjusted based on the user's gait state or gait type.
[0071] When the normal mode is selected, the controller 220 determines a desired (or, optionally, optimal) torque control variable by applying a state variable value corresponding to the current gait phase of the user to the torque control variable determination function based on the parameters obtained from the previous learning result. The controller 220 determines the determined torque control variable as the torque control variable to be applied to the second state variable.
[0072] When the learning mode is selected, the controller 220 determines the retrieved torque control variable generated based on the probability function as the torque control variable. In one example, in the learning mode, the controller 220 determines the score of the previous torque control variable based on the walking assistance power index determined from the user's previous gait. The walking assistance power index includes a first walking assistance power index and a second walking assistance power index, the first walking assistance power index indicating the magnitude of the walking assistance power transmitted by the walking assistance device 200, and the second walking assistance power index indicating the degree of obstruction of the walking user by the walking assistance device 200. The first walking assistance power index is a positive measurement result value of walking assistance, and the second walking assistance power index is a negative measurement result value of walking assistance. For example, the controller 220 calculates the score in the form of a weighted sum of the first walking assistance power index and the second walking assistance power index.
[0073] The walking assistance power index may be calculated in real time, and the control strategy of the walking assistance device 200 may be learned based on the walking assistance power index. In the learning mode, the parameters of the probability function may be determined so that the first walking assistance power index increases and the second walking assistance power index decreases, and the retrieval torque control variable may be determined based on the probability function. When the score determined based on the walking assistance power index does not satisfy the set (or, optionally, preset) condition, the controller 220 generates a retrieval torque control variable based on the probability function, and determines the generated retrieval torque control variable as the torque control variable to be applied to the second state variable. The set (or, optionally, preset) condition may indicate that the score is greater than a threshold value, or the number of times the score is calculated reaches a specific number of times.
[0074] The controller 220 obtains a third state variable by smoothing the determined torque control variable and applying the smoothed torque control variable to the second state variable. The controller 220 generates a control signal based on the obtained third state variable to determine the assisting torque to be provided by the walking assistance device 200. The controller 220 generates an assisting torque control signal by applying a gain for adjusting the intensity of the assisting torque to the third state variable, and controls the assisting torque to be provided by the walking assistance device 200 based on the generated assisting torque control signal.
[0075] Through the above operation, it is possible to reduce the number of mismatches in the walking assistance provided by the walking assistance device 200, improve the efficiency of assisting torque transmission, and / or improve the level of walking regularity. For example, through reinforcement learning, the walking assistance device 200 can improve the assisting torque transmission power by increasing the ratio of the assisting torque provided by the walking assistance device 200 to the user being useful to the user when walking, and reduce the number of mismatches in the walking assistance by reducing the ratio of the assisting torque provided by the walking assistance device 200 to the user hindering the user from walking.
[0076] According to an example, a remote controller (not shown) configured to remotely control the walking assistance device 200 may be provided. The remote controller may control the overall operation of the walking assistance device 200 in response to an input from a user. For example, the remote controller may start or pause the operation of the walking assistance device 200. In addition, the remote controller may generate an auxiliary torque control signal to control the walking assistance operation of the walking assistance device 200, and send the generated auxiliary torque control signal to the walking assistance device 200. The remote controller may provide a user interface (UI) that facilitates the manipulation or operation of the walking assistance device 200. Through the UI, the user may directly set a gain associated with the intensity of the auxiliary torque to be provided by the walking assistance device 200 or a delay in the output time of the auxiliary torque.
[0077] Figure 3 and Figure 4 1 is a flowchart illustrating a method of controlling a walking assistance device according to at least one example embodiment. The method of controlling a walking assistance device is hereinafter referred to as a control method, and the control method may be performed by a device for controlling a walking assistance device (hereinafter referred to as a control device (for example, the control device 210)).
[0078] Reference Figure 3 In operation 310, the control device determines a first state variable of a gait state of a user wearing a walking assistance device based on the gait of the user. For example, the control device determines the first state variable based on the hip joint angle information of the user measured by the sensor of the walking assistance device. The first state variable may indicate the gait or movement of the user associated with the gait state of the user, and may be defined by the angle information of the left hip joint and the right hip joint, as shown in Equation 1.
[0079] [Equation 1]
[0080] y 2 (t) = sinq r (t)-sinq l (t)
[0081] In equation 1, y 2 (t) represents the first state variable based on time t.r (t) and q l (t) represent the angle of the right hip joint and the angle of the left hip joint based on time t, respectively.
[0082] In operation 320, the control device obtains a second state variable that is smoothed and time-delayed from the first state variable. Figure 4 The operation 320 is performed in detail. Figure 4 In operation 410, the control device smoothes the first state variable. By smoothing, noise in the first state variable can be reduced. The control device can smooth the first state variable using a smoothing filter including, for example, a low pass filter (LPF). The control device can obtain the smoothed first state variable by performing low pass filtering based on the first state variable determined for the current gait cycle and the first state variable determined for the previous gait cycle, as shown in Equation 2.
[0083] [Equation 2]
[0084] y=(1–α)y prev +αy raw , (0≤α≤1)
[0085] In Equation 2, y represents the smoothed first state variable, and α represents a defined (or, optionally, predefined) constant having a value between 0 and 1. prev represents the first state variable determined for the previous gait cycle, y raw represents the first state variable determined for the current gait cycle.
[0086] In operation 420, the control device obtains a second state variable by time delaying the smoothed first state variable obtained in operation 410. In another example, the time delay may be performed before smoothing. In this example, the second state variable may be obtained by performing smoothing (such as filtering) on the time-delayed first state variable obtained by the time delay. In the time delay process, the time delay value applied to the first state variable may be a defined (or, alternatively, predefined) value or a value set by a user.
[0087] In operation 430, the control device determines whether the gait phase of the user corresponds to a defined (or, optionally, predefined) gait phase. The control device calculates the current gait phase based on the smoothed first state variable obtained in operation 410, as shown in Equation 3.
[0088] [Equation 3]
[0089]
[0090] In Equation 3, φ represents the gait phase, and y represents the smoothed first state variable. The calculated gait phase may be periodic based on the user's gait.
[0091] In operation 440, when the user's gait phase corresponds to a defined (or, optionally, predefined) gait phase, the control device determines a torque control variable. The torque control variable can be used to more finely adjust the time point of applying the assist torque control signal. In a mode other than the learning mode, a previously determined torque control variable or a defined (or, optionally, predefined) torque control variable can be used. However, in the learning mode, a desired (or, optionally, optimal) torque control variable can be retrieved based on the user's gait. The learning mode can be a mode that includes processing for personalizing the walking assistance device to suit the user, and the user can select whether to enter the learning mode. In the following, reference will be made to Figure 5 Describe in detail how to determine the torque control variable.
[0092] In operation 450, the control device smoothes the torque control variable determined in operation 440. In one example, the torque control variable determined in operation 440 may vary discontinuously, and such discontinuous variation of the torque control variable may produce relatively large jerk motions, which may pose a threat to the safety of the user and the walking assistance device. Therefore, a consistent response may not be easily obtained from the user. In order to suppress (or, optionally, prevent) this situation, the control device may smooth the torque control variable so that the torque control variable is gradually applied. With the smoothed torque control variable, the assist torque to be provided by the walking assistance device may change continuously and smoothly. Therefore, no jerk motion may be generated in the walking assistance device, and safe and stable interactive learning may occur between the user and the walking assistance device.
[0093] As a result of the determination in operation 430 , when the gait phase of the user does not correspond to the defined (or, alternatively, predefined) gait phase, the torque control variable applied to the previous cycle may be applied to the current cycle unchanged.
[0094] Return to reference Figure 3, in operation 330, the control device obtains a third state variable by applying the torque control variable to the second state variable. In operation 340, the control device determines the assisting torque to be provided by the walking assistance device based on the obtained third state variable. The control device generates an assisting torque control signal by applying a gain for adjusting the intensity of the assisting torque to the third state variable, and controls the assisting torque to be provided by the walking assistance device based on the generated assisting torque control signal. The assisting torque controlled in this manner is applied during one gait cycle of the user. When the gait cycle is terminated, operations 310 to 340 are performed again for the next gait cycle.
[0095] According to at least one example embodiment, a walking assistance device can be effectively personalized to suit the user of the walking assistance device based on the user's gait characteristics, and the safety of both the user and the walking assistance device can be improved by smoothly controlling the assisting torque to be provided by the walking assistance device.
[0096] Figure 5 is a flow chart illustrating a method of determining a torque control variable according to at least one example embodiment.
[0097] Reference Figure 5 , when determining the torque control variable in operation 440 , the controller may perform operations 510 to 560 discussed below.
[0098] In operation 510, the control device determines whether the current setting mode is the learning mode. In one example, the user can adjust the mode of the walking assistance device to the learning mode or the normal mode.
[0099] In operation 520, when the currently set mode is the normal mode, the control device determines the desired (or, optionally, optimal) torque control variable without a learning process. For example, the control device determines the desired (or, optionally, optimal) torque control variable by applying the state variable value corresponding to the user's current gait phase to the torque control variable determination function based on the parameters obtained from the previous learning results. In one example, when learning has been completed and the normal mode is currently operating, the control device uses the parameters obtained from the previous learning process to set a control strategy suitable for the user's current gait state, and determines the desired (or, optionally, optimal) torque control variable using the set control strategy. In another example, the control device uses the previously determined optimal torque control variable. The control device determines the determined torque control variable as the torque control variable to be applied to the second state variable.
[0100] When the current setting mode is the learning mode, the control device performs reinforcement learning including operations 530 to 560. Such a learning process may be performed once per gait cycle of the user.
[0101] In operation 530, the control device determines a score (e.g., a reward) of a previous torque control variable based on a walking assistance power index determined for a previous gait of the user. The walking assistance power index includes a first walking assistance power index indicating the magnitude of a walking assistance power transmitted from a walking assistance device and a second walking assistance power index indicating the degree of obstruction of a walking user by the walking assistance device. In one example, the first walking assistance power index indicates how well the auxiliary power is transmitted to the user for walking. The larger the first walking assistance power index is, the larger the auxiliary power to be transmitted is. In addition, the second walking assistance power index indicates the amount of obstruction to the walking user and is indicated by a negative value. The smaller the value is, the greater the resistance to the obstruction to the walking user is.
[0102] In one example, the score may be determined based on a weighted sum of the first walking assist power index and the second walking assist power index, as shown in Equation 4.
[0103] [Equation 4]
[0104] Score (reward) = 0.25 MPP + 5 MNP
[0105] In Equation 4, MPP (mean positive power) and MNP (mean negative power) represent the first walking assist power index and the second walking assist power index, respectively. In Equation 4, the weights applied to MPP and MNP are provided as examples, and the weights may vary according to the examples. For example, when one learning cycle is applied per gait cycle, MPP and MNP respectively indicate the average value of the first walking assist power index and the average value of the second walking assist power index for gait during the previous gait cycle (e.g., two steps or one step length). The above Equation 4 may be an objective function for adjusting the learning result to be desired.
[0106] When the score represented by the weighted sum of the first walking assistance power index and the second walking assistance power index increases, while providing a naturally large assist power, the walking assistance to be provided can be smoother without hindering the user who is walking. The learning process may be performed according to the current gait state of the user. For example, the learning process may be performed so that the value of the first walking assistance power index increases without the absolute value of the second walking assistance power index increasing excessively.
[0107] In operation 540, the control device determines whether a termination condition is satisfied. The termination condition indicates whether the score determined in operation 530 exceeds a threshold value, or whether the number of repetitions of the learning process reaches a defined (or, optionally, predefined) number of times. In operation 550, when the termination condition is not satisfied, the control device updates the parameters of the probability model for determining the retrieval torque control variable. The control device updates the parameters of the probability model so that the score increases. The updating of the parameters may be repeatedly performed for each gait cycle until the termination condition is satisfied.
[0108] In operation 560, the control device generates a retrieval torque control variable according to a function based on the probability model, and determines the generated retrieval torque control variable as a torque control variable to be applied to the second state variable.
[0109] The score of the newly determined retrieval torque control variable in each learning process may be measured, and the parameters of the probability model may be repeatedly updated until a termination condition is met.
[0110] Figure 6 is a diagram illustrating a configuration of a control device for controlling a walking assist device according to at least one example embodiment.
[0111] Reference Figure 6 The control device 610 of the walking assistance device 600 may correspond to Figure 2 The control device 210 is provided.
[0112] The control device 610 may include a state variable smoother 620 , a state variable delayer 630 , a learner 640 , a torque control variable smoother 650 , and a torque control signal generator 660 .
[0113] For example, as discussed in more detail below, the processing circuitry included in the control device 610 may be configured as a dedicated processor for performing operations of the state variable smoother 620, the state variable delayer 630, the learner 640, the torque control variable smoother 650, and the torque control signal generator 660.
[0114] The control device 610 can use such components to safely and effectively personalize walking assistance to suit the user of the walking assistance device 600, and the training or learning will be applied to the control strategy of the walking assistance device 600. In one example, the operation of the state variable smoother 620, the state variable delayer 630, the learner 640, the torque control variable smoother 650, and the torque control signal generator 660 can be described above with reference to Figure 2 The controller 220 described above performs the above operations.
[0115] A first state variable defining a gait state of a user is defined based on information associated with the gait of the user measured by a sensor of the walking assistance device 600. Figure 7 , the sensor of the walking assistance device 600 senses the left hip joint angle q associated with the user's current gait l and right hip angle q r The left hip joint angle information and the right hip joint angle information sensed by the sensor are transmitted to the control device 610. Then, the control device 610 determines the first state variable based on the left hip joint angle information and the right hip joint angle information. For example, as shown by the above equation 1, the control device 610 calculates the first state variable based on the left hip joint angle information and the right hip joint angle information.
[0116] Return to reference Figure 6 The state variable smoother 620 smoothes the first state variable determined based on the sensed hip joint angle information to change the first state variable into a smoother signal. Such smoothing may include, for example, low-pass filtering as shown in Equation 2 above. Figure 8 , the first state variable that is not smooth or includes noise is input to the state variable smoother 620, and the state variable smoother 620 changes the input first state variable into a smoother signal using a filter and outputs the smoother signal.
[0117] Return to reference Figure 6 , the state variable delayer 630 time-delays the smoothed first state variable and outputs a second state variable as a result of performing the time delay. The second state variable corresponds to the result of smoothing the first state variable calculated as shown in the above equation 1 and then time-delaying the smoothed first state variable. Fig. 9 , the smoothed first state variable y(t) is input to the state variable delayer 630, and the state variable delayer 630 applies a time delay Δt to the input first state variable y(t) and outputs a second state variable y(t-Δt). The time delay Δt determines the output time of the increased (or, optionally, the maximum) assisting torque, and may be a value (e.g., a constant) set by the user.
[0118] Return to reference Figure 6 , the learner 640 estimates the current gait phase of the user based on the smoothed first state variable. For example, the learner 640 estimates the gait phase based on Equation 5.
[0119] [Equation 5]
[0120]
[0121] In Equation 5, phase represents the gait phase of the user, y(t) represents the smoothed first state variable, Δt represents the time delay, and c represents a defined (or, optionally, predefined) constant that is a scaling factor for scaling. Fig.10 An example of a change over time of the gait phase calculated based on Equation 5 above is described.
[0122] Fig.10 A graph is shown indicating changes over time of the smoothed first state variable 1010 and changes over time of the gait phase 1020 determined based on the first state variable 1010. Both the first state variable 1010 and the gait phase 1020 have periodicity.
[0123] Return to reference Figure 6 , the learner 640 determines whether the current gait phase corresponds to a defined (or, optionally, predefined) gait phase. When the current gait phase corresponds to a defined (or, optionally, predefined) gait phase, the learner 640 determines the torque control variable. When the setting mode is the learning mode, the learner 640 retrieves the torque control variable by performing reinforcement learning to improve (or, optionally, optimize) the time point of applying the assist torque control signal. When the reinforcement learning is performed, the learner 640 calculates a score based on the first walking assistance power index and the second walking assistance power index. The learner 640 determines the parameters of the function based on the probability model based on the calculated score and the first state variable, and generates the retrieval torque control variable according to the function based on the probability model. The retrieval torque control variable can be used to finely adjust the time delay applied by the state variable delayer 630. Here, the learning process can be a process of finely adjusting the time delay based on the retrieval torque control variable and finding the retrieval torque control variable so that the score calculated as a result of the adjustment increases (or, optionally, maximizes). Such a learning process can be performed for a specific gait phase.
[0124] In a normal mode that is not a learning mode, the learner 640 determines the desired (or, optionally, optimal) torque control variable without performing a learning process. For example, the learner 640 determines the desired (or, optionally, optimal) torque control variable by applying the state variable value corresponding to the current gait phase of the user to the torque control variable determination function based on the parameters obtained from the previous learning result. The desired (or, optionally, optimal) torque control variable can also be used to finely adjust the time delay applied by the state variable delayer 630.
[0125] The torque control variable smoother 650 smoothes the retrieved torque control variable or the desired (or, optionally, optimal) torque control variable. Fig.11The torque control variable smoother 650 performs such smoothing processing that the torque control variable gradually changes near the time point when the torque control variable is applied and near the time point when the application is terminated. Therefore, it is possible to suppress (or, optionally, prevent) the assist torque provided by the walking assistance device 600 from changing rapidly.
[0126] Return to reference Figure 6 , the torque control variable smoother 650 generates a third state variable by applying the smoothed torque control variable to the second state variable output from the state variable delayer 630, and outputs the generated third state variable. The torque control signal generator 660 generates an auxiliary torque control signal based on the third state variable, and sends the generated auxiliary torque control signal to the walking assistance device 600. The torque control signal generator 660 generates the auxiliary torque control signal by applying a gain for adjusting the intensity of the auxiliary torque to the third state variable. The auxiliary torque control signal can be maintained within one gait cycle of the user. In one example, the torque control signal generator 660 inverts the auxiliary torque control signal to generate an inverted auxiliary torque control signal. The auxiliary torque control signal can be applied to assist the user's left leg in walking, and the inverted auxiliary torque control signal can be applied to assist the user's right leg in walking.
[0127] Fig.12 is a diagram illustrating a configuration of a system for controlling a physical interaction device according to at least one example embodiment. The system for controlling a physical interaction device will hereinafter be referred to as a control system.
[0128] Reference Fig.12 , refer to the above Figures 1 to 11 The described example embodiments are scalable and applicable to controlling a physical interaction device 1200 that physically interacts with a person. For example, the physical interaction device 1200 includes a wheel-based mobile robot, an articulated robot limb, a bipedal walking humanoid robot, and the like.
[0129] When there is a physical interaction between the user and the physical interaction device 1200, for the safety of the user and the physical interaction device 1200, the control device 1210 configured to control the physical interaction device 1200 may smooth the control signal used to control the physical interaction device 1200 or intentionally delay the control signal used to control the physical interaction device 1200. Figure 6The control device 610 of the walking assist device 600 shown in FIG. 1 includes a state variable smoother 1220, a state variable delayer 1230, a learner 1240, a power control variable smoother 1250, and a power control signal generator 1260. The respective operations of these components of the control device 1210 correspond to the operations of the components of the control device 610, respectively, and therefore, for the sake of simplicity, a detailed description of the operations will be omitted here. However, Fig.12 The control device 1210 and Figure 6 The difference between the control device 610 and the walking assistance device 600 is that the control device 1210 generates a power control signal through a power control signal generator 1260 to control the driver of the physical interaction device 1200, while the control device 610 generates an auxiliary torque control signal to control the auxiliary torque of the walking assistance device 600.
[0130] The learner 1240 determines whether a defined (or, optionally, predefined) event occurs based on information measured by the sensor of the physical interaction device 1200, and performs a learning process in response to the event occurrence. For example, when there is user-robot contact or interaction, the learner 1240 learns a user-customized control operation that is learned by the user's most preferred reaction. The method of calculating the state variables, scores, etc. used for learning may vary as described in the example of the walking assistance device. In addition, in addition to reinforcement learning, the learner 1240 may use other optimization or learning algorithms (such as evolutionary computing) to perform the learning process.
[0131] As described above, the control device 1210 may facilitate smoother and safer action-reaction interactions between a user and the physical interaction device 1200 .
[0132] FIG. 13A to FIG. 13D are diagrams illustrating different examples of a control system for controlling a physical interaction device according to at least one example embodiment.
[0133] Reference FIG. 13A to FIG. 13D , a control system for controlling the physical interaction device 1300 can be implemented by a selective combination of the physical interaction device 1300, the state variable smoother 1310, the state variable delayer 1320, the learner 1330, the dynamic control variable smoother 1340, and the dynamic control variable delayer 1350. In addition, the connection of the components or elements in each selective combination can be implemented in various ways.
[0134] The units and / or modules described herein can be implemented using hardware components and software components. For example, the hardware components may include a microphone, an amplifier, a bandpass filter, an audio to digital converter, and a processing device. The processing device can be implemented using one or more hardware devices, which are configured to implement and / or execute program codes by performing arithmetic, logic, and input / output operations. One or more processing devices may include a processor, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a field programmable array, a programmable logic unit, a microprocessor, or any other device that can respond and execute instructions in a limited manner. The processing device can run an operating system (OS) and one or more software applications running on the OS. In response to the execution of the software, the processing device can also access, store, manipulate, process, and create data. For simplicity, the description of the processing device is used as a singular, however, those skilled in the art will understand that the processing device may include multiple processing elements and multiple types of processing elements. For example, the processing device may include multiple processors or a processor and a controller. In addition, different processing configurations are possible, such as parallel processors.
[0135] Software may include computer programs, code segments, instructions, or some combination thereof, to independently or collectively instruct and / or configure a processing device to operate as desired, thereby converting the processing device into a special-purpose processor. Software and data may be permanently or temporarily implemented in any type of machine, component, physical or virtual equipment, computer storage medium or device, or permanently or temporarily implemented in a propagating signal wave that can provide instructions or data to a processing device or can be interpreted by a processing device. Software may also be distributed on networked computer systems so that the software is stored and executed in a distributed manner. Software and data may be stored by one or more non-transitory computer-readable recording media.
[0136] The method according to the above-mentioned example embodiment may be recorded in a non-transitory computer-readable medium including program instructions to implement various operations of the above-mentioned example embodiment. The medium may also include program instructions, data files, data structures, etc. separately or in combination with program instructions, data files, data structures, etc. The program instructions recorded on the medium may be program instructions specially designed and constructed for the purpose of the example embodiment, or they may be types known and available to technicians in the field of computer software. Examples of non-transitory computer-readable media include: magnetic media (such as hard disks, floppy disks, and tapes); optical media (such as CD-ROM disks, DVDs, and / or Blu-ray disks); magneto-optical media (such as optical disks); and hardware devices (such as read-only memory (ROM), random access memory (RAM), flash memory (e.g., USB flash drive, memory card, memory stick, etc.)) specially configured to store and execute program instructions. Examples of program instructions include both machine code (such as code generated by a compiler) and files containing high-level code that can be executed by a computer using an interpreter. The above-mentioned devices may be configured as one or more software modules to perform the operations of the above-mentioned example embodiments, and vice versa.
[0137] Several example embodiments have been described above. However, it should be understood that various modifications may be made to these example embodiments. For example, if the described techniques are performed in a different order, and / or if components in the described systems, architectures, devices, or circuits are combined in a different manner and / or replaced or supplemented by other components or their equivalents, suitable results may be achieved. Therefore, other implementations are within the scope of the appended claims.
Claims
1. A method of controlling a walking assist device, the walking assist device being configured to be worn by a user and to operate in a normal mode and a learning mode, the method include: determining a first state variable of a gait phase of the user based on the hip joint angle information of the user; Obtaining a second state variable by smoothing and time-delaying the first state variable; A smoothed torque control variable is obtained by determining the torque control variable and smoothing the determined torque control variable; obtaining a third state variable by applying a smoothed torque control variable to the second state variable; as well as determining an assist torque provided by the walking assistance device based on a third state variable, The method further comprises: in response to the gait phase in the user's gait cycle corresponding to the set gait phase, determining a torque control variable, the torque control variable being used to adjust a time point of applying an assist torque control signal for controlling the operation of the walking assistance device, Wherein, in the normal mode, the step of determining the torque control variable includes: The desired torque control variable is determined by applying the state variable value corresponding to the current gait phase in the user's gait cycle to a torque control variable determination function based on parameters obtained from previous learning results.
2. The method according to claim 1, in, In the learning mode, the steps to determine the torque control variables include: Determine the retrieval moment control variable generated based on the probability function, and Set the retrieved torque control variable to the torque control variable.
3. The method according to claim 2, in, In the learning mode, the steps of determining the torque control variable also include: determining a score for a previous torque control variable based on a walking assist dynamics index determined from a previous gait of the user; and determining whether the score satisfies a condition, Wherein, in response to the score not satisfying the condition, a step of determining a retrieval torque control variable generated based on a probability function is performed.
4. The method according to claim 3, in, The walking assistive power index includes: a first walking assistance power index indicating the magnitude of walking assistance power delivered by the walking assistance device, and The second walking assistance power index indicates a degree of hindrance to the user while walking caused by the walking assistance device.
5. The method according to claim 4, in, The steps of determining the score include: The score is determined based on a weighted sum of the first walking assist power index and the second walking assist power index.
6. The method according to claim 1, in, The step of determining a first state variable comprises: A first state variable is determined based on hip joint angle information of a user measured by a sensor of the walking assist device.
7. The method according to claim 1, in, The steps of obtaining the second state variable include: smoothing the first state variable to generate a smoothed first state variable; and The second state variable is obtained by time-delaying the smoothed first state variable.
8. The method according to claim 7, in, The step of smoothing the first state variable comprises: A smoothed first state variable is obtained based on the first state variable determined for the current gait cycle and the first state variable determined for the previous gait cycle.
9. A non-transitory computer-readable medium comprising computer-readable instructions which, when executed by a computer, cause the computer to perform the method of claim 1.
10. A device configured to control a walking assistance device, the device include: Memory; as well as A controller is configured to control the assist torque provided by the walking assistance device based on hip joint angle information of a user wearing the walking assistance device by: Determine a first state variable of the user's gait phase based on the hip joint angle information, The second state variable is obtained by smoothing and time-delaying the first state variable, By determining the torque control variable and smoothing the determined torque control variable, a smoothed torque control variable is obtained. The third state variable is obtained by applying the smoothed torque control variable to the second state variable, and determining an assist torque to be provided by the walking assistance device based on a third state variable, wherein the controller is configured to: in response to the gait phase in the user's gait cycle corresponding to the set gait phase, determine a torque control variable, the torque control variable being used to adjust a time point of applying an assist torque control signal for controlling the operation of the walking assistance device, The walking assistance device is configured to operate in a normal mode and a learning mode, and in the normal mode, the process of determining the torque control variable includes: The desired torque control variable is determined by applying the state variable value corresponding to the current gait phase in the user's gait cycle to a torque control variable determination function based on parameters obtained from previous learning results.
11. The device according to claim 10, in, The controller is configured as: The gait phase in a gait cycle of the user is determined based on the first state variable.
12. The device according to claim 11, in, The controller is configured to operate in a learning mode to: Determine the torque control variable, determining a score for a previous torque control variable based on a walking assist dynamics index determined from a previous gait of the user, In response to the score not satisfying the set condition, determining a retrieval torque control variable generated based on a probability function, and Set the retrieved torque control variable to the torque control variable.
13. The device according to claim 12, in, The walking assistance power index includes: a first walking assistance power index indicating a magnitude of walking assistance power to be delivered by the walking assistance device, and The second walking assistance power index indicates a degree of hindrance to the user while walking caused by the walking assistance device.
14. A walking assistance device, include: A sensor configured to: measure hip joint angle information of a user wearing the walking assistance device; a driver configured to: operate an actuator of the walking assistance device based on a torque control signal to provide an assist torque to a user; as well as The controller is configured to generate a torque control signal to control the assist torque based on the hip joint angle information measured by the sensor by the following processing: Determine a first state variable of the user's gait phase based on the hip joint angle information, The second state variable is obtained by smoothing and time-delaying the first state variable, By determining the torque control variable and smoothing the determined torque control variable, a smoothed torque control variable is obtained. The third state variable is obtained by applying the smoothed torque control variable to the second state variable, and determining an assist torque provided by the walking assistance device based on a third state variable, wherein the controller is configured to: in response to the gait phase in the user's gait cycle corresponding to the set gait phase, determine a torque control variable, the torque control variable being used to adjust a time point of applying an assist torque control signal for controlling the operation of the walking assistance device, The walking assistance device operates in a normal mode and a learning mode, and in the normal mode, the process of determining the torque control variable includes: The desired torque control variable is determined by applying the state variable value corresponding to the current gait phase in the user's gait cycle to a torque control variable determination function based on parameters obtained from previous learning results.
15. The walking assistance device according to claim 14, in, The controller is configured as: The gait phase in a gait cycle of the user is determined based on the first state variable.
