Artificial limb control method, system and equipment

Abstract

The invention belongs to the field of artificial limb control, and particularly discloses an artificial limb control method, system and equipment, and the method comprises the steps: obtaining real-time motion data of an artificial limb and a plurality of point positions on a user trunk; inputting the real-time motion data of the plurality of point locations into a first neural network, and predicting motion data of each point location in different preset motion modes at a future moment; taking the predicted motion data of each point location as a reference sequence, taking the real-time motion data of each point location as a contrast sequence, and determining the similarity of the reference sequence and the contrast sequence corresponding to each point location; in combination with the similarity of each point location in each preset motion mode, determining the possibility of being in each preset motion mode in the future moment; inputting the real-time motion data of the plurality of point locations into a second neural network, and obtaining corresponding artificial limb control parameters in different preset motion modes; and performing weighted summation on the artificial limb control parameters in different preset motion modes to obtain final control parameters of the artificial limb. According to the invention, smooth control of the artificial limb is realized.

Description

Technical Field

[0001] The present application relates to the field of prosthetic limb control, and more specifically, to a prosthetic limb control method, system, and device. Background Art

[0002] Traditional lower limb prostheses, such as passive prostheses worn by above-the-knee amputees, are directly connected to the amputee's thigh stump, resulting in a loss of rotational freedom at the knee and ankle joints. Consequently, asymmetric muscle compensation is required to complete movements such as walking, climbing stairs, and walking up and down slopes. This, however, can lead to strain on the compensatory muscles and increase the risk of injury. Therefore, powered thigh prostheses, which can mimic the amputee's thigh movements and autonomously rotate the knee and ankle joints, offer significant advantages in reducing asymmetric compensatory movements.

[0003] For powered thigh prostheses, the primary condition and key to assisting amputees in their movements lies in accurately identifying the amputee's movement purpose, and then accurately matching the movement of the prosthetic knee and ankle joints with the movement of the amputee's thigh stump. Good human-machine gait cycle matching helps amputees improve their trust in the prosthesis, enhances user experience, and further promotes the popularization of thigh powered prostheses. The more common method for matching the gait cycle progress of existing thigh prostheses is to construct a function based on the movement angle and cycle progress of the thigh stump, and then map the gait cycle progress based on the thigh angle, the integral of the thigh angle, or a more complex function relationship; there are also control methods for customizing the prosthetic movement speed, which allows users to independently control the landing time of the prosthesis, and thus control the joint angle movement of the prosthesis.

[0004] There are several main methods for existing prosthetic limbs to determine the motion mode based on external environmental factors and the amputee's motion information. The most common method is to determine based on the swing speed of the amputee's thigh and whether the prosthesis is touching the ground, and design the collected sensor information into a state determination of a finite state machine; there is also a method of installing a depth camera on the prosthetic thigh receiving cavity to obtain the terrain state in the direction of travel to change the motion state of the prosthesis. These motion state determinations by defining special nodes of terrain and amputee motion information usually have the following problems: the amputee is usually required to make specific actions to switch motion states (slopes, stairs, obstacles), and after entering the state, the problem of a single control parameter cannot adapt well to the user's own motion habits. In summary, although the existing control methods can meet the motion needs of amputees, they cannot achieve the maximum embodiment of the amputee's motion intentions, allowing the user to completely dominate the prosthesis's motion progress and speed. For this reason, it is necessary to further explore new methods for identifying amputee motion intentions. Contents of the invention In response to the defects of the existing technology, the purpose of this application is to provide a prosthetic control method, system and device, aiming to solve the problem that the existing prosthetic control method only considers a single motion state or terrain state and cannot better adapt to the user's motion habits.

[0005] To achieve the above objectives, in a first aspect, the present application provides a prosthesis control method, wherein the prosthesis is provided at the user's amputated thigh to assist the user in walking, and the control method comprises: Acquire real-time motion data of multiple points on the prosthesis and the user's torso; Inputting the real-time motion data of the plurality of points into a first neural network to predict the motion data of each point under different preset motion modes at future moments; Using the predicted motion data of each point as a reference sequence and the real-time motion data of each point as a control sequence, the similarity between the reference sequence and the control sequence corresponding to each point in each preset motion mode is determined; combining the similarity of each point in each preset motion mode, the probability of the user wearing the prosthesis being in each preset motion mode at a future moment is determined; Inputting the real-time motion data of the plurality of points into a second neural network to obtain corresponding prosthetic control parameters under different preset motion modes; The prosthesis control parameters in different preset motion modes are weighted and summed in combination with the probability of being in each preset motion mode to obtain the final control parameters of the prosthesis.

[0006] In one possible implementation, real-time motion data of multiple points on the prosthesis and the user's torso is obtained, including: Acquire real-time motion data of multiple first-category points through multiple sensors set on the prosthesis; The real-time motion data of multiple first-category points on the prosthesis are combined to solve the inverse motion equation and obtain the real-time motion data of multiple second-category points on the user's torso.

[0007] Among them, the sensors installed on the prosthesis may include simple sensors such as an inertial measurement unit (IMU) and a six-axis force sensor.

[0008] In a possible implementation, the plurality of first-category points include: an amputated thigh, a prosthetic shank, a prosthetic ankle, and a prosthetic foot; The plurality of second-category points include: a chest and abdomen center point, a pelvic center point, a healthy-side thigh, and a healthy-side calf.

[0009] In a possible implementation, the first neural network is trained using real-time motion data samples of multiple points and corresponding motion data samples under different preset motion modes at future moments; The second neural network is trained by real-time motion data samples of multiple points and corresponding prosthetic control parameter samples under different preset motion modes; The different preset motion modes correspond to different walking environments of users wearing prosthetic limbs; the walking environments include: flat ground, slopes with different gradients, stairs of different heights and / or different lengths, and at least one type of obstacles of different heights and / or different lengths.

[0010] In a possible implementation, the similarity between the reference sequence and the control sequence corresponding to each point in each preset motion mode is obtained by the following steps: Obtain a real-time updated control sequence based on the real-time motion data of the points; A heuristic loop algorithm is used to transform a control sequence updated at a single moment of a point, and an optimal transformation parameter is obtained when the dynamic time warping (DTW) distance between the transformed control sequence and the reference sequence is minimized; the transformation includes: scaling and / or translation; Determining the single-moment similarity between the control sequence updated at each moment of the point and the reference sequence in combination with the optimal transformation parameters; The similarities at each moment in the preset time period are weighted and summed to obtain the similarity between the reference sequence and the control sequence in the preset time period, which is used as the similarity between the reference sequence and the control sequence at the corresponding point; wherein, the weight of the single moment similarity at the later moment in the preset time period is greater than the single moment similarity at the earlier moment.

[0011] In a possible implementation, the single-moment similarity is:

[0012]

[0013] in, Indicates the similarity at time n; For Numerical related functions, for or ; represents the preset calculation coefficient, t represents the sampling period of real-time motion data, Indicates the sampling rate of real-time motion data; represents the maximum value in the reference sequence within time period t, and Represents the horizontal scaling parameters and vertical scaling parameters respectively, and Represent the horizontal translation parameters and vertical translation parameters respectively; The similarity between the reference sequence and the control sequence within the preset time period is: in, express The similarity between the reference sequence and the control sequence within the time period.

[0014] In a possible implementation, the possibility of being in each preset motion mode is obtained by the following steps: In a single preset motion mode, the similarities of the reference sequence and the control sequence of each point are weighted and summed to obtain the total similarity of the preset motion mode; wherein, the farther the point is from the point on the user's amputated thigh, the lower the weight value in the weighted summation; Normalizing the total similarity of each preset motion pattern; In combination with the normalization of the total similarity, the likelihood of each preset motion mode at the current moment is updated to obtain the likelihood of each preset motion mode at the future moment.

[0015] In a possible implementation, the possibility of each preset motion mode at the future moment is for:

[0016] in, Indicates the possibility of each preset sports mode at the current moment, represents the likelihood update weight, Indicates the normalized total similarity of each preset motion mode.

[0017] In one possible implementation, the final control parameter of the prosthesis is for:

[0018] in, Represents real-time motion data of multiple points, n Indicates the total number of preset sports modes. Indicates the future moment i Possibility of preset sports modes, Indicates the i Prosthetic control parameters for a preset motion pattern.

[0019] In a second aspect, the present application provides a prosthetic control system. The prosthetic limb is provided at the user's amputated thigh to assist the user in walking. The control system includes: A motion data acquisition module is used to obtain real-time motion data of multiple points on the prosthesis and the user's torso; A motion data prediction module, configured to input the real-time motion data of the plurality of points into a first neural network, and predict the motion data of each point under different preset motion modes at a future moment; A motion pattern determination module is configured to use the predicted motion data of each point as a reference sequence and the real-time motion data of each point as a control sequence to determine the similarity between the reference sequence and the control sequence for each point under each preset motion pattern; and to determine the likelihood that the user wearing the prosthesis will be in each preset motion pattern at a future moment based on the similarity of each point under each preset motion pattern; The control parameter acquisition module is used to input the real-time motion data of the multiple points into the second neural network to obtain the prosthesis control parameters corresponding to different preset motion modes; and to perform weighted summation of the prosthesis control parameters under different preset motion modes based on the probability of being in each preset motion mode to obtain the final control parameters of the prosthesis.

[0020] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a computer program product, which, when executed on a processor, enables the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0023] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the existing technologies: The present application provides a prosthetic control method, system and device. By comparing the predicted motion data and real-time motion data for similarity, the possibility of an amputee wearing a prosthesis being in different motion modes at any time in the future is predicted. The control parameters under different motion modes are then weighted and summed to generate a mixed control parameter, rather than using only a single motion state or terrain state. In this way, the amputee's autonomous motion habits are fully considered, smooth transition control of the prosthesis is achieved, and seamless switching and rapid identification of motion states are realized.

[0024] This application provides a prosthetic control method, system, and device that achieves automatic parameter debugging by directly weighting the possible motion parameters and corresponding motion states of each amputee. This not only saves engineers time in debugging, but also helps amputees quickly adapt to prosthetics. In addition, the solution provided by this application does not require the use of common terrain or motion state judgment sensors, including depth vision cameras, infrared rangefinders, and ultrasonic rangefinders. Only simple sensors such as IMUs and six-axis force sensors can be used to achieve a comprehensive judgment of motion data and terrain patterns to derive the amputee's motion state, thereby achieving prosthetic control that is integrated with the amputee's movement habits. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of a prosthetic limb control method provided in an embodiment of the present application; Figure 2 This is a diagram of the prosthetic limb sensor distribution and kinematic measurement points provided by the embodiments of the present application; Figure 3 is a schematic diagram of a second neural network architecture provided in an embodiment of the present application; Figure 4 Schematic diagram of a prosthetic control parameter calculation framework provided by an embodiment of the present application; Figure 5 This is a label framework diagram of two neural network output data provided by an embodiment of the present application; Figure 6 is an architectural diagram of a prosthetic control system provided in an embodiment of the present application; Figure 7 This is an architectural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0027] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0028] In the specification and claims herein, the terms "first," "second," and the like are used to distinguish between different objects, rather than to describe a specific order of objects. For example, a first neural network and a second neural network are used to distinguish between different neural networks, rather than to describe a specific order of neural networks.

[0029] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0030] In the description of the embodiments of the present application, unless otherwise specified, “multiple” means two or more than two. For example, multiple points refer to two or more than two points.

[0031] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0032] Figure 1 is a flow chart of the prosthetic control method provided in the embodiment of the present application; Figure 1 As shown, the following steps are included: Step S101: Acquire real-time motion data of multiple points on the prosthesis and the user's torso.

[0033] Optionally, real-time motion data of multiple points on the prosthesis and the user's torso is obtained, including: Acquire real-time motion data of multiple first-category points through multiple sensors set on the prosthesis; The real-time motion data of multiple first-category points on the prosthesis are combined to solve the inverse motion equation and obtain the real-time motion data of multiple second-category points on the user's torso.

[0034] like Figure 2 As shown, as an example, multiple first-category points include: amputated thigh, prosthetic calf, prosthetic ankle and prosthetic foot; multiple second-category points include: chest and abdomen center point, pelvic center point, healthy thigh and healthy calf.

[0035] The core of the solution provided by the embodiments of this application is to solve the inverse kinematic equation using sensor data such as IMU sensors located on the patient's prosthetic thigh and calf, as well as other parts of the body, and a six-axis force sensor on the prosthetic ankle joint. This allows calculation of the velocity of the main trunk, such as the center of the chest and abdomen, the healthy leg and the amputated leg, as well as the angular velocity and acceleration of the hip, knee, and ankle joints, as well as relative motion data.

[0036] More specifically, the real-time motion data of the plurality of points include: real-time motion data I of a first type of points and real-time motion data O of a second type of points.

[0037]

[0038]

[0039] in, represents the thigh acceleration, is the thigh angular velocity, is the calf acceleration, is the angular velocity of the lower leg, represents the foot acceleration, represents the foot angular velocity, Indicates that the ankle joint is stressed. represents the moment on the ankle joint, Indicates thigh length, Indicates the length of the calf, Represents the distance vector from the center of the ankle joint to the heel point, the center of the ankle joint to the landing point of the forefoot, and the landing point of the forefoot to the landing point of the heel. Indicates the distance from the center of the chest and abdomen to the center of the pelvis. Indicates the distance from the pelvic center to the healthy side hip joint rotation center, It represents the distance from the healthy side hip joint rotation center to the healthy side knee joint rotation center. It represents the distance from the healthy side knee joint rotation center to the healthy side ankle joint rotation center. Represents the distance vector from the center of the healthy side ankle joint to the healthy side heel point, the healthy side ankle joint center to the healthy side forefoot landing point, and the healthy side forefoot landing point to the healthy side heel landing point; represents the estimated angular velocity of the center point of the thorax and abdomen, represents the estimated acceleration of the thoracic and abdominal center, represents the estimated angular velocity of the pelvic center point, represents the estimated pelvic center acceleration, represents the estimated angular velocity of the healthy thigh, represents the estimated healthy thigh acceleration, represents the estimated angular velocity of the healthy leg, represents the estimated acceleration of the healthy leg.

[0040] Step S102: input the real-time motion data of the plurality of points into a first neural network to predict the motion data of each point under different preset motion modes at future moments.

[0041] Optionally, the first neural network is trained by real-time motion data samples of multiple points and corresponding motion data samples in different preset motion modes at future moments; the different motion modes correspond to different walking environments of users carrying prosthetic limbs; the walking environments include: flat ground, slopes with different gradients, stairs of different heights and / or different lengths, and at least one type of obstacles of different heights and / or different lengths.

[0042] In step S103, the predicted motion data of each point is used as a reference sequence, and the real-time motion data of each point is used as a control sequence to determine the similarity between the reference sequence and the control sequence corresponding to each point in each preset motion mode; combined with the similarity of each point in each preset motion mode, the possibility of the user wearing the prosthesis being in each preset motion mode at a future moment is determined.

[0043] Specifically, the similarity between the reference sequence and the control sequence corresponding to each point in each preset motion mode is obtained by the following steps: Obtain a real-time updated control sequence based on the real-time motion data of the points; A heuristic loop algorithm is used to transform the control sequence updated at a single moment to obtain the optimal transformation parameters when the DTW distance between the transformed control sequence and the reference sequence is minimized; the transformation includes: scaling and / or translation; Determining the single-moment similarity between the control sequence updated at each moment and the reference sequence in combination with the optimal transformation parameters; The similarities at each moment in the preset time period are weighted and summed to obtain the similarity between the reference sequence and the control sequence in the preset time period, which is used as the similarity between the reference sequence and the control sequence at the corresponding point; wherein, the weight of the single moment similarity at the later moment in the preset time period is greater than the single moment similarity at the earlier moment.

[0044] It can be understood that the heuristic loop algorithm is used here to scale and translate the control sequence after each update to obtain a transformed control sequence. Then, the data segments corresponding to the time length positions of the transformed reference sequence and the control sequence are intercepted to calculate the DTW distance. According to the previous heuristic algorithm, an optimal transformation parameter will be obtained (including two for translation and scaling, a total of four transformation parameters). When the minimum DTW distance is obtained, the single-moment similarity of the control sequence and the reference sequence at this moment is obtained according to the evaluation function of the four transformation parameters; when the control sequence is updated to the same length as the time label of the reference sequence, a weighted calculation of the single-moment similarity is performed to obtain the time period similarity. At this point, the similarity calculation of a data segment in a time segment is completed.

[0045] Among them, the larger the amplitude of the transformation, that is, the larger the transformation parameter, the greater the difference between the reference sequence and the control sequence before the transformation, that is, the lower the similarity between the reference sequence and the control sequence.

[0046] In a possible implementation, the single-moment similarity is:

[0047]

[0048] in, Indicates the similarity at time n; For Numerical related functions, for or ; represents the preset calculation coefficient, t represents the sampling period of real-time motion data, Indicates the sampling rate of real-time motion data; represents the maximum value in the reference sequence within time period t, and Represents the horizontal scaling parameters and vertical scaling parameters respectively, and Represent the horizontal translation parameters and vertical translation parameters respectively; Because the longer the motion data appears, the more obvious the motion characteristics are, so the similarity between the reference sequence and the control sequence within a preset time period can be defined as:

[0049] in, express The similarity between the reference sequence and the control sequence within a time period; referring to the above formula, it can be seen that the larger n is, the greater the corresponding weight value is.

[0050] In a possible implementation, the possibility of being in each preset motion mode is obtained by the following steps: In a single preset motion mode, the similarity of each point is weighted and summed to obtain the total similarity of the preset motion mode; wherein, the farther the point is from the point where the user's amputated thigh is located, the lower the weight value in the weighted summation; Normalizing the total similarity of each preset motion pattern; In combination with the normalization of the total similarity, the likelihood of each preset motion mode at the current moment is updated to obtain the likelihood of each preset motion mode at the future moment.

[0051] It can be understood that after completing the motion data prediction, the trunk motion data O is calculated by inverse kinematics using the real-time prosthetic motion data I. The gait progress prediction algorithm uses similarity matching calculation to calculate the corresponding gait progress in the most recent predicted gait cycle, so the matching parameters of each data (the angular velocity and speed of the chest and abdomen, thigh, and the relative motion data between the trunk) can be returned; the predicted gait pattern and the motion data of each major trunk should be a perfect match for the actual motion data of the human body and prosthesis, and the various parameters of the matching calculation are ideal values, such as the translation and rotation parameters are 0, and the scaling parameters are uniformly 1, that is, no scaling; in reality, due to the error accumulation between the inverse kinematics solution and the neural network prediction, there will be a certain difference between the matching calculation parameters and the ideal matching values.

[0052] When an amputee walks on a flat surface, even with the cumulative inverse kinematics and neural network predictions, as long as the predicted motion pattern is also that of walking on a flat surface, the error in the matching parameters will not be too large. Only when the motion pattern prediction is incorrect, and the real-time motion trajectory data differs significantly from the motion data under this motion pattern, causing the matching parameters to deviate significantly from the ideal values, will the error in the matching calculation exceed the acceptable error range. Therefore, these differences can be used to correct the gait pattern predicted by the neural network, and then correct the prosthetic motion pattern to better match the amputee's own movement habits and characteristics.

[0053] This means the prediction progress system is self-correcting and highly discriminative, automatically adjusting its prediction strategy and gait progress based on the amputee's real-time motion state. Furthermore, the neural network's attention mechanism can increase the proportion of losses in the prosthetic thigh, knee, calf, and ankle joints in the loss function, ensuring that the prosthesis can assist the amputee in completing the movement task normally despite a certain amount of matching error. Step S104: input the real-time motion data of multiple points into the second neural network to obtain corresponding prosthetic control parameters under different preset motion modes.

[0054] Optionally, the second neural network is trained by real-time motion data samples of multiple points and corresponding prosthetic control parameter samples under different preset motion modes.

[0055] like Figure 3 As shown, by inputting the calculated estimated data and the real data into the second neural network, the possibility of outputting the corresponding state ( Figure 3 Indicated by thin black arrows in the figure), Figure 3 The vertical axis represents the probability of the second neural network predicting the corresponding time, arranged from large to small, and the horizontal axis represents time. The user's actual movement intention conversion can be seen in Figure 3 The true state transformation in the figure, the light green box represents the maximum possible state after the state possibility adjustment calculated by similarity.

[0056] It should be noted that in this application, the user's real movement intention is the corresponding real movement mode. The user's real movement intention is reflected by his or her real-time movement data, and considering the uncertainty of the movement environment, the real intention may be adjusted in real time, and the user can smoothly transition between different movement modes when adjusting the movement mode in real time; while when traditional prosthetic limbs control the conversion between different movement modes, there may be a mechanical process of completing the previous movement mode and then starting the next movement mode. Such a control method cannot achieve a smooth transition between movement modes, resulting in the control of the prosthesis being unable to better match the user's real movement intention.

[0057] Therefore, the present application matches the predicted motion data of the future moment with the actual motion data to obtain the possibility of being in each motion mode at the future moment, that is, the possibility of being in each motion mode at the future moment is determined by referring to the amputee's actual motion intention; then refers to the control parameters under each motion mode, combines the probability of each motion mode for weighted summation, and obtains a comprehensive control parameter to achieve control of the prosthesis; through the above method, the control of the prosthesis can be better matched with the amputee's actual motion intention, and smooth transition control of the prosthesis can be achieved in combination with the amputee's actual motion intention, allowing the amputee to completely dominate the prosthesis's motion progress and speed, and achieve the maximum embodiment of the amputee's motion intention.

[0058] Step S105 , combining the probability of being in each preset motion mode, weighted summation of the prosthesis control parameters in different preset motion modes is performed to obtain the final control parameters of the prosthesis.

[0059] Furthermore, the similarity of the data corresponding to all states and all points has been calculated. However, due to the motion and inverse kinematics calculation errors of the prosthetic motor, the accuracy (or the reliability of the similarity) of the points farther away from the thigh is actually smaller. Therefore, when calculating the total similarity of a state, the similarity weight of the points farther away from the thigh point is lower:

[0060] Among them, f, d, e, ... is the credibility weight, which can be adjusted according to the actual situation.

[0061] Furthermore, after the credibility calculation is completed, the state probability needs to be adjusted, including: A. Normalization:

[0062] B. Possibility Update:

[0063] in is the updated weight of the likelihood. The calculation speed is 5hz, and the feedforward calculation The update rate is 10hz, so The next calculation will be calculated at the same time Continue to use.

[0064] After that, the possibility is calculated, and the possibility and the control parameters of the corresponding state are weighted with the possibility as the weight to obtain the final control parameters, which are sent to the lower computer to complete a system calculation.

[0065] In one possible implementation, the final control parameter of the prosthesis is for:

[0066] in, Represents real-time motion data of multiple points, n Indicates the total number of preset sports modes. Indicates the future moment i Possibility of preset sports modes, Indicates the i Prosthetic control parameters for a preset motion pattern.

[0067] It can be understood that the embodiment of the present application adds the current state and the possible future states, which can directly avoid the non-smooth transition caused by the finite state machine state conversion (for the possible states, we use parameter weights to select the calculation weights of the control parameters), based on the weight of the current state and the advance parameter setting of the most likely predicted state in the future; in the method provided in the embodiment of the present application, the possible states and probabilities in the future are first predicted, and then the prosthetic control parameters and probabilities of these states are weighted and added to generate the control parameters of the collective state.

[0068] More specifically, if Figure 4 As shown, it is a calculation sequence diagram arranged in the order of calculation time provided by the embodiment of the present application. Figure 4 The dotted box in the figure represents the multi-input feature matching algorithm MFGE, where M stands for multi-input and FGE is the feature matching algorithm.

[0069] MFGE includes a data network and similarity calculation. The data network, known as the first neural network, is used to predict the amputee's motion data in the future. Similarity calculation involves calculating similarities between the predicted and real-time motion data. Event probability adjustment involves adjusting the probability of each motion pattern at the current moment based on the calculated similarities to determine the probability of each motion pattern in the future.

[0070] The event network, i.e. the second neural network, is used to combine real-time motion data to obtain the corresponding prosthetic control parameters in different motion modes. Figure 5 As shown, the event network outputs the probabilities (i.e., prosthetic control parameters for different motion modes) of all states (movement modes). Motion modes include slopes (variables: slope), stairs (variables: stair height and length), and obstacles (variables: obstacle height and length). Each combination of variables in each state is considered a sub-state and assigned a probability. In other words, any combination of variables is considered a state. The parameters for each state are manually set in advance.

[0071] The data network outputs motion data for all states (i.e., motion modes), including predicted output data for all input points. Specifically, if the data in a row of the input matrix represents the angular velocity of the thigh, then the data in the corresponding position of the output matrix represents the predicted motion data for the next t time period.

[0072] The above two data networks and time networks can be obtained using existing network training methods, which will not be described in detail here.

[0073] Figure 6 This is the architecture diagram of the prosthetic control system provided by this application, such as Figure 6 Shown, including: The motion data acquisition module 610 is used to acquire real-time motion data of multiple points on the prosthesis and the user's torso; A motion data prediction module 620 is configured to input the real-time motion data of the plurality of points into a first neural network and predict the motion data of each point under different preset motion modes at a future moment; The motion pattern determination module 630 is configured to use the predicted motion data of each point as a reference sequence and the real-time motion data of each point as a control sequence to determine the similarity between the reference sequence and the control sequence for each point under each preset motion pattern; and to determine the likelihood that the user wearing the prosthesis will be in each preset motion pattern at a future moment based on the similarity of each point under each preset motion pattern; The control parameter acquisition module 640 is used to input the real-time motion data of the multiple points into the second neural network to obtain the prosthesis control parameters corresponding to different preset motion modes; and to perform weighted summation of the prosthesis control parameters under different preset motion modes based on the possibility of being in each preset motion mode to obtain the final control parameters of the prosthesis.

[0074] It should be understood that the above-mentioned system is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the system are similar to those described in the above-mentioned method. The working process of the system can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0075] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, such as Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the method in the above embodiment.

[0076] In addition, the logic instructions in the aforementioned memory 730 can be implemented in the form of a software functional unit and, when sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0077] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0078] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0079] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0080] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC.

[0081] The above embodiments can be implemented in whole or in part using software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions. When loaded and executed on a computer, the computer program instructions fully or partially produce the processes or functions described in the embodiments of this application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)).

[0082] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0083] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A prosthesis control method, wherein the prosthesis is placed on the user's amputated thigh to assist the user in walking, characterized in that: Control methods include: Acquire real-time motion data of multiple points on the prosthesis and the user's torso; Inputting the real-time motion data of the plurality of points into a first neural network to predict the motion data of each point under different preset motion modes at future moments; Using the predicted motion data of each point as a reference sequence and the real-time motion data of each point as a control sequence, the similarity between the reference sequence and the control sequence corresponding to each point in each preset motion mode is determined; combining the similarity of each point in each preset motion mode, the probability of the user wearing the prosthesis being in each preset motion mode at a future moment is determined; Inputting the real-time motion data of the plurality of points into a second neural network to obtain corresponding prosthetic control parameters under different preset motion modes; The prosthesis control parameters in different preset motion modes are weighted and summed in combination with the probability of being in each preset motion mode to obtain the final control parameters of the prosthesis.

2. The method according to claim 1, characterized in that Acquire real-time motion data from multiple points on the prosthesis and the user's torso, including: Acquiring real-time motion data of multiple first-category points through multiple sensors disposed on the prosthesis; solving inverse motion equations based on the real-time motion data of multiple first-category points on the prosthesis to obtain real-time motion data of multiple second-category points on the user's torso; and / or The multiple first-category points include: amputated thigh, prosthetic calf, prosthetic ankle and prosthetic foot; the multiple second-category points include: chest and abdomen center point, pelvic center point, healthy thigh and healthy calf.

3. The method according to claim 1, characterized in that The first neural network is trained by real-time motion data samples of multiple points and corresponding motion data samples under different preset motion modes at future moments; The second neural network is trained by real-time motion data samples of multiple points and corresponding prosthetic control parameter samples under different preset motion modes; The different preset motion modes correspond to different walking environments of users wearing prosthetic limbs.

4. The method according to claim 1, wherein The similarity between the reference sequence and the control sequence corresponding to each point in each preset motion mode is obtained by the following steps: Obtain a real-time updated control sequence based on the real-time motion data of the points; A heuristic loop algorithm is used to transform the control sequence updated at a single moment of the point, and an optimal transformation parameter is obtained when the dynamic time adjustment distance between the transformed control sequence and the reference sequence is minimized; Determining the single-moment similarity between the control sequence updated at each moment of the point and the reference sequence in combination with the optimal transformation parameters; The similarities at each moment in the preset time period are weighted and summed to obtain the similarity between the reference sequence and the control sequence in the preset time period, which is used as the similarity between the reference sequence and the control sequence at the corresponding point; wherein, the weight of the single moment similarity at the later moment in the preset time period is greater than the single moment similarity at the earlier moment.

5. The method according to claim 4, characterized in that The transformation includes: scaling and / or translation; The single moment similarity is: in, Indicates the similarity at time n; For Numerical related functions, for or ; represents the preset calculation coefficient, t represents the sampling period of real-time motion data, Indicates the sampling rate of real-time motion data; represents the maximum value in the reference sequence within time period t, and Represents the horizontal scaling parameters and vertical scaling parameters respectively, and Represent the horizontal translation parameters and vertical translation parameters respectively; The similarity between the reference sequence and the control sequence within the preset time period is: in, express The similarity between the reference sequence and the control sequence within the time period.

6. The method according to claim 1, characterized in that The probability of being in each preset motion mode is obtained by the following steps: In a single preset motion mode, the similarities of the reference sequence and the control sequence of each point are weighted and summed to obtain the total similarity of the preset motion mode; wherein, the farther the point is from the point on the user's amputated thigh, the lower the weight value in the weighted summation; Normalizing the total similarity of each preset motion pattern; In combination with the normalization of the total similarity, the likelihood of each preset motion mode at the current moment is updated to obtain the likelihood of each preset motion mode at the future moment.

7. The method according to claim 6, characterized in that The possibility of each preset motion mode at the future moment for: in, Indicates the possibility of each preset sports mode at the current moment, represents the likelihood update weight, Indicates the normalized total similarity of each preset motion mode.

8. The method according to claim 1, characterized in that Final control parameters of the prosthesis for: in, Represents real-time motion data of multiple points, n Indicates the total number of preset sports modes. Indicates the future moment i Possibility of preset sports modes, Indicates the i Prosthetic control parameters for a preset motion pattern.

9. A prosthetic control system, wherein the prosthetic limb is placed on the user's amputated thigh to assist the user in walking, characterized in that: The control system includes: A motion data acquisition module is used to obtain real-time motion data of multiple points on the prosthesis and the user's torso; A motion data prediction module, configured to input the real-time motion data of the plurality of points into a first neural network, and predict the motion data of each point under different preset motion modes at a future moment; A motion pattern determination module is configured to use the predicted motion data of each point as a reference sequence and the real-time motion data of each point as a control sequence to determine the similarity between the reference sequence and the control sequence for each point under each preset motion pattern; and to determine the likelihood that the user wearing the prosthesis will be in each preset motion pattern at a future moment based on the similarity of each point under each preset motion pattern; The control parameter acquisition module is used to input the real-time motion data of the multiple points into the second neural network to obtain the prosthesis control parameters corresponding to different preset motion modes; and to perform weighted summation of the prosthesis control parameters under different preset motion modes based on the probability of being in each preset motion mode to obtain the final control parameters of the prosthesis.

10. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 8.

Images