Artificial Intelligence-Based Robot-assisted Orthopedic Exercise System and Method for controlling the system
The AI-based robot-assisted orthopedic exercise system addresses the challenge of individual adaptation and therapist dependency by providing personalized training plans and effective evaluation, enhancing gait rehabilitation efficiency.
Patent Information
- Application Number
- KR1020240138425
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2024-04-01
- Filing Date
- 2024-10-11
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2044-10-11
AI Technical Summary
Existing robot-assisted orthopedic exercise systems face challenges in adapting to individual patient needs and require significant therapist intervention, leading to ineffective gait rehabilitation outcomes.
An AI-based robot-assisted orthopedic exercise system that includes a walking exercise device with an actuator, an input unit, indicator prediction, training planning, and evaluation units to create personalized training plans, reducing therapist intervention and enhancing rehabilitation efficiency.
The system enables patient-friendly, personalized, and efficient rehabilitation by predicting gait evaluation indicators, establishing tailored training plans, and accurately evaluating training results, thereby improving gait rehabilitation outcomes.
Smart Images

Figure 112024110541046-PAT00009_ABST
Abstract
Description
Technology Field
[0001] The present disclosure relates to a robot-assisted orthopedic exercise system, and more specifically, to a patient-friendly AI-based assisted orthopedic exercise system and a control method thereof, which is based on artificial intelligence (AI) and capable of automatically establishing and executing a personalized orthopedic exercise plan and automatically modifying the exercise plan according to the rehabilitation effect of the exercise. Background Technology
[0002] The robot-assisted orthopedic exercise system was developed as a device to aid in the gait rehabilitation of patients with walking difficulties. This system helps improve walking function and restore lower body muscle strength in patients with mobility impairments; as a so-called end-effector type robotic device, it supports and assists the patient's physical movements, enabling them to undergo gait rehabilitation more safely and effectively.
[0003] This system can be adjusted according to the patient's physical ability and condition, provides gait training by simulating or assisting walking movements, and also detects and corrects the patient's movements to help improve walking ability more safely and efficiently and restore independence in daily life.
[0004] In these existing systems, a therapist is present to guide and assist with the rehabilitation exercise process. However, in some cases, patients are unable to adapt to the operation of the pedals and fail to respond to or comply with the pedal movements, and there are instances where effective rehabilitation is difficult even after a long period of gait training.
[0005] Therefore, the development of a rehabilitation training system that enables safer and more effective training, thereby helping patients rehabilitate quickly and return to daily life, is highly beneficial.
[0006] delete Prior art literature
[0007] 1. Patent Publication 10-2023-0024499 The problem to be solved
[0008] The present disclosure presents an artificial intelligence-based robot-assisted orthopedic exercise system and a method capable of establishing a patient-friendly gait training plan.
[0009] The present disclosure presents an artificial intelligence-based robot-assisted orthopedic exercise system and a method thereof that can reduce therapist intervention based on a medical professional's prescription.
[0010] The present disclosure presents a patient-personalized and friendly AI-based robot-assisted orthopedic exercise system and a method thereof. means of solving the problem
[0011] Artificial intelligence-based robot-assisted orthopedic exercise system according to the present disclosure:
[0012] An artificial intelligence-based robot-assisted orthopedic exercise system comprising: a walking exercise device having a pedal on which a patient stands and an actuator having one or more links that drive the pedal, and a system operating device that controls the main actuator according to a training plan established for each individual to conduct walking training for the patient standing on the pedal;
[0013] The above system operating device; is
[0014] An input unit that receives basic information including personal information and initial walking state information for each patient;
[0015] An indicator prediction unit that receives initial walking state information or walking training results after walking training and determines walking evaluation indicators;
[0016] A training planning unit having an autonomous training plan model that establishes a personalized training plan for the patient, performed by the gait exercise unit based on the results of the gait training; and
[0017] It comprises a training evaluation unit that evaluates training results based on training result data according to the results of the rehabilitation training of the patient performed by the above-mentioned walking exercise unit.
[0018] According to one or more embodiments of the present disclosure,
[0019] The results of the gait training may include at least one of cadence, stride length, stride height, initial contact angle, toe off angle, ground reaction force (GRF), and gait cycle rate.
[0020] According to one or more embodiments of the present disclosure,
[0021] The above training planning unit further includes a reference training planning model learned from patient information, training protocols, training time, training parameters, etc. set by a medical professional, and
[0022] The above training planning unit can output a corrected or compensated final training plan based on the outputs of the above autonomous training planning model and the reference training planning model.
[0023] According to one or more embodiments of the present disclosure,
[0024] The above training planning unit may further include a final training planning model that outputs a final training plan based on the outputs of the above autonomous training planning model and the standard training planning model.
[0025] According to one or more embodiments of the present disclosure,
[0026] The above may further include a detection unit that detects at least one of the following information as training information: Ground Reaction Force (GRF), Body Weight Support (BWS), Upper body inclination or tilting, Joint angle, Cadence, Stride length, and Ankle angle of a patient undergoing gait training.
[0027] According to one or more embodiments of the present disclosure,
[0028] A monitoring unit may be further provided that monitors the state of rehabilitation training performed by the gait exercise unit according to the above training plan, acquires information during the training, and generates training state information related to the patient's gait training therefrom.
[0029] According to one or more embodiments of the present disclosure,
[0030] The monitoring unit may present training guidelines or recommended guidelines or the current training status to the patient to improve gait related to the gait training, based on information during the training.
[0031] According to one or more embodiments of the present disclosure,
[0032] The above monitoring unit may further include a Head-Mounted Display (HMD) that presents the patient with necessary information related to the patient's walking exercise, training status information, or recommended guidelines related to training.
[0033] An exercise system according to one or more embodiments according to the present disclosure: is
[0034] The HMD may further be equipped with a device that presents at least one of necessary information related to the patient's walking exercise, training status information, or recommended guidelines related to training to the patient.
[0035] According to one or more embodiments of the present disclosure,
[0036] The above training evaluation unit may include a time-series information-based anomaly detection model as a training evaluation model, which analyzes current training results or analyzes trends in results following repeated training by analyzing previous training results and current training results to predict future training.
[0037] An exercise system according to one or more embodiments of the present disclosure is,
[0038] The above reporting unit may further be equipped with a reporting model that generates report content in the form of text, images, and videos by analyzing previous training results and current training results.
[0039] A method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure:
[0040] A step in which an input unit receives basic information including personal information and initial walking state information for each patient;
[0041] A step in which an indicator prediction unit receives initial walking state information or walking training result information after walking training and predicts the walking evaluation indicator of the patient;
[0042] A training planning department establishes a personalized training plan for the patient using an autonomous training plan model based on the above-mentioned gait evaluation indicators;
[0043] A walking exercise device having an end-effector type pedal and an actuator having one or more links driving the pedal, the step of performing rehabilitation training of the patient according to the training plan; and
[0044] The training evaluation department includes a step of evaluating training results based on training result data according to the training results.
[0045] In a method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure,
[0046] The results of the gait training may include at least one of cadence, stride length, stride height, initial contact angle, toe-off angle, GRF, and gait cycle.
[0047] In a method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure,
[0048] The above indicator prediction unit, as an indicator prediction model, can predict the patient's gait evaluation indicator using either a Recurrent Neural Network (RNN) model or a Long Short-Term Memory (LSTM) model.
[0049] In a method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure,
[0050] The above training planning department:
[0051] An autonomous training plan model that establishes a personalized training plan for a patient based on initial gait state information or gait training results after gait training; and
[0052] It may further include a reference training plan model learned from patient information, training protocols, training time, training parameters, etc. set by medical professionals, and
[0053] The above training planning unit can output a corrected or compensated final training plan based on the outputs of the above autonomous training planning model and the reference training planning model.
[0054] Method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure:
[0055] The detection unit can detect at least one of the following information as training information: Ground Reaction Force (GRF), Body Weight Support (BWS), Upper body inclination or tilting, Joint angle, Cadence, Stride length, and Ankle angle of a patient undergoing gait training.
[0056] Method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure:
[0057] The monitoring unit can monitor the state of the rehabilitation training performed by the gait exercise unit according to the training plan, acquire information during the training, and generate training status information related to the patient's gait training from this.
[0058] In a method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure,
[0059] Based on information during the training, the monitoring unit may present training guidelines or recommended guidelines or the current training status to the patient to improve the gait related to the gait training.
[0060] A method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure:
[0061] The HMD can present the patient with necessary information related to the patient's walking exercise, training status information, or recommended guidelines related to training.
[0062] In the method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure, a time-series information-based anomaly detection model as a training evaluation model can perform future training prediction by analyzing current training results or analyzing the trend of results following repeated training by analyzing previous training results and current training results.
[0063] In a method for providing artificial intelligence-based robot-assisted orthopedic exercise according to one or more embodiments of the present disclosure, the reporting unit may analyze previous training results and current training results to generate report content in the form of text, images, or videos. Effects of the invention
[0064] According to the present invention, it is possible to establish a patient-friendly gait training plan, reduce the burden on therapists compared to conventional methods, and, in particular, enable efficient personalized and patient-friendly rehabilitation training. Rehabilitation training using this system and method utilizes an artificial intelligence model at every stage to predict accurate gait evaluation indicators, automatically establish training plans based thereon, and more accurately evaluate training results. Brief explanation of the drawing
[0065] FIG. 1 is a block diagram of an artificial intelligence-based robot-assisted orthopedic motion system according to the present disclosure, and FIG. 2 is a schematic perspective view of an artificial intelligence-based robot-assisted orthopedic motion system according to one embodiment of the present disclosure, and FIG. 3 is a side view of an artificial intelligence-based robot-assisted orthopedic motion system according to one embodiment of the present disclosure, and FIG. 4 is a front view of an artificial intelligence-based robot-assisted orthopedic motion system according to one embodiment of the present disclosure, and FIG. 5 illustrates a scene in which a patient trains while wearing a Head-mounted Display (HMD) in a gait training system according to the present disclosure, and FIG. 6 is a schematic excerpt perspective view of a walking motion unit of an artificial intelligence-based robot-assisted walking training system according to the present disclosure. FIG. 7 illustrates a normal walking cycle, showing the posture of the foot, knee, and thigh during normal walking. Figure 8 illustrates the regulations at each position in the walking pattern, FIG. 9 illustrates functional parts based on the software and hardware structure of a system operating device according to the present disclosure, and FIG. 10 illustrates the configuration of a data set applied to a gait evaluation indicator prediction model applied to an artificial intelligence-based robot-assisted orthopedic exercise method and system according to one embodiment of the present disclosure, and FIG. 11 is a flowchart of a training process for a new patient in an artificial intelligence-based robot-assisted orthopedic exercise presentation method according to one embodiment of the present disclosure, and FIG. 12 is an overall flowchart of an artificial intelligence-based robot-assisted orthopedic motion presentation method according to one embodiment of the present disclosure, and FIG. 13 illustrates a code snippet for a class of a Long Short-Term Memory (LSTM) model in an artificial intelligence-based robot-assisted orthotic motion method and system according to one embodiment of the present disclosure, and FIG. 14 illustrates a code snippet defining a forward propagation process of a model in an artificial intelligence-based robot-assisted orthotic motion method and system according to one embodiment of the present disclosure, and FIG. 15 illustrates a code snippet for the definition of a Python class implementing early stopping in an artificial intelligence-based robot-assisted orthopedic exercise method and system according to one embodiment of the present disclosure, and FIG. 16 illustrates a code snippet for the definition of train_model, a function for model training in an artificial intelligence-based robot-assisted orthotic motion method and system according to one embodiment of the present disclosure, and FIG. 17 illustrates a code snippet for a data evaluation function that evaluates new data from a gait training patient and predicts the patient's gait evaluation index (FAC, Functional Ambulation Categories) in an artificial intelligence-based robot-assisted orthopedic exercise method and system according to one embodiment of the present disclosure. FIG. 18 illustrates a code snippet for the execution of an LSTM model in an artificial intelligence-based robot-assisted orthotic motion method and system according to one embodiment of the present disclosure, and FIG. 19 is a code snippet for data input for training a model and the generation of a data frame accumulated therefrom, according to one embodiment of the present disclosure. FIG. 20 is a code snippet for the declaration, training settings, model creation, etc. of a transformer model according to one embodiment of the present disclosure, and FIG. 21 is a code snippet for the prediction of training and test data of a transformer model according to one embodiment of the present disclosure, and FIG. 22 illustrates a graphic interface showing various parameters of gait training, and Figure 23 is a butterfly diagram showing the average plantar pressure distribution of the patient during training as a result of the patient's gait training. Specific details for implementing the invention
[0066] Hereinafter, preferred embodiments of the concept of the present invention will be described in detail with reference to the attached drawings. However, embodiments of the concept of the present invention may be modified in various different forms, and the scope of the concept of the present invention should not be interpreted as being limited by the embodiments described below. It is preferable to interpret the embodiments of the concept of the present invention as being provided to more completely explain the concept of the present invention to those with average knowledge in the art. Identical reference numerals denote identical elements throughout. Furthermore, various elements and areas in the drawings are depicted schematically. Accordingly, the concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.
[0067] Terms such as first, second, etc. may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the concept of the present invention, the first component may be named the second component, and conversely, the second component may be named the first component.
[0068] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the concept of the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, expressions such as “comprising” or “having” are intended to indicate the existence of the features, number, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, actions, components, parts, or combinations thereof.
[0069] Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by those skilled in the art to which the concept of the present invention pertains. Furthermore, it will be understood that commonly used terms, such as those defined in advance, should be interpreted as having meanings consistent with their intent in the context of the relevant technology, and should not be interpreted in an overly formal sense unless explicitly defined herein.
[0070] Where an embodiment can be implemented differently, a specific process sequence may be performed differently from the order described. For example, two processes described in succession may be performed substantially simultaneously or in the reverse order of the description.
[0071] FIG. 1 is a block diagram of an artificial intelligence-based robot-assisted orthopedic motion system according to the present disclosure, FIG. 2 illustrates an embodiment of an artificial intelligence-based robot-assisted orthopedic motion system according to the present disclosure, FIG. 3 is a side view thereof, and FIG. 4 is a front view.
[0072] First, referring to FIG. 1, the exercise system (100) comprises a system operating device (111) formed by a complex configuration of hardware and software and a walking exercise unit (105) controlled by it, a main or control monitor (110) that controls the system operating device (111), a status display monitor (113) that displays the patient's exercise status, etc., a complex input unit (114) that inputs various electrical and optical data including a keyboard (112), and an additional option of a Head-Mounted Display (HMD, 115) worn by the patient. The system operating device (111) according to the present disclosure has the structure of a computer system based on a virtual agent comprising one or more models trained in machine learning or deep learning, etc., to perform data processing and prediction.
[0073] The above walking exercise unit (105) is equipped with two left and right pedals on which the patient stands and a left and right driving device that operates the two pedals, and is equipped with a detection unit (105a) equipped with various sensors that detect the patient's condition information during walking training on the walking exercise unit (105) itself or around it.
[0074] The detection unit (105a) may include a sensor that detects a signal, as well as a signal processing unit that generates a signal to be sent to the system operating device (111) from the signal from the sensor. The HMD (115) presents information necessary for a patient during training and is part of a monitor device in the exercise system (100) of the present disclosure.
[0075] Additionally, the system operating device (111) may be further equipped with an optical detection device for detecting the patient's movement status, such as a video camera, such as a webcam, which is an optical input device.
[0076] Referring to FIGS. 2 to 4, the exercise system (100) is a type of robot that provides walking training to a patient and provides two walking exercise units (105) for left and right that are controlled by the system operating device (111). The two walking exercise units (105) are positioned between the left main body base (101) and the right main body base (101) which are arranged side by side at a predetermined interval. The two pedals (105a) of the two walking exercise units (105) move in harmony with each other in correspondence with the movement trajectories of both left and right feet during walking, and in some cases, they may intentionally move in disharmony.
[0077] The above pedal moves to enable the patient to perform walking exercises suitable for the patient by means of a control signal from the system operating device (111), and the patient steps on it and walks by complying with the movement of the pedal. An ankle band that can be installed here secures the patient's foot to the pedal appropriately loosely or appropriately tightly so that the patient's foot can move within a certain range on the pedal.
[0078] The movement of these two pedals (105a) is for patient gait training and is controlled by the system operating device to accommodate an abnormal gait pattern while training the gait pattern to be closer to a normal gait pattern.
[0079] For such walking training, the left walking exercise unit (105) is equipped with a pedal (105a) and an operating link (105b) to which it is dynamically connected, and the operating link (105b) is coupled to the left main body base (101) to enable linear reciprocating and rotational movement via a first driving device (105c) provided on the left main body base (101), and the right walking exercise unit (105) is also coupled to the right main body base (101) to enable reciprocating and rotational movement via the first driving device (105c) of the right main body base (101). The pedal (105a) is connected to the operating link (105b) via a driving device, for example, a driving device capable of controlling the rotational angle, and the pedal (105a) is forcibly moved relative to the operating link (105b) by the driving device within a predetermined angle range.
[0080] The linear reciprocating motion of each of the walking motion units (105) by the first driving device is performed independently, but they move away from each other or move closer to each other and cross each other in response to the movement of both left and right feet during walking. The first driving device (105c) causes linear reciprocating motion and rotational motion of the operating link (105b) and may include a plurality of driving devices. For example, the first driving device (105c) may have a combination of a linear motion driving device for controlling the linear reciprocating motion of the operating link or the entire walking motion unit (105) equipped with it, and a rotational or pivotal motion driving device for controlling the rotational motion of the operating link (105b).
[0081] An assembly is provided in front of the above-mentioned walking exercise unit (105) in which a saddle (104) and a safety bar (106) above it, or a fence unit (107) that supports the chest, etc. including the above, are combined as one unit. This assembly is installed via an up-down frame (103) with respect to a lift device (102). The lift device (102) is an assembly lifting and lowering device that adjusts the height of the assembly to match the patient's physical condition. A system status display unit (109) facing the patient is provided at the top of the lift device (102).
[0082] At the foremost part of the system (100) viewed by the patient, a lifting or fixed support column (113a) is installed, and a status display monitor (113) that displays the operating status of the entire system is coupled thereto.
[0083] FIG. 5 illustrates a scene in which a patient (1) trains while wearing an HMD (115) described below in a walking training system (100) according to the present disclosure. As shown in FIG. 5, for example, in the case of a severe patient, the patient (1) can perform walking training while sitting on a saddle (104) and holding a safety bar (106). Depending on the condition of the patient (1), for example in the case of a mild patient, the saddle (104) can be folded, and thus the patient (1) can proceed with walking training without relying on the saddle (104).
[0084] The above pedal (105a) is operated by the system operating device to force the patient to walk regardless of the patient's will. At this time, pressure sensors, etc. are installed on the saddle (104) and the pedal (105a) to detect the load placed on the saddle (104) and the load placed on the pedal during walking training. In particular, a plurality of pressure sensors are provided at the front and rear of the pedal to detect local pressure applied to the pedal, thereby detecting the degree of pressure or contact of the sole of the foot against the pedal.
[0085] FIG. 6 is a schematic excerpt perspective view of the walking movement unit (105) dynamically coupled to the main body base (101).
[0086] Referring to FIG. 6, the operating link (105b) is coupled to the main body base (101) via a first driving device (105c) to enable linear movement (LM) and rotational movement (RM1). According to one embodiment, the first driving device (105c) is a driving motor, and a separate reciprocating movement driving device may be connected to one side or the lower part thereof. Meanwhile, the pedal (105a) is coupled to the operating link (105b) via a second driving device (105d) to enable rotational movement (RM2) of a predetermined angle relative to the operating link (105b), and a band capable of binding the patient's foot or ankle may be installed therein.
[0087] Figure 7 illustrates a normal walking cycle, showing the posture of the foot, knee, and thigh during normal walking, and Figure 8 shows the regulations for each position in the walking pattern.
[0088] Referring to Fig. 7, in one gait cycle, the stance phase is the period when the foot is in contact with the ground, and the swing phase is the period when the foot is lifted off the ground.
[0089] There are three tasks within each walking cycle as follows.
[0090] 1. Weight Acceptance
[0091] This period consists of an initial contact phase (when the foot first touches the ground) and a loading response phase (the moment the sole of the foot steps on the ground).
[0092] 2. Single Limb Support
[0093] During this period, the sole of the foot touches the ground, the mid-stance is the moment the foot of the opposite leg lifts off the ground, and the heel is lifted while the opposite leg swings.
[0094] 3. Limb Advancement
[0095] This period is a period during which the other foot is placed, and includes a free swing in which the toes of the front foot lift off the ground as the back foot lifts off the ground, a mid swing in which the foot that previously touched the ground begins to lift while the feet are together, and a terminal swing in which the heel of the front foot begins to touch the ground while pushing off the back foot that touched the ground.
[0096] The walking pattern described above is a normal walking pattern, and patients are trained to obtain this normal walking pattern.
[0097] However, in the case of patients who have difficulty walking normally, the patient's feet may become discoordinated during walking training, failing to match each cycle. For example, during the mid-stance cycle, the heel of the back foot may remain stuck to the ground instead of lifting off the ground at the time when the heel of the back foot is supposed to lift. This is because the patient's body does not follow a normal walking pattern. The robot disclosed herein can forcibly bend the knee or ankle joints to match the patient's walking pattern during this period, thereby causing the heel of the back foot to lift off the ground. All such forced joint movements can be performed during the cycle, which allows the patient to forcibly adopt a walking pattern similar to a normal one, despite their own physical limitations during walking training.
[0098] Below, a system operating device for effectively, accurately, and efficiently performing gait training for the rehabilitation of a patient as described above is described.
[0099] The system operating device in the present disclosure controls the walking motion unit by means of hardware, while being largely based on software.
[0100] Such a system operating device operates based on hardware, specifically a computer system for the execution of software. Accordingly, the system operating device according to the present disclosure may have all or part of a general computer system. The software utilizes a plurality of artificial intelligence models to predict a patient's gait evaluation index from initial gait state information or gait training result information after gait training, establish a training plan for each individual patient based on the gait evaluation index, evaluate training results based on training result data according to the training results, and report training results based on the evaluation of training results.
[0101] According to the present disclosure, training of a general deep learning-based model is performed with each functional model, and training using the trained models and applying them to establish, execute, evaluate, and report on a patient's training plan, and in this process, the training plan for long-term training is updated.
[0102] FIG. 9 illustrates functional parts based on the software and hardware structure of a system operating device (111) according to the present disclosure.
[0103] As illustrated in FIG. 9, the system operating device (111) includes an input unit (111a), an indicator prediction unit (111b), a training planning unit (111c), a training progress unit (111d), a training evaluation unit (111e), and a reporting unit (111f), etc.
[0104] The above input unit (111a) includes at least a keyboard, or a keyboard, a webcam, and a communication unit, and an indicator prediction model (M1) that predicts the patient's gait evaluation indicators necessary for establishing a patient's training plan.
[0105] The above training planning unit (111c) includes a training planning model (M2) that plans training for individual patients based on patient symptoms by referring to the indicators predicted by the indicator prediction unit (111b).
[0106] The above training progress unit (111d) includes a training progress model (M3) that monitors the training status according to the training plan and provides guidance on the training status.
[0107] The above training evaluation unit (111e) includes an evaluation model (M4) that evaluates the results of training performed by the training progress unit (111d) and predicts future trends.
[0108] The above reporting unit (111f) includes a reporting model (M5) that reports the results of training obtained from the training evaluation unit (111e) in various forms to patients and training personnel.
[0109] In addition to the elements listed above, such system operating devices (111) may further include other additional software and / or hardware depending on the design of the system operating device (111).
[0110] FIG. 10 is a schematic flowchart of a training process by the system operating device (111) in an artificial intelligence-based walking training method according to the present disclosure, which is performed by the exercise system (100) equipped with the system operating device (111) described above.
[0111] Referring to Fig. 10, in the case of a new patient during the training process, the process proceeds to the next step S13 after undergoing the data preparation step of data input (S11) and initial gait evaluation index generation (S12). In the case of an already registered patient, the process proceeds to the above step S13 after verifying personal information and calling up the existing gait evaluation index stored in the system.
[0112] In step S13, a training plan is generated using initial or existing gait evaluation indicators.
[0113] In step S14, the patient's gait training is performed according to the above training plan.
[0114] In stage S15, intermediate gait evaluation indicators are generated based on the current gait training results.
[0115] In the S16 stage, the training results are analyzed by comparing the current gait evaluation indicators based on the current training results with the existing gait evaluation indicators.
[0116] Step S17 involves forming and presenting various types of reports based on the analysis of training results. After completing this step, a decision is made as to whether to continue the training, and the training is either continued or terminated. If continuing the training is selected, the process proceeds to Step S13 or S14, depending on whether the training plan is updated.
[0117] Figure 11 is a flowchart of a more specific training process for new training patients.
[0118] Each step schematically described in FIG. 11 may be used to perform gait training on a patient once or several times by using, for example, five types of models (M1, M2, M3, M4, M5) in a stepwise manner. According to other embodiments, some of the models may be excluded.
[0119] When training begins, new patients proceed from the process of S11 below, and already registered patients proceed from the process of creating and updating the training plan in step S13 below after verifying and selecting their own information. That is, as already mentioned in the description of Fig. 7, processes S11 and S12 are performed only once initially, and thereafter, when training begins, patients whose personal information has been identified proceed from step S13.
[0120] Step S11: An initial test data set for gait training for a patient is entered.
[0121] S12: The input initial test data set is input into the gait evaluation index prediction model (M1) to generate an initial gait evaluation index.
[0122] S13: The above gait evaluation indicators are input into the training plan generation model (M2) to generate a training plan. In this case, if this process is not an initial training plan generation process but a generation process during the iterative training process, the training plan from the preceding process is updated.
[0123] S14: When a training plan is generated or established, walking training for the patient is performed using it. This training is carried out by the walking exercise unit that the patient is riding on. During the training process, monitoring of the training status is performed, and depending on the training status, information is provided to the patient and encouraging sound information and / or text information is presented. In the monitoring process, the patient's walking video is acquired in real time through a webcam or Kinect sensor included as a component of the detection unit, and from this, it is possible to extract skeleton display, joint angle information, and walking trajectory by extracting the patient's major joint positions, such as key points, using a machine learning-based model.
[0124] This information may be presented through a status display monitor (113, FIG. 4), or, on the other hand, through an HMD (115, FIG. 4) worn by the patient. The model M3 applied here may be a conversational model, such as a chatbot, for example.
[0125] S15: In this step, an intermediate gait evaluation index is generated based on the current gait training results. In this step, the gait evaluation index prediction model (M1) used in the initial gait evaluation index prediction above is applied.
[0126] S16: In this step, the training results are analyzed by comparing the current gait evaluation indicators based on the current training results with the existing gait evaluation indicators. At this time, a trained training analysis model (M4) is used to analyze the training results.
[0127] S17: This stage involves forming and presenting various forms of reports based on the analysis of training results. In this stage, a model trained to generate reports from training results, such as a multimodal model (M5), may be applied.
[0128] After this stage, a decision is made as to whether to continue training. If continuing training is selected, the process proceeds to either stage S13 or S14. If the current training results require an update to the training plan, the process proceeds to S13; otherwise, if training is resumed with the current plan, the process proceeds directly to S14, which is the gait training course.
[0129] The aforementioned artificial intelligence models (M1, M2, M3, M4, M5) included in each part of the system operating device are described below.
[0130] I. 모델 1 (M1)
[0131] Model 1 (M1) is a model that predicts a patient's gait evaluation index and may be either a Recurrent Neural Network (RNN) model or a Long Short-Term Memory Model (LSTM).
[0132] One embodiment according to the present disclosure applies an LSTM. The LSTM is a type of recurrent neural network (RNN) and is a model designed to solve long-term dependency problems, having a structure capable of effectively learning sequence information of data.
[0133] In training such an LSTM, dropout and early stopping techniques are applied. Dropout is a regularization technique designed to prevent overfitting of a neural network; it randomly deactivates (drops) some neurons during the training process to prevent the model from overfitting to a specific pattern, and this technique can be applied to the embodiments of the present disclosure. Early stopping is a technique designed to prevent overfitting while the model is training, and it terminates training early if the performance of the validation data does not improve.
[0134] FIG. 12 illustrates the configuration of a data set from a patient to generate an initial gait evaluation index by the gait evaluation index prediction model (M1).
[0135] Basic input data is basic information entered into the exercise device (100) at the medical clinic, and this must be managed in compliance with personal information protection procedures.
[0136] This dataset includes patient information data, such as patient personal information, patient medical condition information, gait evaluation indicators (FAC, Functional Ambulation Categories) determined by medical staff, and BBS (Berg Balance Scale).
[0137] The above patient personal information includes age, gender, height, weight, leg length, foot size, and medical condition, and the medical condition information includes patient information such as the medical condition and reference details for each medical condition, date of onset of the medical condition, and affected areas of the lower extremities (both feet, left foot, right foot).
[0138] In addition, it includes the results of medical diagnosis, such as clinical evaluation indicators utilizing clinical assessment tools like FAC based on medical diagnosis, and pre-gait training data.
[0139] Meanwhile, as gait information data, the data set for measuring gait training results during walking on flat ground before boarding the gait rehabilitation training robot includes information on upper body tilt and lower limb joint angles during walking. Additionally, as the data set for measuring gait training results after boarding the gait rehabilitation training robot includes the data set for training results of boarding the exercise system of the present disclosure. This includes information on average Ground Reaction Force (GRF), Center of Pressure (COP), Body Weight Support (BWS), lower limb joint angles, and left-right balance for each gait cycle (0 to 100%) during walking.
[0140] Meanwhile, as optional input data, information provided by the hospital or patient includes physician comments, gait images, gait training videos (MOV), and training measurement information, such as GRF information measured while walking on a GRF measurement pad, COP-based average (Butterfly Diagram) image information during walking, and upper body tilt information.
[0141] Therefore, Model 1 (M1) is trained to receive the above data as input and output a gait evaluation index.
[0142] FAC, one of the input data, has six indicators and can be summarized as follows.
[0143] FAC 0: Inability to walk - Inability to walk.
[0144] FAC 1: A condition in which walking ability is present but walking capacity is very limited and walking assistance is required.
[0145] FAC 2: Weak walking ability and a condition requiring walking assistance.
[0146] FAC 3: Walking ability has been partially recovered, and some walking assistance is required.
[0147] FAC 4: Walking ability has been almost fully recovered, and independent walking without assistance is possible.
[0148] FAC 5: A state in which functional walking ability is fully recovered and the patient can walk independently without walking assistance.
[0149] Meanwhile, the aforementioned BBS consists of several items designed to evaluate the ability to maintain balance while performing various activities. These items include standing posture, the ability to transition from a seated to a standing posture, the ability to maintain a seated to a standing posture, and the ability to turn the head while standing; each item is rated on a scale of 0 to 4 points, with a total score of 56 points, where a higher score indicates better balance ability.
[0150] Model 1 predicts the gait evaluation indicators of patients undergoing gait rehabilitation training by applying grades that reflect gait evaluation indicator standards (FAC, BBS, etc.) applied in the medical field, and this can be a Recurrent Neural Network (RNN) model or a Long Short-Term Memory (LSTM) model.
[0151] As a model training method to improve judgment resolution in preparation for FAC (applying Grade 6), time-series walking training result data (3 minutes of training each for walking on flat ground and stairs) obtained from a group of non-disabled people (multiple people) using the present initiation exercise system is set as a supervised learning model group (correct answer: 100%), and walking grade can be determined by quantifying the time-series walking training result data of people with walking disabilities using a time-series information-based anomaly detection model (Anomaly Detection model) to a value of 0 to 100%.
[0152] The output data of the model may vary depending on the design. However, when FAC ratings are applied, the determination can be made by comparing the measurement value (%) reflected in the time-series information-based anomaly detection model with the measurement value of a disabled person (100%) corresponding to the measurement value of a non-disabled person as follows.
[0153] yes)
[0154] 0 ~ 20% = FAC 0
[0155] 21 ~ 40% = FAC 1
[0156] 41 ~ 60% = FAC 2
[0157] 61 ~ 70% = FAC 3
[0158] 71 ~ 80% = FAC 4
[0159] 81 ~ 100% = FAC 5
[0161] In cases where the FAC grade is not applied, the gait grade can be determined more precisely by directly applying the measurement value (%) reflected in the Anomaly Detection Model, which is based on time-series information with the measurement value of a disabled person corresponding to the measurement value of a non-disabled person (100%).
[0163] The above-mentioned model is described below as an example.
[0164] Figure 13 illustrates a code snippet for the initialization of the ImprovedLSTMModelWithDropout class, which improves the Long Short-Term Memory (LSTM) model by adding a Dropout layer. This code was written using PyTorch.
[0165] The ImprovedLSTMModelWithDropout class defines an LSTM model by inheriting from nn.Module.
[0166] This class has initialization methods “__init__” and “forward”, and the parameters of the initialization methods are as follows.
[0167] - input_size: Number of features in the input data
[0168] - hidden_size: Size of the LSTM hidden state
[0169] - num_layers: Number of LSTM layers
[0170] - output_size: Number of features in the output data
[0171] - dropout: dropout rate
[0172] And, his instance variables are as follows.
[0173] - self.lstm: Defines the LSTM layer
[0174] - self.fc1: Definition of the first fully connected layer
[0175] - self.dropout: defines the dropout layer, and
[0176] - self.fc2: Defines the second fully connected layer
[0178] Figure 14 illustrates a code snippet defining the forward propagation process of a model.
[0179] This includes a series of steps that process input data to generate final predicted values, and each step transforms the input data to progressively generate the model's predicted values.
[0180] Meanwhile, the main processing steps of the forward method are as follows.
[0181] (1) Forward Step 1:
[0182] h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
[0183] c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(x.device)
[0184] - At this stage, the initial hidden state (h0) and cell state (c0) are defined. Objects h0 and c0 are tensors representing the initial hidden state and cell state of the LSTM network, respectively; these are state values set before the network processes the first sequence, providing the initial state for each layer and each element of the LSTM. Additionally, a tensor is created with all elements initialized to zero using the torch.zeros function. These objects, or tensors, have a specific batch size (x.size(0)) and the size of the hidden state (self.hidden_size), and their dimensions are set according to the number of network layers (self.num_layers). Furthermore, the created tensor is moved to the same device as the input data x (e.g., CPU or GPU) using the to(x.device) method.
[0185] (2) Forward Phase 2:
[0186] out = self.lstm(x, (h0, c0))
[0187] In this step, input data x and initial state (h0, c0) are passed to an LSTM layer for processing. Here, x is the input data tensor (batch size, sequence length, number of input features), and out is the output of the LSTM layer, which is a tensor of the form (batch size, sequence length, size of hidden state).
[0188] (3) Forward Step 3:
[0189] out = self.fc1(out[:, -1, :])
[0190] out[:, -1, :]: Uses only the output of the last time step of the LSTM. This output passes through the first fully connected layer (self.fc1).
[0191] (4) Forward Step 4:
[0192] out = torch.relu(out)
[0193] The ReLU activation function is applied to the output of the first fully connected layer. This adds non-linearity to improve the model's expressiveness.
[0194] (5) Forward Step 5:
[0195] self.dropout(out)
[0196] Apply dropout to randomly disable some neurons.
[0197] (6) Forward Step 6:
[0198] self.fc2(out)
[0199] The output of the dropout layer is passed to the second fully connected layer (self.fc2) to generate the final prediction.
[0200] (7) Forward Step 7:
[0201] return out
[0202] Only at this stage is the model's final predicted value returned.
[0204] As such, the forward method defines the process in which input data is transformed into a final output value by passing through an LSTM, a fully connected layer, an activation function, and dropout, and each step plays the role of gradually transforming the data to generate a final prediction.
[0205] Figure 15 illustrates a code snippet for the definition of an Early Stopping class.
[0206] This class is used to prevent overfitting and support efficient learning in machine learning models during the training process, and its main components and functions are as follows.
[0207] (1) Initialization method (__init__)
[0208] def __init__(self, patience=15, delta=0.0005):
[0209] - patience (int) : An instance variable that sets how many epochs to wait when performance does not improve, with a default value of 15 passed as a parameter.
[0210] - delta (float) : An instance variable that stores the minimum change amount recognized as a performance improvement. This value defines how much the validation loss must be improved compared to the previous minimum loss, and the default value is 0.0005, which is also passed as a parameter.
[0211] - best_score (float) : This is an instance variable that stores the best observed performance score (the negative value of the validation loss).
[0212] - early_stop (bool) : It is an instance variable that stores a flag indicating whether to terminate early.
[0213] - val_loss_min (float) : This is an instance variable that stores the observed minimum validation loss.
[0214] - counter (int) : It is an instance variable that stores the number of epochs at which performance is not improved.
[0215] (2) Call method (__call__)
[0216] def __call__(self, val_loss, model):
[0217] - val_loss (float) : This is the validation loss at the current epoch.
[0218] - model (torch.nn.Module): A PyTorch model object being evaluated.
[0219] This method evaluates the model's validation loss after each epoch to check for early stopping conditions. If performance does not improve by more than delta compared to the previous best, the counter is incremented; when this counter reaches patience, the early_stop flag is set to True to stop training. However, if performance improves, the counter is reset and the model's state is saved as a checkpoint.
[0221] (3) Checkpoint saving method (save_checkpoint)
[0222] def save_checkpoint(self, val_loss, model):
[0223] This method saves the model's state to a file (checkpoint.pt). It is used when you want to reload the model later or maintain a state with better performance, and updates val_loss_min with the latest validation loss value.
[0224] The main purpose of this early stopping class is to facilitate efficient learning and prevent overfitting, thereby reducing unnecessary training time and helping to maintain the model's generalization ability. This early stopping is one of the important techniques in many machine learning scenarios, and is particularly useful in deep learning models that require high computational costs.
[0225] Figure 16 illustrates a code snippet for the definition of train_model, a function for training a model.
[0226] def train_model(model, X, y, num_epochs=100, learning_rate=0.0001):
[0227] The `train_model` function is used to train a model and takes `model` (PyTorch model object to train), `X` (input data tensor), `y` (output data tensor), `num_epochs` (number of epochs to train, default 100), and `learning_rate` (learning rate, default 0.0001) as parameters.
[0228] (1) Loss function: Mean Squared Error (MSE)
[0229] criterion = nn.MSELoss()
[0230] - A loss function is defined. Here, the Mean Squared Error (MSE) loss function is used. MSE is the average of the squared differences between predicted and actual values, and is primarily used in regression problems.
[0231] (2) Create Optimizer Class
[0232] optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
[0233] - Create an Adam optimizer object to update the model's weights using the torch.optim.Adam class. The optimizer adjusts the model's weights along the gradient using the learning rate learning_rate.
[0234] (3) Creating an early termination class instance
[0235] early_stopping = EarlyStopping(patience=15, delta=0.0005)
[0236] - Create an instance of the EarlyStopping class for early termination. patience is the number of epochs to wait even if the loss does not improve, and delta is the minimum change considered as an improvement.
[0237] (4) Learning loop
[0238] for epoch in tqdm(range(num_epochs), desc="Training Progress"):
[0239] - This is a loop where training takes place. It repeats a total of num_epochs and uses the tqdm library to visually display the training progress.
[0240] (4.1) Set the model to training mode
[0241] model.train()
[0242] - Switch the model to training mode. This is necessary to allow layers such as dropout or batch normalization to behave differently in training mode.
[0243] (4.2) Generating Predicted Values
[0244] outputs = model(X)
[0245] - Input data X is passed to the model to generate predicted values. outputs are the model's prediction results.
[0246] (4.3) Optimizer Gradient Initialization
[0247] optimizer.zero_grad()
[0248] - Initializes the optimizer's gradient buffer, which is necessary to initialize the gradient before calculating the gradient through backpropagation.
[0249] (4.4) Loss Calculation
[0250] loss = criterion(outputs, y)
[0251] - The loss between the predicted values (outputs) and the actual values (y) is calculated. Here, the loss value is calculated using the MSE loss function.
[0252] (4.5) Calculate Loss Gradient
[0253] loss.backward()
[0254] - The gradient of the loss is calculated through backpropagation, and the gradient of each parameter is calculated at each step.
[0255] (4.6) Weight Update
[0256] optimizer.step()
[0257] - Update the model's weights using the optimizer based on the calculated gradient.
[0258] (4.7) Save current loss value
[0259] val_loss = loss.item()
[0260] - The current loss value is converted into a scalar value and stored. This is used later as a criterion for early termination.
[0261] (4.8) Check Early Termination Conditions
[0262] early_stopping(val_loss, model)
[0263] - Check the early stopping condition using the loss value of the current epoch. The early_stopping object tracks the number of epochs without improvement, and stops training the model when the condition is met.
[0264] (4.9) Output early termination message
[0265] if early_stopping.early_stop: print("\nEarly stopping")
[0266] - At this stage, if early_stopping.early_stop is True, an early termination message is printed, the loop is exited, and training is stopped.
[0267] (5) Output learning completion message
[0268] print('모델 학습 완료')
[0269] - Finally, output a message indicating that the learning is complete.
[0271] The code snippet described step-by-step above implements a basic framework for training a PyTorch model with early stopping. The use of the MSE loss function and the Adam optimizer is common in regression tasks, and the early stopping mechanism helps prevent overfitting by stopping training when the validation loss is no longer significantly improved.
[0272] Figure 17 illustrates a code snippet for a data evaluation function that evaluates new data from a patient undergoing gait training to predict the patient's FAC.
[0273] The function shown in the example uses a PyTorch model to generate predicted values for new data and calculates similarity by comparing them with actual values.
[0274] (1) Data Load:
[0275] def evaluate_new_data(file_path, model, data_mean, data_std, device):
[0276] The function evaluate_new_data reads a CSV file from file_path, selects only the necessary columns, and converts them into a DataFrame. The necessary columns are, for example, left Current Gait Cycle Rate (p_leftCurrentGaitCycleRate), left Current Ground Reaction Force (p_leftCurrentGRF), right Shape Gait Cycle Rate (p_rightCurrentGaitCycleRate), and right Current GRF (p_rightCurrentGRF).
[0277] # Data Preprocessing:
[0278] new_data['p_leftCurrentGaitCycleRate'] =
[0279] new_data['p_leftCurrentGaitCycleRate'].astype(int)
[0280] new_data['p_rightCurrentGaitCycleRate'] =
[0281] new_data['p_rightCurrentGaitCycleRate'].astype(int)
[0282] Convert p_leftCurrentGaitCycleRate and p_rightCurrentGaitCycleRate to integers and then perform normalization.
[0283] (2) Prepare model input data:
[0284] The input data (X_new) and output data (y_new) are separated from the normalized data. The input data includes p_leftCurrentGaitCycleRate and p_rightCurrentGaitCycleRate, and the output data includes p_leftCurrentGRF and p_rightCurrentGRF.
[0285] (2) 모델 예측 :
[0286] The input data is converted into a PyTorch tensor and passed to the model to generate predictions. The model is set to eval mode, and gradient calculation is disabled using torch.no_grad().
[0287] (3) Calculate result:
[0288] The difference between the predicted value and the actual value is calculated in percentages to obtain the average error rate. Based on this, the similarity is calculated.
[0289] (4) FAC rating determination:
[0290] The patient's FAC grade is determined based on similarity. For example, if the similarity is 0–40%, the rules for FAC 1, etc. are followed.
[0291] Figure 18 illustrates a code snippet for the execution of the above LSTM model.
[0292] (1) Specify necessary columns and set file paths
[0293] In this process, the required data columns (required_columns) and file paths (file_path) are specified first.
[0294] - file_paths: Paths to multiple CSV files to be used for model training
[0295] - new_data_path and eval_data_path: Paths for new data and evaluation data.
[0296] - model_save_path: Specifies the path to the checked-point file of the trained model.
[0297] (2) Data Loading and Preprocessing
[0298] combined_data = load_and_combine_csv(file_paths, required_columns)
[0299] X, y, data_mean, data_std = preprocess_data(combined_data)
[0300] - load_and_combine_csv: A function that reads and combines data from multiple CSV files
[0301] - preprocess_data: Preprocesses combined data to return input data X, output data y, data mean data_mean, and data standard deviation data_std.
[0302] (3) Processor (GPU or CPU) usage settings
[0303] device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
[0304] - In this process, release the GPU memory, - set the device to cuda if the GPU is available, or cpu otherwise, and then transfer the data to the set device.
[0305] (4) Model initialization and training
[0306] An improved LSTM model with dropout is created using the ImprovedLSTMModelWithDropout class, and training of the model using training data is performed using the train_model function. In this embodiment, training is performed for 100 epochs, and the learning rate is set to 0.0001.
[0307] (5) Data evaluation
[0308] At this stage, a data evaluation function (e.g., the evaluate_new_data function described earlier) evaluates the data, and the results are stored in the similarity (similarity_percentage) and FAC grade (fac_grade) variables.
[0309] In this step, data evaluation is performed by a data evaluation function, for example, the function evaluate_new_data described earlier, and the results are stored in the similarity_percentage and fac_grade variables.
[0310] This code snippet loads and preprocesses data from a CSV file containing information used for FAC evaluation of a given patient's gait training, trains an LSTM model, evaluates new data using the trained model, and determines the FAC grade by calculating the similarity between the predicted result and the actual result. This can be used for evaluating initial data for a patient's gait training, such as gait analysis, or for evaluating the patient's advanced FAC during subsequent gait training.
[0312] II. 모델 2 (M2)
[0313] Model 2 is an AI agent that automatically generates a gait training plan for a patient symptom-specific gait rehabilitation training robot in an artificial intelligence-based robot-assisted orthopedic exercise system according to the present disclosure, and this model may be, for example, a Transformer model, and according to other embodiments, an LSTM or a GRU (Gated Recurrent Unit) may also be applied.
[0314] The input data for this Model 2 includes 1) patient information, 2) prior gait training result data, and 3) gait training standard protocol and training plan definition data.
[0315] The above patient information includes gender, weight, foot size, year of birth, and the FAC (Gait Assessment Criteria) grade assigned to the patient, which were entered when the patient was registered in the training system. The above prior training result data includes a dataset of training protocols such as climbing / descending stairs and climbing / descending inclines, training time, and training parameters such as stride length, stride height, initial contact angle, toe-off angle, ground reaction force, and gait cycle. Furthermore, the above gait training standard protocol and training plan definition data are data determined by medical staff.
[0316] Model 2 receives data such as the above and currently determined patient's FAC, patient information, and current training result information as input, and derives (outputs) a training plan, for example, training protocols such as walking on flat ground, climbing / descending stairs, climbing / descending slopes, training time, training parameters, etc., and this may be a multi-input, multi- or single-output model.
[0317] According to another embodiment, the input is current gait evaluation indicators and patient information, and the output is a standard training model group (Classification); in yet another embodiment, the input is current gait evaluation indicators, patient information, and training information, and the output may be patient-tailored training plan information.
[0318] When applying an AI autonomous judgment training plan, the output data of Model 2 is a Training Plan Definition Dataset that reflects information on previously performed training results for each patient. Modification of the real-time training plan, which reflects information on the training status during training for each patient, includes changes to training time, training parameters, and training protocols, as well as settings for robot stop / move.
[0319] The M2 model for generating AI autonomous judgment training plans is an AI agent that predicts training parameters using patient gait training data, and like the aforementioned M1 model, it can be generated and trained using Python and PyTorch based on it. To this end, multiple libraries including PyTorch, as listed below, can be imported.
[0320] - import os : Module for file and directory operations
[0321] - import torch: A library for basic tensor operations and building neural networks
[0322] - import torch.nn as nn: Neural network module (e.g., layer definition)
[0323] - import torch.optim as optim: Optimization algorithm for model training (e.g., Adam, SGD, etc.)
[0324] - from torch.utils.data import DataLoader, TensorDataset: Module for data processing (batch generation, etc.)
[0325] - import pandas as pd : Library for creating and manipulating DataFrames
[0326] Python code that imports the above library sequentially includes the following steps for declaring and training the M2 model.
[0327] (1) Data input and data frame creation
[0328] Figure 19 is a code snippet of the entire Python code for creating an M2 model, specifically regarding data input for training the model and the creation of a data frame that accumulates the data.
[0329] The above M2 model is an instance of a Transformer model that utilizes PyTorch's neural network module, nn, to predict training parameters using patient gait training data. As described sequentially below, the input data is provided in the form of multiple CSV files; this input data undergoes a preprocessing step, is combined into a 2D data frame, and is then used as training data.
[0330] (1.1) Setting the directory path for CSV files
[0331] csv_directory = 'train_data_csv'
[0332] (1.2) List Initialization
[0333] all_dataframes = []
[0334] (1.3) Read all CSV files in the directory and combine them into a single DataFrame
[0335] for filename in os.listdir(csv_directory):
[0336] if filename.endswith('.csv'):
[0337] file_path = os.path.join(csv_directory, filename)
[0338] df = pd.read_csv(file_path)
[0339] all_dataframes.append(df)
[0340] (1.4) Combine all data into a single dataframe
[0341] combined_dataframe = pd.concat(all_dataframes, ignore_index=True)
[0343] (2) data preprocessing and tensor conversion
[0344] (2.1) The definition of a term
[0345] relevant_columns = [
[0346] 'p_patientGender', 'p_weight', 'p_footSize', 'p_patientBirthYear', 'p_fac',
[0347] 'p_trainingCurrentCadance', 'p_strideLength', 'p_strideHeight',
[0348] 'p_initialContactAngle', 'p_toeOffAngle',
[0349] 'p_leftCurrentGRF', 'p_rightCurrentGRF',
[0350] 'p_leftCurrentGaitCycleRate', 'p_rightCurrentGaitCycleRate'
[0351] ]
[0352] As defined in the array definition statement above, the input data includes patient information such as gender 'p_patientGender', weight 'p_weight', foot size 'p_footSize', year of birth 'p_patientBirthYear', and FAC 'p_fac', and prior training result data such as cadence 'p_trainingCurrentCadance', stride length 'p_strideLength', stride height 'p_strideHeight', initial contact angle 'p_initialContactAngle', toe-off angle 'p_toeOffAngle', current left foot GRF 'p_leftCurrentGRF', right foot GRF 'p_rightCurrentGRF', left foot gait cycle rate 'p_leftCurrentGaitCycleRate', and left foot gait cycle rate 'p_rightCurrentGaitCycleRate'.
[0353] (2.2) Copy DataFrame with Selected Columns
[0354] combined_dataframe_relevant = combined_dataframe[relevant_columns].copy()
[0355] (2.3) Data Preprocessing
[0356] combined_dataframe_relevant['p_patientGender'] = combined_dataframe_relevant['p_patientGender'].map({'Male': 0, 'Female': 1})
[0357] During the preprocessing process, male "male" is converted to "0" and female "female" to "1", and missing values (NaN, Not a Number) are replaced with the average value of each column by the method "fillna".
[0358] combined_dataframe_relevant.fillna(combined_dataframe_relevant.mean(), inplace=True)
[0359] (2.4) Convert to PyTorch Tensor "X" and Create Tensor "Y"
[0360] mean_target = combined_dataframe_relevant[['p_trainingCurrentCadance', 'p_stepLength', 'p_stepHeight', 'p_initialContactAngle', 'p_toeOffAngle']].mean().values
[0361] The preprocessed data is converted into a PyTorch tensor "X", then the mean_target of the target parameters to be predicted is calculated, and a tensor "Y" with the same target value for all samples is generated.
[0362] Y = torch.tensor([mean_target] * len(X), dtype=torch.float32)
[0364] (3) Definition of Transformer-based model classes
[0365] Figure 20 is a code snippet of the entire Python code, including the declaration of the Transformer model, training setup, model creation, etc.
[0366] class TransformerModel(nn.Module):
[0367] Inheriting from PyTorch's nn.Module TransformerModel A class is defined.
[0368] TransformerModel The initialization method __init__ is the constructor method of the class, which defines the structure of the model and each layer, and is the initialization method of the parent class. super(TransformerModel, self).__init__ It performs the initialization function of the base class by calling it, and its parameters are as follows.
[0369] - input_dim : Specifies the dimension (number of features) of the input data.
[0370] - output_dim : Specifies the output dimension of the model (number of parameters to predict).
[0371] - num_heads : Specifies the number of heads in multi-head attention, and the default value is 1.
[0372] - num_layers : Specifies the number of Transformer encoder layers, and the default value is 2.
[0373] - hidden_dim : Sets the dimension of the forward network, and the default value is 128.
[0375] 클래스 인스턴스 nn.TransformerEncoderLayer 는 Define self.encoder_layer, a Transformer encoder layer provided by PyTorch, as follows.
[0376] self.encoder_layer = nn.TransformerEncoderLayer(d_model=input_dim, nhead=num_heads, dim_feedforward=hidden_dim)
[0377] The parameters of this instance are as follows.
[0378] - d_model=input_dim : It is the input dimension.
[0379] - head=num_heads : This is the number of heads in multi-head attention.
[0380] - dim_feedforward=hidden_dim : It is the dimension of the forward propagation network.
[0381] A fully connected layer (fc) is defined by the method nn.Linear, where the parameter input_dim is the input dimension, and output_dim 은 The final output dimension is the number of training plan parameters to be predicted.
[0382] Meanwhile, the above TransformerMode의 The forward method preprocesses the input data and returns it. To do this, a dimension is first added to the input data x, and the input x with the added sequence dimension is passed to the Transformer_encoder to convert it into an encoded output. Then, after the sequence dimension is removed from the encoded output x by the squeeze method, the transformed data x is returned as output through a fully connected layer.
[0383] (4) Training Settings
[0384] As shown in Fig. 20, the hyperparameters in the learning setup are set as follows.
[0385] - input_dim = X.shape[1] # Dimensions of input data (number of features)
[0386] - output_dim = 5 # Dimension of output data (number of parameters to predict)
[0387] - batch_size = 64 # Batch size (number of data samples to train at once)
[0388] - learning_rate = 0.001 # Learning rate (magnitude of change when updating weights)
[0389] - epochs = 100 # Number of training iterations
[0391] After this process, the dataset and data loader are created as follows.
[0392] - dataset = TensorDataset(X, Y) # Define dataset
[0393] - train_loader = DataLoader(dataset, batch_size=batch_size, shuffle=True) # Create data loader
[0394] - model = transformerMode(input_dim, output_dim): Create an instance of the M2 model
[0396] In the following process, the model to be trained, as well as the loss function and optimizer, are initialized. In this embodiment, the M2 model is initialized as a Transformer model, and then "MSE" is set as the loss function and "Adam" is set as the optimizer, after which the model is trained.
[0397] (5) Model Training
[0398] Figure 21 is a code snippet for the model's training and test data prediction.
[0399] As illustrated in FIG. 21, according to one embodiment of the present disclosure, the training of the model is repeated for 100 epochs predetermined in the above step.
[0400] - for epoch in range(epochs): Start a loop to repeat the entire learning process. Epochs define the number of times to repeat the learning.
[0401] - model.train() : Set the model to training mode. In this mode, specific layers such as dropout and batch normalization are enabled during training.
[0402] - running_loss = 0.0 : Initializes a variable to store the total loss of the current epoch, and uses this variable to calculate the average loss at the end of each epoch.
[0403] - for inputs, targets in train_loader: The train_loader loads input data and target data in batches. The data loader is responsible for dividing the dataset into batches and repeatedly fetching them.
[0404] - optimizer.zero_grad() : Initializes the optimizer's gradient. Since PyTorch accumulates the previous gradient by default, the gradient must be initialized for every batch.
[0405] - outputs = model(inputs): Input data is passed through the model to calculate predicted values (outputs), and the model's forward method is called in the step.
[0406] - loss = criterion(outputs, targets): Calculate the loss between the predicted value and the actual target value. Here, the criterion defines the loss function (e.g., MSELoss).
[0407] - loss.backward(): Backpropagation is performed based on the loss to calculate the gradient for each parameter of the model.
[0408] - optimizer.step() : Update the model parameters based on the gradient calculated using the optimizer.
[0409] - running_loss += loss.item(): The loss value of the current batch is accumulated and added to running_loss, which allows the average loss to be calculated at the end of the epoch. And,
[0410] - print(): For each epoch, output the average loss, but calculate the average by dividing running_loss by the length of train_loader, format it, and output it. For example, as the output for the 6th epoch out of a total of 100 epochs, if the real-time loss is 250.123 and the length of train_loader is 50, " Epoch [6 / 100], Training Loss: 5.0025" will be output.
[0411] The above model training process is performed iteratively, calculating and outputting the loss at each epoch. Here, the model parameters are updated using an optimizer and a loss function, and the progress of training can be monitored through the loss value.
[0412] (6) Predicting Test Data
[0413] As shown in Figure 21, the prediction of test data using the M2 model obtained through the above process is also performed on the input data of the CSV file.
[0414] - test_data = pd.read_csv('test.csv') : Read test data file
[0416] - test_data_relevant = test_data[relevant_columns].copy() : Select necessary columns from the data during test data preprocessing
[0417] test_data_relevant['p_patientGender'] = test_data_relevant['p_patientGender'].map({'Male': 0, 'Female': 1}): Convert gender data to numbers
[0418] test_data_relevant.fillna(test_data_relevant.mean(), inplace=True) : Handling missing values
[0420] X_test = torch.tensor(test_data_relevant.values, dtype=torch.float32): Convert test data to a PyTorch tensor
[0421] (7) Model prediction
[0422] model.eval() : Sets the model to evaluation mode (disables dropout, etc.)
[0423] with torch.no_grad(): Set to not calculate gradients
[0424] prediction = model(X_test).mean(dim=0).numpy(): Calculate the mean after calculating the predicted values
[0425] (8) Output prediction result
[0426] predicted_df = pd.DataFrame([prediction], columns=['p_trainingCurrentCadance', 'p_stepLength', 'p_stepHeight', 'p_initialContactAngle', 'p_toeOffAngle'])
[0427] print(predicted_df) : Converts the prediction results into a DataFrame and outputs them.
[0429] The model trained as described above is used as a model for establishing patient-customized training plans.
[0430] In the actual usage method, to establish a training plan by reflecting information on previously performed training results, the most recent data is extracted from the existing training data and prediction is performed using the M2 model, and the resulting values are input into the new training to proceed with training.
[0431] When establishing a training plan using the M2 model, if the patient has prior training information, the patient's prior training information is input into the M2 model to extract an autonomous training plan. Additionally, if necessary, the information is input into an Expert System to further improve reliability.
[0432] The Expert System includes a Reference Model trained according to theoretical standards. This Reference Model has the same structure as the M2 Model and outputs a dataset of standard gait training protocols and training plan definitions determined by medical professionals as a Reference Training Plan. This output is matched with training plan establishment information corresponding to patient information and assessed gait grade evaluation indicators. This Reference Model is a model trained on patient information (gender, weight, foot size, year of birth, FAC grade, etc.), training protocols, training time, training parameters, etc., set by medical staff.
[0433] This reference model can be applied as a model for determining the reliability of a pre-established AI autonomous judgment training plan model and the aforementioned M2 model. To this end, the patient's prior training information, which was input into the pre-established AI autonomous judgment training plan model and the aforementioned M2 model, is input into the reference model to extract a reference training plan.
[0434] This reference training plan is compared with the autonomous training plan output from the M2 model to determine reliability, and the autonomous training plan can be appropriately modified based on the degree of reliability. In actual training, either the reference training plan or the autonomous training plan may be selected. The aforementioned reference model is an optional element that can be applied as a means to more precisely compensate for or correct the autonomous training plan.
[0435] According to another embodiment, the autonomous training plan can be corrected based on the reference training plan. This correction may be performed by a final training plan model that outputs an optimal training plan from the output of the M2 model and the output of the reference model. This final training plan model can be trained using data obtained by comparing the aforementioned M2 model and reference model.
[0436] III. Model 3 (M3)
[0437] Model 3 may be a model that monitors and guides the training status during gait rehabilitation training, for example, a chatbot-type model. The input data for this model includes patient information, robot status information, and real-time input / output information during gait.
[0438] Figure 22 illustrates a graphic interface showing various parameters of gait training.
[0439] Real-time input / output information during gait training includes training protocols, training parameters, and training data detected in real-time at fixed step intervals (e.g., including oxygen saturation and heart rate).
[0440] Figure 23 is a butterfly diagram showing the average plantar pressure distribution of the patient during training as a result of the patient's gait training.
[0441] This training data includes at least one of the following information regarding a patient undergoing gait training: Ground Reaction Force (GRF), Body Weight Support (BWS), upper body inclination or tilting, joint angle, cadence, stride length, ankle angle, oxygen saturation data, and heart rate.
[0442] As a separate option, if a front camera, such as a webcam, is installed, the patient's facial expressions, voice and gesture information, as well as the therapist's voice and gesture information, can be acquired and used as input data.
[0443] The output data of the model may be state feedback during training and training parameter modification values. The state feedback may include acoustic feedback such as artificial intelligence voice or beep sound, or status message output to a user interface, for example, a virtual reality HMD, such as when a patient boards, sound or text such as “Welcome, Mr. / Ms. OOO,” “Your training status is very good today,” “Press your left foot more,” “Straighten your upper body,” “You have improved compared to yesterday,” “Keep going,” or “Your heart rate is rising.” According to another embodiment, a request for a real-time change of training parameters may be made through the user interface based on the results of monitoring the training status.
[0444] This Model 3 can be trained with a Vision-Language Model (VLM), which enables the performance of training support agent functions through real-time interaction with patients / therapists, and can provide feedback on training status information by comparing and judging sequential pre-training results and real-time training results during training.
[0446] IV. 모델 4 (M4)
[0447] Model 4 is a model that evaluates training results and predicts future trends. The input data can be past training results data and current training results data. In this case, the output data is an analysis of the trends of the previous training results and current training results data and future predictions.
[0448] This Model 4 is a time-series information-based anomaly detection model trained to predict future training by analyzing current training results or analyzing the trends of results following repeated training by analyzing previous training results and current training results.
[0449] This Model 4 outputs a graph showing the average trend line of past training, the average trend line of current training, and the predicted trend analysis graph of future training. To do this, it analyzes the trend of current training results compared to previous training results and analyzes the predicted trend analysis of future training based on the analysis of past training results.
[0451] V. 모델 5 (M5)
[0452] Model 5 is an artificial intelligence model that supports smart training result reporting, and the input data is the training result data before and after and the report generation type, and the output is text, images, videos, etc.
[0453] The input report generation type is a prompt input based on a GPT (Generative Pre-trained Transformer) model that supports simple or multimodal generation types. Specifically, the output includes, for example, text in PDF document format, images of patient-specific training results and status (various backgrounds and progress status), or videos of patient-specific gait trajectories. Another output is video-based results based on the patient gait training analysis, such as virtual avatar movement, self-image movement, or output visualized as a butterfly diagram.
[0454] The "butterfly diagram" in a gait rehabilitation training system is a visual tool primarily used for gait analysis and evaluation. This diagram helps analyze the gait cycle, such as footprint analysis, gait cycle evaluation, measurement of gait effects, and analysis of left-right asymmetry of gait, by visualizing the pattern and pressure of the foot contacting the ground. The diagram is named "butterfly" because it typically depicts the shape of the footprint and pressure distribution similar to the shape of a butterfly's wings, and an example of a butterfly diagram is shown in Fig. 20.
[0455] In the case of multimodal-supported GPT, it will be possible to induce prompt-based report generation (text, images, videos) for the above work.
[0456] In addition, in addition to the outputs mentioned above, tasks such as training and report generation via voice output can be enabled. By doing so, it is expected that support for smart reporting on the patient's condition and results after training will lead to more intuitive and effective delivery of training results to the patient, as well as support for increased training achievement and satisfaction.
[0457] According to the present invention, it is possible to establish a patient-friendly gait training plan, and personalized and patient-friendly rehabilitation training can be efficiently carried out. Rehabilitation training using this system and method utilizes an artificial intelligence model at each stage to predict accurate gait evaluation indicators, automatically establish a training plan based thereon, and more accurately evaluate training results.
[0458] Rehabilitation training using these systems and methods utilizes artificial intelligence models at every stage to predict accurate gait evaluation indicators, automatically establish training plans based on this, and more accurately evaluate training results; thereby, compared to existing methods and systems, it may also reduce the need for therapist intervention according to medical staff prescriptions.
[0459] As described above according to exemplary embodiments, the exemplary embodiments of the present invention have been described in detail; however, a person skilled in the art to which the present invention pertains will be able to modify the present invention in various ways without departing from the spirit and scope of the present invention as defined in the appended claims. Therefore, future modifications to the embodiments of the present invention will not depart from the technology of the present invention.
Claims
Claim 1 An AI-based robot-assisted orthopedic exercise system comprising: a gait exercise unit having a pedal that forces the patient to walk by stepping on it, and one or more actuation links that drive the pedal; and a system operating device that controls the gait exercise unit according to a training plan established for each individual to conduct gait training of the patient standing on the pedal along a walking trajectory; wherein the system operating device comprises: an input unit that receives basic information including personal information and initial gait state information for each patient; an indicator prediction unit that receives the initial gait state information or the gait training results after gait training and determines a gait evaluation indicator; a training planning unit having an autonomous training planning model trained by machine learning or deep learning to establish a personalized training plan for the patient performed by the gait exercise unit based on the gait training results; and a training evaluation unit that evaluates the training results based on training result data according to the results of the patient's rehabilitation training performed by the gait exercise unit; wherein the gait exercise unit comprises: a base; and a left gait exercise unit installed on the left side of the base. An artificial intelligence-based robot-assisted orthopedic exercise system comprising: a right walking exercise unit installed on the right side of the base; and two left and right walking exercise units: each having a first driving device that reciprocates (LM) with respect to the base and causes rotational movement (RM1) of the operating link; and a second driving device installed at the end of the operating link and causes rotational movement (RM2) of the pedal; wherein the system operating device controls the walking exercise unit so that the pedal, which is moved by the first driving device and the second driving device, moves along a walking trajectory according to the training plan, thereby performing walking training for the patient. Claim 2 An artificial intelligence-based robot-assisted orthopedic exercise system according to claim 1, wherein the gait training results include at least one of cadence, stride length, stride height, initial contact angle, toe off angle, ground reaction force (GRF), and gait cycle rate. Claim 3 In claim 1, the training planning unit further includes a reference training planning model learned from patient information, training protocols, training time, training parameters, etc. set by a medical professional, and the training planning unit outputs a corrected or compensated final training plan based on the output of the autonomous training planning model and the reference training planning model, an artificial intelligence-based robot-assisted orthopedic exercise system. Claim 4 In claim 3, the training planning unit further comprises a final training planning model that outputs a final training plan based on the outputs of the autonomous training planning model and the reference training planning model, an artificial intelligence-based robot-assisted orthopedic motion system. Claim 5 An artificial intelligence-based robot-assisted orthopedic exercise system according to claim 1, further comprising a detection unit that detects at least one of the following information as training information: Ground Reaction Force (GRF), Body Weight Support (BWS), Upper body inclination or tilting, Joint angle, Cadence, Stride length, and Ankle angle of a patient undergoing gait training. Claim 6 An artificial intelligence-based robot-assisted orthopedic exercise system further comprising, in claim 5, a monitoring unit that monitors the state of rehabilitation training performed by the gait exercise unit according to the training plan, acquires information during the training, and generates training state information related to the patient's gait training therefrom. Claim 7 In claim 6, the monitoring unit presents to the patient training guidelines or recommended guidelines or current training status for improving gait related to the gait training, based on information during the training, an artificial intelligence-based robot-assisted orthopedic exercise system. Claim 8 An artificial intelligence-based robot-assisted orthopedic exercise system according to claim 6 or 7, wherein the monitoring unit further comprises a Head-Mounted Display (HMD) that presents necessary information related to the patient's walking exercise, training status information, or recommended guidelines related to training to the patient. Claim 9 An artificial intelligence-based robot-assisted orthopedic exercise system according to claim 1, further comprising a Head-Mounted Display (HMD) that presents to the patient at least one of necessary information related to the patient's walking exercise, training status information, or recommended guidelines related to training. Claim 10 In claim 1, the training evaluation unit comprises, as a training evaluation model, a time-series information-based anomaly detection model that analyzes current training results or analyzes the trend of results following repeated training by analyzing previous training results and current training results to perform future training prediction, an artificial intelligence-based robot-assisted orthopedic movement system. Claim 11 An artificial intelligence-based robot-assisted orthopedic exercise system according to claim 1, further comprising a reporting unit equipped with a reporting model that analyzes previous training results and current training results to generate report content in the form of text, images, and videos. Claim 12 The method comprises: an input unit receiving basic information including personal information and initial gait state information for each patient; an indicator prediction unit receiving initial gait state information or gait training result information after gait training and predicting a gait evaluation indicator for the patient; a training planning unit establishing a personalized training plan for the patient using an autonomous training planning model trained by machine learning or deep learning based on the gait evaluation indicator; a gait exercise unit having an end-effector type pedal forcing the patient's gait and one or more actuation links driving the pedal, performing rehabilitation training for the patient according to the training plan; and a training evaluation unit evaluating the training results based on training result data according to the training results; wherein the gait exercise unit comprises: a base; a left gait exercise unit installed on the left side of the base; and a right gait exercise unit installed on the right side of the base; and each of the two left and right gait exercise units comprises a first driving device that reciprocates (LM) relative to the base and causes rotational movement (RM1) of the actuation link. A method for providing AI-based robot-assisted orthopedic exercise, comprising: a second driving device installed at the end of the operating link to cause rotational movement (RM2) of the pedal; and a walking exercise unit such that the pedal, which is moved by the first driving device and the second driving device, moves along a walking trajectory according to the training plan to perform walking training for the patient. Claim 13 In claim 12, the gait training results include at least one of cadence, stride length, stride height, initial contact angle, toe off angle, ground reaction force (GRF), and gait cycle rate, an artificial intelligence-based method for providing robot-assisted orthopedic exercise. Claim 14 In claim 12, the indicator prediction unit is an indicator prediction model in which either a Recurrent Neural Network (RNN) model or a Long Short-Term Memory (LSTM) model predicts the patient's gait evaluation indicator, an artificial intelligence-based robot-assisted orthopedic exercise providing method. Claim 15 In claim 12, the training planning unit further comprises: an autonomous training planning model that establishes a personalized training plan for a patient trained by initial gait state information or gait training results after gait training; and a reference training planning model learned from patient information, training protocols, training time, training parameters, etc. set by a medical professional, wherein the training planning unit outputs a corrected or compensated final training plan based on the outputs of the autonomous training planning model and the reference training planning model, thereby providing an artificial intelligence-based robot-assisted orthopedic exercise method. Claim 16 In claim 12, the detection unit detects at least one of the following information as training information: Ground Reaction Force (GRF), Body Weight Support (BWS), Upper body inclination or tilting, Joint angle, Cadence, Stride length, and Ankle angle of a patient undergoing gait training. Claim 17 In claim 16, a method for providing artificial intelligence-based robot-assisted orthopedic exercise, wherein a monitoring unit monitors the state of rehabilitation training performed by the gait exercise unit according to the training plan, acquires information during the training, and generates training state information related to the patient's gait training therefrom. Claim 18 In claim 16, a method for providing artificial intelligence-based robot-assisted orthopedic exercise, wherein a monitoring unit presents to the patient training guidelines or recommended guidelines or current training status for improving gait related to the gait training, based on information during the training. Claim 19 A method for providing artificial intelligence-based robot-assisted orthopedic exercise according to claim 17 or 18, wherein the HMD (Head-Mounted Display) presents to the patient necessary information related to the patient's gait exercise, training status information, or recommended guidelines related to training. Claim 20 In claim 12, the time-series information-based anomaly detection model as the training evaluation model analyzes current training results or analyzes the trend of results following repeated training by analyzing previous training results and current training results to perform future training prediction, an artificial intelligence-based robot-assisted orthopedic exercise provision method. Claim 21 In claim 17, a method for providing artificial intelligence-based robot-assisted orthopedic exercises, wherein a reporting unit analyzes previous training results and current training results to generate report content in the form of text, images, and videos.
Citation Information
Patent Citations
Walking diagnosis support system, walking pattern generator, walking pattern generation program, and walking pattern generation method
JP2011019669A
Learning device, rehabilitation support system, method, program, and learned model
JP2021003541A
Medical rehab body weight support system and method with horizontal and vertical force sensing and motion control
KR1020170110136A
Method for estimating gait index of user, and wearable device and electronic device performing the same
KR1020240009528A
Method for real-time adjustment of gait training parameters
KR1020240013925A
