Artificial intelligence-based device and method for providing guide information about rehabilitation exercise

Abstract

The present disclosure relates to an artificial intelligence-based device and method for providing guide information about rehabilitation exercise. The device comprises: a communication module for communication with an external device; a storage module for storing at least one process for providing guide information about rehabilitation exercise on the basis of artificial intelligence; and a control module for performing, on the basis of the at least one process, an operation for providing the guide information about rehabilitation exercise on the basis of artificial intelligence. The control module: collects medical data and a medical image, captured prior to surgery, of a corresponding patient on the basis of the storage module and a separate database; pre-processes the medical image and then inputs the pre-processed medical image to a first pre-trained model based on artificial intelligence to predict the probability that post-surgical complications will occur; and generates prediction information on the basis of the predicted result and provides the prediction information to a medical staff terminal.

Description

Device and method for providing guidance information for artificial intelligence-based rehabilitation exercise

[0001] The present disclosure relates to a device and method for providing guide information, and more specifically, to a device and method for providing guide information for artificial intelligence-based rehabilitation exercise.

[0002] It is known that tens to hundreds of thousands of surgical procedures are performed nationwide annually for various orthopedic conditions, and the frequency is increasing exponentially with the aging population. Orthopedic surgery often requires joint protection for a certain period of time, using casts, splints, and braces. However, this can lead to joint stiffness. Even in surgeries that do not require joint protection, the body's joints rapidly develop inflammation, making postoperative stiffness more likely.

[0003] However, the treatment of joint stiffness, which inevitably accompanies orthopedic surgery, is entirely the responsibility of the patient, and patients learn exercise methods on their own through videos or pamphlets and then proceed with rehabilitation exercises on their own in a rule-of-thumb manner.

[0004] Additionally, if proper rehabilitation is not achieved due to persistent pain or uncertainty about whether the patient is progressing with the exercise correctly, a vicious cycle of visiting a hospital late to receive rehabilitation or manual therapy is inevitable.

[0005] Therefore, there is a need to develop technologies that enable patients to perform rehabilitation exercises more efficiently and systematically after orthopedic surgery.

[0006] The present disclosure provides an artificial intelligence-based surgical planning and rehabilitation exercise guide information providing device and method, which can predict complications of a surgical site after surgery for a patient based on artificial intelligence, and allow medical staff or medical institutions to perform surgery and post-surgical rehabilitation by considering the possibility of post-surgical complications when performing surgery or when the patient performs post-surgical rehabilitation exercises, thereby reducing the probability of complications occurring during the post-surgical rehabilitation period.

[0007] Meanwhile, the present disclosure provides an artificial intelligence-based rehabilitation exercise guide information providing device and method, which checks (self-checks) the individual patient's condition based on artificial intelligence and provides customized guide information corresponding to the condition, thereby enabling the patient to perform rehabilitation exercise more efficiently and systematically.

[0008] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] According to one aspect of the present disclosure for achieving the above-described technical task, a device for providing guide information for artificial intelligence-based rehabilitation exercise includes: a communication module for performing communication with an external device; a storage module for storing at least one process for providing guide information for artificial intelligence-based rehabilitation exercise; and a control module for performing an operation for providing guide information for artificial intelligence-based rehabilitation exercise based on the at least one process, wherein the control module collects medical data and medical images taken before surgery of a corresponding patient based on the storage module and a separate database, performs preprocessing on the medical images, and then inputs the preprocessed medical images into an artificial intelligence-based first pre-learning model to predict the probability of occurrence of complications after surgery, and generates prediction information based on the predicted result and provides the information to a medical staff terminal.

[0010] Meanwhile, a method for providing guide information for artificial intelligence-based rehabilitation exercise according to one aspect of the present disclosure may include a step of collecting medical data and medical images taken before surgery of a patient based on a storage module and a separate database; a step of performing preprocessing on the medical images; and a step of inputting the preprocessed medical images into an artificial intelligence-based first pre-learning model to predict the probability of occurrence of complications after surgery, and a step of generating prediction information based on the predicted result and providing the information to a medical staff terminal.

[0011] In addition, a computer program stored in a computer-readable recording medium for executing a method for implementing the present disclosure may be further provided.

[0012] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0013] According to the aforementioned problem solving means of the present disclosure, the probability of occurrence of complications in the surgical site of a patient is predicted based on artificial intelligence, and when medical staff or medical institutions perform surgery on the patient or perform rehabilitation exercises after the surgery on the patient, they consider the probability of occurrence of complications and proceed with the surgery and rehabilitation, thereby reducing the probability of occurrence of complications after the surgery.

[0014] Meanwhile, according to the present disclosure, based on artificial intelligence, the patient's individual condition is checked (self-checked) and customized guide information corresponding to the condition is provided, so that the patient can perform rehabilitation exercises more efficiently and systematically.

[0015] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0016] FIG. 1 is a diagram showing a system for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure.

[0017] FIG. 2 is a diagram showing the configuration of a device for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure.

[0018] FIG. 3 is a diagram showing a method for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure.

[0019] FIG. 4 is a diagram illustrating a method for performing learning on an artificial intelligence-based first pre-learning model according to one embodiment of the present disclosure.

[0020] FIG. 5 is a diagram illustrating a series of operations for preprocessing at least one medical image for each of the different patients collected according to one embodiment of the present disclosure.

[0021] FIG. 6 is a diagram illustrating a method for analyzing the patient's movements using an artificial intelligence-based second pre-learning model according to an embodiment of the disclosure.

[0022] FIG. 7 is a diagram illustrating a method for performing learning on an artificial intelligence-based second pre-learning model according to an embodiment of the disclosure.

[0023] FIG. 8 is a diagram illustrating a series of operations for preprocessing at least one rehabilitation exercise image for each of the different patients collected according to one embodiment of the present disclosure.

[0024] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the present disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined solely by the scope of the claims.

[0025] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present disclosure. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the specification, and "and / or" includes each and any combination of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present disclosure.

[0026] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those of ordinary skill in the art to which this disclosure pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0027] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general contents in the technical field to which this disclosure belongs or contents overlapping between embodiments are omitted. The term "part" or "module" as used in the specification means a hardware component such as a software or FPGA or ASIC, and the "part" or "module" performs certain roles. However, the "part" or "module" is not limited to software or hardware. The "part" or "module" may be configured to be in an addressable storage medium and may be configured to play one or more processors. Thus, as an example, the "part" or "module" includes components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and “sub-components” or “modules” may be combined into a smaller number of components and “sub-components” or “modules” or further separated into additional components and “sub-components” or “modules”.

[0028] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0029] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0030] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0031] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0032] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0033] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0034] The terms used in the following explanation are defined as follows.

[0035] In this specification, the first and second AI-based pre-learning models are deep learning-based predictive models, and each of them can be used to predict the re-rupture rate of a patient's surgical site before surgery or to predict the patient's condition after surgery, thereby generating guidance information. The deep learning method is not limited, and at least one method may be applied depending on the situation (need). Examples of AI algorithms that can be applied include, but are not limited to, a Recurrent Neural Network (RNN) or a Transformer, and other AI algorithms may also be applied.

[0036] Although the description is limited to the 'providing device (100)' in this specification, this may include various devices capable of performing computational processing, such as a medical staff terminal (including an administrator terminal, a medical institution terminal, etc.) for generating and providing guide information based on the condition of the patient predicted based on artificial intelligence. That is, the providing device (100) may further include a server, a computer, a server, and / or a portable terminal, or may be in the form of any one of them, but is not limited thereto.

[0037] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0038] The above server is a server that processes information by communicating with external devices, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0039] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, a smart phone, and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD).

[0040] In this specification, "medical data" includes the patient's personal information and medical records (including medical history, surgical history, prescription history, medical images, etc.), and may be collected from medical staff terminals (200) as well as devices (servers, terminals, etc.) connected to various management organizations, administrators, medical staff, medical institutions, etc. In this case, the medical data includes time series data for a certain period of time.

[0041] First, the present disclosure aims to predict the probability of post-operative complications in a patient's surgical site prior to surgery and provide the predicted information to a medical professional's terminal, thereby allowing the medical professional to consider the predicted information during surgery. Furthermore, after surgery, the patient can easily take a video using their own terminal, a user terminal, and analyze the range of joint motion and muscle strength using an AI-based pre-learning model. This analysis then compares the patient's progress and target values ​​for rehabilitation exercise, thereby generating and providing customized guidance information suitable for the patient.

[0042] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0043] FIG. 1 is a diagram illustrating a system for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure.

[0044] Referring to FIG. 1, a system (10) for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure (hereinafter referred to as the "providing system") may be configured to include a providing device (100), a medical staff terminal (200), and a user terminal (300). In this case, FIG. 1 illustrates one medical staff terminal (200) and one user terminal (300), but both may be configured to include at least one, and the number and type thereof are not limited.

[0045] First, the provision device (100) collects the patient's medical data and pre-operative medical images based on the storage module (130) and a separate database, performs preprocessing on the medical images, and inputs them into an artificial intelligence-based first pre-learning model to predict the probability of post-operative complications. Accordingly, the provision device (100) can generate prediction information based on the prediction results and provide it to the medical staff terminal (200).

[0046] In addition, when the provision device (100) receives a rehabilitation exercise image taken after the patient's surgery from the user terminal (300), it performs preprocessing on the rehabilitation exercise image and inputs it into an artificial intelligence-based second pre-learning model, thereby confirming rehabilitation exercise progress information according to the patient's range of joint motion and muscle strength. Accordingly, the provision device (100) can generate guide information based on the rehabilitation exercise progress information and provide it to the user terminal (300).

[0047] Meanwhile, the medical staff terminal (200) is a terminal possessed by medical staff, medical institutions, administrators, management institutions, etc., which records and manages medical data (including medical images) for at least one patient using the provision device (100), and can decide whether to provide the medical data at the request of the provision device (100), and can provide the medical data to the provision device (100) as well as various institutions as needed.

[0048] In addition, the medical staff terminal (200) can receive prediction information generated by the provision device (100) based on the medical data of the patient provided, and consider the prediction information when performing surgery on the patient.

[0049] Meanwhile, the user terminal (300) may refer to a terminal possessed by the patient as well as at least one guardian preset for the patient, and may refer to a user who has the authority to transmit the patient's rehabilitation exercise image to the provision device (100) or receive guide information from the provision device (100). That is, the user terminal (300) is a concept that includes a patient terminal and a guardian terminal, and may be pre-registered to enable access to the provision device (100) by the provision device (100) or the medical staff terminal (200).

[0050] Meanwhile, the medical staff terminal (200) and the user terminal (300) may be a computer, UMPC (Ultra Mobile PC), workstation, netbook, PDA (Personal Digital Assistants), portable computer, web tablet, wireless phone, mobile phone, smart phone, pad, smart watch, wearable terminal, e-book, PMP (portable multimedia player), portable game console, navigation device, black box or digital camera, other mobile communication terminal, etc., on which medical staff (including medical institutions, administrators, and management institutions) and / or users (including patients and guardians) can install and execute multiple application programs (i.e., applications) desired. That is, the medical staff terminal (200) and the user terminal (300) can be provided in various forms and are not limited thereto.

[0051] FIG. 2 is a diagram showing the configuration of a device for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure.

[0052] Referring to FIG. 2, a device (100) for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure (hereinafter referred to as “providing device”) may be configured to include a communication module (110), a storage module (130), and a control module (150).

[0053] The communication module (110) transmits and receives at least one piece of information or data with at least one device / terminal. Here, the at least one device / terminal may be a device / terminal that seeks to receive artificial intelligence-based predictive information or guide information, or a device / terminal that provides various data / information that enables the generation of such information, and the type and form thereof are not limited.

[0054] In addition, this communication module (110) can also perform communication with other devices, and transmit and receive wireless signals in a communication network according to wireless Internet technologies.

[0055] Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc., and the providing device (100) transmits and receives data according to at least one wireless Internet technology, including Internet technologies not listed above.

[0056] As for short range communication, short range communication can be supported using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. Such short range wireless communication network (Wireless Area Networks) can support wireless communication between a provision device (100), a medical staff terminal (200), and / or a user terminal (300). At this time, the short range wireless communication network can be a short range wireless personal area network (Wireless Personal Area Networks).

[0057] The storage module (130) can store data for at least one process (algorithm) for providing artificial intelligence-based prediction information or guidance information, or a program that reproduces the process. Furthermore, the storage module (130) can further store processes for performing other operations, but this is not limited thereto.

[0058] The storage module (130) can store medical data for at least one patient as well as various data supporting various functions of the provision device (100). The storage module (130) can store a plurality of application programs (or applications) run on the provision device (100), data for the operation of the provision device (100), and commands. At least some of these application programs can be downloaded from an external server via wireless communication. Meanwhile, the application programs can be stored in at least one memory provided in the storage module (130), installed on the provision device (100), and driven to perform operations (or functions) by at least one processor stored in the storage module (130) via the control module (150).

[0059] Meanwhile, at least one memory may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory may store information temporarily, permanently, or semi-permanently, and may be provided as a built-in or removable type.

[0060] The storage module (130) may build a database to provide artificial intelligence-based prediction information or guide information, or may be linked to a separate external server.

[0061] In addition to operations related to the application program, the control module (150) controls all components within the provision device (100) based on at least one processor to process input or output signals, data, information, etc., or executes commands, algorithms, and application programs stored in at least one memory to perform various processes, and can provide or process appropriate information or functions for providing artificial intelligence-based prediction information or guide information.

[0062] Specifically, the control module (150) collects the medical data of the patient and the medical images taken before surgery based on the storage module (130) or a separate database, performs preprocessing on the medical images, and then inputs the preprocessed medical images into an artificial intelligence-based first pre-learning model to predict the probability of post-surgical complications, and generates prediction information based on the prediction result and provides it to the medical staff terminal (200). Here, the medical images are images taken for the patient over a certain period of time (a preset period), and may be, for example, images acquired through X-ray, CT (Computer Tomography), MRI (Magnetic Resonance Imaging). At this time, the medical images may include images acquired for the patient over a certain period of time in chronological order.

[0063] To this end, the control module (150) can construct learning data for the first pre-training model and perform pre-training based on this data. At this time, the control module (150) can preprocess at least one medical image for each patient collected over a preset period of time and construct at least one pre-processed medical image as learning data. Thereafter, the learning data can be input into the first pre-training model to perform learning.

[0064] At this time, when performing preprocessing on at least one medical image, the control module (150) specifies at least one area in each of the at least one medical image, and labels at least one of whether a complication occurred after surgery and the time of occurrence of the complication for each of the at least one area previously specified based on medical data of each other patient.

[0065] Accordingly, the control module (150) can predict the probability of post-operative complications in the surgical site prior to surgery based on the medical image of the patient, and generate prediction information as a result of the prediction. Here, the surgical site refers to a site corresponding to each joint of the body, and may include the shoulder joint, elbow joint, hand joint, hip joint, knee joint, ankle joint, and spinal joint.

[0066] Meanwhile, when the control module (150) receives a rehabilitation exercise video of the patient taken after surgery from the user terminal (300), it analyzes the patient's motion based on the medical data and the rehabilitation exercise video through an artificial intelligence-based second pre-learning model, and based on the analysis result, generates guide information including information on each of at least one rehabilitation exercise corresponding to the time point and provides the generated guide information to the user terminal (300). Here, the rehabilitation exercise video is an image including an image of the patient performing the rehabilitation exercise, and may include time series data on joint movement.

[0067] To this end, the control module (150) can construct learning data for the second pre-learning model and perform pre-learning based on the data. At this time, the control module (150) can preprocess at least one rehabilitation exercise video for each patient collected over a preset period of time and construct at least one pre-processed rehabilitation exercise video as learning data. Thereafter, the learning data can be input into the second pre-learning model to perform learning.

[0068] At this time, when performing preprocessing on at least one rehabilitation exercise image, the control module (150) classifies the at least one rehabilitation exercise image by each rehabilitation stage, and labels the range of joint motion and muscle strength in each of the previously classified rehabilitation exercise images based on the previously stored average target information.

[0069] Accordingly, the control module (150) can predict the patient's condition (progress of rehabilitation exercise) after surgery based on the patient's rehabilitation exercise video, and generate guide information as a result of the prediction. Here, the guide information can include at least one of the type, method, intensity, time, and number of repetitions for each of at least one rehabilitation exercise that the patient must perform to reach the target value.

[0070] Meanwhile, the control module (150) will be specifically examined below with reference to FIG. 6 regarding the operation of analyzing the movement based on the rehabilitation exercise video.

[0071] In addition, the specific operation of the control module (150) will be described below based on each drawing.

[0072] FIG. 3 is a diagram illustrating a method for providing guide information for artificial intelligence-based rehabilitation exercise according to one embodiment of the present disclosure.

[0073] Referring to FIG. 3, the provision device (100) collects the patient's medical data and pre-operative medical images based on the storage module (130) or a separate database (S210), and performs preprocessing on the medical images (S220). Here, medical data and medical images are described separately, but medical images may be included in the concept of medical data and are not limited thereto.

[0074] Next, the provision device (100) inputs the preprocessed medical image into an artificial intelligence-based first pre-learning model to predict the probability of post-operative complications (S230), and generates prediction information based on the prediction result and provides it to the medical staff terminal (200) (S240).

[0075] Next, when the provision device (100) receives a rehabilitation exercise video of the patient taken after surgery from the user terminal (300), it analyzes the motion based on the patient's medical data and the rehabilitation exercise video through an artificial intelligence-based second pre-learning model (S250), and generates guide information including information on at least one rehabilitation exercise corresponding to the time based on the analysis result (S260).

[0076] Next, the providing device (100) provides the guide information generated by step S260 to the user terminal (300). At this time, the user terminal (300) may be a terminal possessed by at least one patient or guardian, and the number and type thereof are not limited.

[0077] Steps S210 to S240 described above relate to pre-operative movements, and the prediction information generated as a result of their execution is provided to the medical staff terminal (200), while steps S250 to S270 relate to post-operative movements, and the guide information generated as a result of their execution is provided to the user terminal (300). These two movements do not need to be performed consecutively, and may be performed separately.

[0078] FIG. 4 is a diagram illustrating a method for performing learning on an artificial intelligence-based first pre-learning model according to one embodiment of the present disclosure.

[0079] Referring to FIG. 4, the provision device (100) collects at least one medical image for each patient over a preset period of time (S201) and performs preprocessing on the collected at least one medical image (S202). Here, the at least one medical image includes various images captured for each patient over a preset period of time in chronological order.

[0080] Next, the provision device (100) constructs at least one preprocessed medical image as learning data and inputs it into the first pre-learning model to perform learning (S203).

[0081] FIG. 5 is a diagram illustrating a series of operations for preprocessing at least one medical image for each of the collected different patients according to one embodiment of the present disclosure, and illustrates step S202 in detail.

[0082] Referring to FIG. 5, the providing device (100) specifies at least one region in each of at least one medical image (S2021), and labels whether a complication occurs after surgery and / or the time of occurrence of a complication after surgery for each of the specified at least one region (S2022).

[0083] In the S2021 step described above, the providing device (100) needs to specify the part to be segmented, and this is specified. For example, various areas can be specified, such as specifying the surface formed by the tendon.

[0084] FIG. 6 is a diagram illustrating a method for analyzing the patient's movements using an artificial intelligence-based second pre-learning model according to one embodiment of the disclosure, and illustrates step S250 of FIG. 3 in more detail.

[0085] Referring to FIG. 6, the providing device (100) measures the patient's range of motion and muscle strength through the patient's rehabilitation exercise video to check the progress of the rehabilitation exercise (S251), and determines whether the target value has been reached based on the previously stored average target information (S252). At this time, in order to check the progress of the rehabilitation exercise in step S251, the gyro sensor equipped on the user terminal (200) that captured the rehabilitation exercise video may be further utilized. Meanwhile, when the providing device (100) measures the range of motion of the patient's rehabilitation exercise video, it measures the motion angle in different ways depending on each surgical site (joint site).

[0086] Through this, the patient's exercise amount is judged based on the second pre-learning model, and by comparing it with the previously stored average target information, it is judged whether the patient's exercise amount and exercise method are appropriate, i.e., whether they are not insufficient.

[0087] At this time, the appropriateness of a patient's exercise volume and method may vary depending on the patient's rehabilitation stage. To this end, the previously stored average target information may include data on joint movement and target values ​​for each part of each rehabilitation stage.

[0088] Next, the provision device (100) identifies at least one rehabilitation exercise suitable for the patient's rehabilitation stage or rehabilitation exercise progress (S253). Here, the information for each of the at least one rehabilitation exercise may include at least one of the type, method, intensity, time, and number of times for each of the at least one rehabilitation exercise that the patient must perform to reach the target value. In other words, by identifying at least one rehabilitation exercise mapped to the patient's rehabilitation stage or rehabilitation exercise progress, a rehabilitation exercise suitable for the patient to perform is recommended and provided along with the exercise method.

[0089] FIG. 7 is a diagram illustrating a method for performing learning for an artificial intelligence-based second pre-learning model according to an embodiment of the disclosure.

[0090] Referring to FIG. 7, the providing device (100) collects at least one rehabilitation exercise image for each of the other patients during a preset period of time (S204) and performs preprocessing on the at least one collected rehabilitation exercise image (S205). Here, at least one rehabilitation exercise image includes various images taken in chronological order of the images of each of the other patients performing rehabilitation exercise during a preset period of time, and each rehabilitation exercise image includes joint movements for each patient.

[0091] Next, the provision device (100) constructs at least one preprocessed rehabilitation exercise image as learning data and inputs it into the second pre-learning model to perform learning (S206).

[0092] FIG. 8 is a diagram illustrating a series of operations for preprocessing at least one rehabilitation exercise video for each of the collected different patients according to one embodiment of the present disclosure, and illustrates step S205 in detail.

[0093] Referring to FIG. 8, the providing device (100) classifies at least one rehabilitation exercise image for each rehabilitation stage (S2051), and labels the range of joint motion and muscle strength in each of the classified rehabilitation exercise images (S2052).

[0094] Meanwhile, the present disclosure performs prediction (inference) for a predetermined purpose using a model implemented in an artificial neural network manner. In the following, the artificial neural network will be examined.

[0095] A model in this specification may refer to any form of computer program that operates based on a network function, an artificial neural network, and / or a neural network. Throughout this specification, the terms model, neural network, network function, and neural network may be used interchangeably. A neural network is a network in which one or more nodes are interconnected through one or more links to form input node and output node relationships within the neural network. The characteristics of a neural network can be determined based on the number of nodes and links within the neural network, the correlation between the nodes and links, and the weight value assigned to each link. A neural network may be composed of a set of one or more nodes. A subset of the nodes constituting the neural network may constitute a layer.

[0096] A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to an input layer and an output layer. As shown in FIG. 4 as an example of this concept, a deep neural network is composed of one or more, preferably two or more, hidden layers in the middle.

[0097] These deep neural networks may include convolutional neural networks (CNNs), vision transformers, recurrent neural networks (RNNs), long short-term memory (LSTM) networks, generative pre-trained transformers (GPTs), autoencoders, generative adversarial networks (GANs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), Q-networks, U-networks, Siamese networks, generative adversarial networks (GANs), transformers, etc.

[0098] Alternatively, depending on the embodiment, the deep neural network may be a model trained using transfer learning. Here, transfer learning refers to a learning method in which a large amount of unlabeled training data is pre-trained using a semi-supervised learning or self-learning method to obtain a pre-trained model (or base part) having a first task using techniques such as MLM and NSP as shown in FIGS. 11 and 12, and then the pre-trained model is trained using labeled training data using a supervised learning method to fine-tune it to be suitable for a second task, thereby implementing a target model. One of the models trained using this transfer learning method includes, but is not limited to, BERT (Bidirectional Encoder Representations from Transformers).

[0099] The description of the aforementioned deep neural network is merely an example, and the present disclosure is not limited thereto. In the case of the aforementioned convolutional neural network, it is composed of a feature extraction unit (feature learning) that extracts features from an image, and a classification unit (classification) that performs classification using the extracted features. The feature extraction unit may include, but is not limited to, a convolution layer that extracts features from an image using a kernel, a ReLU layer as one of the activation functions, and a pooling layer for reducing the dimensionality of the data. In addition, the classification unit may include, but is not limited to, a flatten layer that lines up the features extracted from the feature extraction unit, and a fully connected layer and a softmax function that actually perform classification.

[0100] Neural networks can learn through at least one of the following methods: supervised learning, unsupervised learning, semi-supervised learning, self-supervised learning, or reinforcement learning. Neural network learning can be the process of applying knowledge to the neural network to perform a specific action.

[0101] Neural networks can be trained to minimize output errors. This process involves repeatedly inputting training data into the neural network, calculating the neural network output and target error for the training data, and backpropagating the neural network error from the output layer to the input layer to update the weights of each node in the neural network to reduce the error. In supervised learning, labeled data is used for each training data, while unsupervised learning uses unlabeled data. The amount of change in the connection weights of each updated node can be determined by the learning rate. The neural network's calculation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate can vary depending on the number of iterations in the neural network's training cycle. Additionally, to prevent overfitting, methods such as increasing the training data, regularization, dropout that disables some nodes, and batch normalization layers can be applied.

[0102] Meanwhile, the model disclosed in one embodiment may borrow at least a portion of a transformer. The transformer may be composed of an encoder that encodes embedded data and a decoder that decodes the encoded data. The transformer may have a structure that receives a series of data, performs encoding and decoding steps, and outputs a series of data of different types. In one embodiment, the series of data may be processed into a form operable by the transformer. The process of processing the series of data into a form operable by the transformer may include an embedding process. Expressions such as data tokens, embedding vectors, and embedding tokens may refer to data embedded in a form operable by the transformer.

[0103] To encode and decode a series of data, a transformer can utilize an attention algorithm to process the encoders and decoders within the transformer. An attention algorithm can refer to an algorithm that, for a given query, calculates the similarity for one or more keys, reflects this similarity in the values ​​corresponding to each key, and then weights and sums the values ​​to which the similarity is reflected to calculate an attention value.

[0104] Depending on how the query, key, and value are configured, various types of attention algorithms can be categorized. For example, if attention is obtained by setting the query, key, and value all to the same value, this could be a self-attention algorithm. If attention is obtained by reducing the dimensionality of the embedding vector to process a series of input data in parallel and then generating individual attention heads for each segmented embedding vector, this could be a multi-head attention algorithm.

[0105] In one embodiment, the transformer may be composed of modules that perform multiple multi-head self-attention algorithms or multi-head encoder-decoder algorithms. In one embodiment, the transformer may also include additional components other than attention algorithms, such as embedding, normalization, and softmax. Methods for constructing a transformer using attention algorithms may include methods disclosed in Vaswani et al., Attention Is All You Need, 2017 NIPS, which is incorporated herein by reference.

[0106] A transformer can be applied to various data domains, such as embedded natural language, segmented image data, and audio waveforms, to transform a series of input data into a series of output data. To transform data with various data domains into a series of data that can be input to a transformer, the transformer can embed the data. The transformer can process additional data that expresses the relative positional relationship or phase relationship between the series of input data. Alternatively, vectors expressing the relative positional relationship or phase relationship between the input data can be additionally reflected in the series of input data to embed the series of input data. In one example, the relative positional relationship between the series of input data may include, but is not limited to, word order within a natural language sentence, the relative positional relationship between each segmented image, and the time order of segmented audio waveforms. The process of adding information expressing the relative positional relationship or phase relationship between the series of input data may be referred to as positional encoding.

[0107] The above-described program may include codes coded in a computer language, such as C, C++, JAVA, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary for executing the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0108] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0109] The steps of a method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0110] While the embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present disclosure can be implemented in other specific forms without altering the technical spirit or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. A communication module that communicates with external devices; A storage module storing at least one process for providing guidance information on rehabilitation exercise based on artificial intelligence; and A control module for performing an operation to provide guidance information for rehabilitation exercise based on artificial intelligence based on at least one of the above processes, The above control module, Based on the above storage module and separate database, the patient's medical data and medical images taken before surgery are collected, After performing preprocessing on the above medical images, the preprocessed medical images are input into the first pre-learning model based on artificial intelligence to predict the probability of post-operative complications. A device providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that it generates prediction information based on the predicted results and provides it to a medical staff terminal.

2. In paragraph 1, The above first pre-learning model is, A device for providing guidance information for artificial intelligence-based rehabilitation exercise, characterized in that at least one medical image for each different patient collected during a preset period is preprocessed, and at least one preprocessed medical image is input as learning data to perform learning.

3. In paragraph 2, The above control module, When performing preprocessing on at least one of the above medical images, Identifying at least one region in each of the at least one medical image, An artificial intelligence-based rehabilitation exercise guide information providing device characterized by labeling at least one of occurrence and occurrence time of postoperative complications for each of the at least one area based on the medical data of each of the other patients.

4. In paragraph 1, The above control module, when receiving a rehabilitation exercise video of the patient taken after surgery from the user terminal, analyzes the motion based on the medical data and the rehabilitation exercise video through an artificial intelligence-based second pre-learning model, and generates guide information including information on at least one rehabilitation exercise corresponding to the corresponding time based on the analysis result and provides the guide information to the user terminal. A device providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that the above rehabilitation exercise video includes time series data on joint movement.

5. In paragraph 4, The above control module, A device for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that when analyzing a movement based on the rehabilitation exercise video, the patient's range of joint motion and muscle strength are measured to check the progress of the rehabilitation exercise, and then whether the target value has been reached is determined based on the average target information previously stored, and at least one rehabilitation exercise suitable for the patient is determined based on the determination result.

6. In paragraph 4, The information for each of the at least one rehabilitation exercise includes at least one of the type, method, intensity, time, and number of times for each of the at least one rehabilitation exercise that the patient must perform to reach the target value. A device providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that the above-mentioned stored average target information includes data on joint movement for each part of each rehabilitation stage and information on target values.

7. In paragraph 6, The above second pre-learning model preprocesses at least one rehabilitation exercise video for each of the other patients collected during a preset period, and performs learning by inputting at least one preprocessed rehabilitation exercise video as learning data. The control module, when performing preprocessing on the at least one rehabilitation exercise image, classifies the at least one rehabilitation exercise image by each rehabilitation stage, and labels the range of joint motion and muscle strength in each of the classified rehabilitation exercise images based on the previously stored average target information, is a device providing guide information for artificial intelligence-based rehabilitation exercise.

8. A method for providing guide information for artificial intelligence-based rehabilitation exercise performed by a device, A step of collecting the patient's medical data and medical images taken before surgery based on a storage module and a separate database; A step of performing preprocessing on the above medical image; A step of predicting the probability of occurrence of complications after surgery by inputting the above preprocessed medical image into an artificial intelligence-based first pre-learning model; and A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized by including a step of generating prediction information based on the predicted results and providing the information to a medical staff terminal.

9. In paragraph 8, A step of collecting at least one medical image for each different patient over a preset period of time; A step of performing preprocessing on at least one of the above medical images; and A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that it further includes a step of performing learning on the first pre-learning model by inputting at least one of the above-mentioned pre-processed medical images as learning data.

10. In paragraph 9, The steps for performing the above preprocessing are: a step of specifying at least one region in each of said at least one medical image; and A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that it further includes a step of labeling at least one of occurrence and occurrence time of postoperative complications for each of the at least one area based on medical data of each of the other patients.

11. In paragraph 8, When the rehabilitation exercise video of the patient taken after surgery is received from the user terminal, a step of analyzing the motion based on the medical data and the rehabilitation exercise video through an artificial intelligence-based second pre-learning model; and Further comprising a step of generating guide information including information on at least one rehabilitation exercise corresponding to the time based on the analysis results and providing the information to the user terminal, A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that the above rehabilitation exercise video includes time series data on joint movement.

12. In paragraph 11, The step of analyzing the movement based on the above rehabilitation exercise video is: A step for measuring the range of joint motion and muscle strength of the patient in question to check the progress of rehabilitation exercise; A step for determining whether the target value has been reached based on the previously stored average target information; and A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that it further includes a step of identifying at least one rehabilitation exercise suitable for the patient based on the judgment result.

13. In paragraph 11, The information for each of the at least one rehabilitation exercise includes at least one of the type, method, intensity, time, and number of times for each of the at least one rehabilitation exercise that the patient must perform to reach the target value. A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that the above-mentioned stored average target information includes data on joint movement for each part of each rehabilitation stage and information on target values.

14. In paragraph 13, A step of collecting at least one rehabilitation exercise video for each different patient over a preset period of time; A step of performing preprocessing on at least one of the above rehabilitation exercise images; and Further comprising a step of inputting at least one preprocessed rehabilitation exercise video as learning data to perform learning for the second pre-learning model, The steps for performing the above preprocessing are: A step of classifying at least one of the above rehabilitation exercise images into each rehabilitation stage; and A method for providing guide information for artificial intelligence-based rehabilitation exercise, characterized in that it further includes a step of labeling the range of joint motion and muscle strength in each of the classified rehabilitation exercise images based on the above-mentioned stored average target information.

15. A computer-readable recording medium having recorded thereon a computer program for performing the method of providing guide information for artificial intelligence-based rehabilitation exercise of Article 8.

Images