A device and method for evaluating the musculoskeletal system using images and algorithms.

Abstract

A device and method for evaluating the musculoskeletal system using images and algorithms are provided. [Solution] The device includes a memory and a processor that performs actions according to the process, which include at least one process for performing musculoskeletal system evaluation using images and algorithms, acquires images captured by a camera, calculates an evaluation index for evaluating at least one of the subject's spine and joints based on body coordinate information obtained from the images, and determines the state of at least one of the spine and joints based on the calculated evaluation index.

Description

Technical Field

[0001] The present disclosure relates to a musculoskeletal system evaluation apparatus and method using images and algorithms.

Background Art

[0002] Musculoskeletal diseases are pains or injuries that occur in the musculoskeletal system such as muscles, nerves, tendons, ligaments, bones, and peripheral tissues, and are mainly changes that progress due to aging, and are aggravated by factors such as repetitive movements, stimuli such as compression and vibration, inappropriate working postures, and excessive use of force. They are observed in all joints of the body such as the neck, waist, shoulders, knees, arms, and legs, and exhibit specific aspects depending on the joint.

[0003] Conventionally, in order to examine the health status of the musculoskeletal system, the subject directly visited an examination room equipped with certain equipment, and the evaluation was performed with the assistance of an examiner.

[0004] However, such an examination method involves the subjective judgment of professional staff who analyze data, and since it is carried out in a specific space or location, the availability of the examination is reduced, and it is difficult to confirm by screening tests at an early stage of the disease or to perform regular follow-up observations. Furthermore, there is a problem that the movements and postures in daily life cannot be reflected and the evaluation is impossible.

Summary of the Invention

Problems to be Solved by the Invention

[0005] An object of an embodiment of the present disclosure is to provide a musculoskeletal system evaluation apparatus and method using images and algorithms.

[0006] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can also be clearly understood by those skilled in the art from the following description.

Means for Solving the Problems

[0007] A musculoskeletal system evaluation device using images and algorithms according to one aspect of the present disclosure for achieving the technical problems described above includes a memory including at least one process for performing musculoskeletal system evaluation using images and algorithms, and a processor that performs operations according to the process, wherein the processor acquires images captured by a camera, calculates an evaluation index for evaluating at least one of the subject's spine and joints based on body coordinate information acquired from the images, determines the state of at least one of the spine and joints based on the calculated evaluation index, and the images may include a first image acquired when the evaluation mode is a daily activity mode and a second image acquired when the evaluation mode is an examination activity mode.

[0008] Furthermore, the camera may include a first camera capable of acquiring 3D information using a posture estimation algorithm and a second camera capable of providing depth information.

[0009] Furthermore, the processor may acquire three-dimensional body coordinate information based on the three-dimensional information obtained by applying the pose estimation algorithm to the image captured by the first camera, and may acquire three-dimensional body coordinate information by integrating the two-dimensional body coordinate information obtained by applying the pose estimation algorithm to the image captured by the second camera and the depth information obtained by the second camera.

[0010] Furthermore, the first image may include a specific posture or movement related to the evaluation from the subject's daily activities, the specific posture or movement being determined by an activity classifier, and the second image may be an image taken when the subject performs a pre-set test action for the specific posture or movement.

[0011] Furthermore, the processor may calculate an evaluation index for evaluating at least one of the vertebrae and joints associated with the movement, based on the changes in movement when the subject takes the specific posture or movement in the first image and the body coordinate information that changes in response to the changes in movement, and may compare the calculated evaluation index with a previous evaluation index to determine whether there is any deterioration or improvement in at least one of the vertebrae and joints.

[0012] Furthermore, the processor calculates an evaluation index for evaluating at least one of the spine and joints associated with the action based on the changes in movement when the subject performs the examination action in the second image and the body coordinate information that changes in response to the changes in movement, and can compare the calculated evaluation index with a previous evaluation index to determine whether there is any deterioration or improvement in at least one of the spine and joints.

[0013] Furthermore, the aforementioned evaluation indicators may be set differently depending on the mechanism and characteristics of the degeneration and disease of each vertebra or joint being evaluated.

[0014] Furthermore, if the target of evaluation is the shoulder joint, the evaluation index may include an index for evaluating the range of motion of the shoulder joint and an index for the workspace that quantifies the movement of the shoulder complex in three dimensions; if the target of evaluation is the lumbar spine, the evaluation index may include an index for evaluating the sagittal vertical axis, lumbar lordosis angle, and scoliosis in the coronal plane of the lumbar spine; if the target of evaluation is the cervical spine, the evaluation index may include an index for evaluating the cervical lordosis angle, chin-brow vertical axis (CBVA), and sagittal vertical axis of the cervical spine; if the target of evaluation is the knee joint, the evaluation index may include an index for evaluating the degree of genu varum and genu valgus deformity and flexion contracture angle; and if the target of evaluation is the ankle joint, the evaluation index may include an index for evaluating the range of motion and dynamic balance of the ankle joint.

[0015] Furthermore, the cloud, which is linked to the communication unit of the device, may store at least one of the following forms: the entire first image, the first image converted to a lower resolution, the first image with reduced fps (frames per second), and skeleton information of the first image.

[0016] Furthermore, a musculoskeletal system evaluation method using images and algorithms according to another aspect of the present disclosure for achieving the technical challenges described above includes the steps of acquiring images captured by a camera, calculating an evaluation index for evaluating at least one of the subject's spine and joints based on body coordinate information acquired from the images, and determining the state of at least one of the spine and joints based on the calculated evaluation index, wherein the images may include a first image acquired when the evaluation mode is a daily activity mode and a second image acquired when the evaluation mode is an examination activity mode.

[0017] Furthermore, the camera may include a first camera capable of acquiring 3D information using a posture estimation algorithm and a second camera capable of providing depth information.

[0018] Furthermore, the step of analyzing the image may involve obtaining 3D body coordinate information based on the 3D information obtained by applying the posture estimation algorithm to the image captured by the first camera, and integrating the 2D body coordinate information obtained by applying the posture estimation algorithm to the image captured by the second camera and the depth information obtained by the second camera to obtain 3D body coordinate information.

[0019] Furthermore, the first image may include a specific posture or movement related to the evaluation from the subject's daily activities, the specific posture or movement being determined by an activity classifier, and the second image may be an image taken when the subject performs a pre-set test action for the specific posture or movement.

[0020] Furthermore, the step of calculating the evaluation index may involve calculating an evaluation index for evaluating at least one of the vertebrae and joints associated with the movement, based on the changes in movement when the subject takes the specific posture or movement in the first image and the body coordinate information that changes in accordance with the changes in movement, and the step of determining the state may involve comparing the calculated evaluation index with a previous evaluation index to determine whether there is a deterioration or improvement in at least one of the vertebrae and joints.

[0021] Furthermore, the step of calculating the evaluation index involves calculating an evaluation index for evaluating at least one of the spine and joints associated with the movement, based on the changes in movement when the subject performs the examination movement in the second image and the body coordinate information that changes in accordance with the changes in movement. The step of determining the state involves comparing the calculated evaluation index with a previous evaluation index to determine whether at least one of the spine and joints has worsened or improved.

[0022] Furthermore, the aforementioned evaluation indicators may be set differently depending on the mechanism and characteristics of the degeneration and disease of each vertebra or joint being evaluated.

[0023] Furthermore, if the target of evaluation is the shoulder joint, the evaluation index may include an index for evaluating the range of motion of the shoulder joint and an index for the workspace that quantifies the movement of the shoulder complex in three dimensions; if the target of evaluation is the lumbar spine, the evaluation index may include an index for evaluating the sagittal vertical axis, lumbar lordosis angle, and scoliosis in the coronal plane of the lumbar spine; if the target of evaluation is the cervical spine, the evaluation index may include an index for evaluating the cervical lordosis angle, chin-brow vertical axis (CBVA), and sagittal vertical axis of the cervical spine; if the target of evaluation is the knee joint, the evaluation index may include an index for evaluating the degree of genu varum and genu valgus deformity and flexion contracture angle; and if the target of evaluation is the ankle joint, the evaluation index may include an index for evaluating the range of motion and dynamic balance of the ankle joint.

[0024] In addition, in the cloud that cooperates with the communication unit of the device, at least one of the entire first image, the first image converted to low resolution, the first image with a reduced fps (frame per second), and the skeleton information of the first image can be stored.

[0025] In addition, a computer program stored in a computer-readable recording medium according to another aspect of the present disclosure for achieving the above-described technical problems, when executed by one or more processors, performs operations for performing a muscle and skeletal system evaluation method using an image and an algorithm. The operations include an operation of acquiring an image captured by a camera, an operation of calculating an evaluation index for evaluating at least one of the spine and joints of a subject based on the body coordinate information acquired from the image, and an operation of determining a state of at least one of the spine and the joints based on the calculated evaluation index. The image may include a first image acquired when the evaluation mode is the daily operation mode and a second image acquired when the evaluation mode is the inspection operation mode.

[0026] In addition, a computer-readable recording medium for recording a computer program for executing a method for realizing the present disclosure may be further provided.

Advantages of the Invention

[0027] According to the above-described problem-solving means of the present disclosure, non-intrusive analysis can be performed without the need for expert support and without restricting the user, without being limited by constraints such as cost, equipment, or the situation in which evaluation is performed.

[0028] In addition, it is possible to detect and treat spinal / joint deformities or diseases before they deteriorate severely, and by utilizing them in the rehabilitation treatment of spinal / joint diseases, the treatment efficiency can be improved. As a result, clinical outcomes can be improved and costs can be reduced.

[0029] Furthermore, by providing accurate and useful assessment tools in situations where access to medical care is limited, the efficiency of medical resources can be improved, and unnecessary social costs can be reduced. In particular, since degenerative diseases of the spine and joints are more common in the elderly, utilizing the aforementioned assessment tools can help reduce the rapidly increasing medical costs associated with an aging population.

[0030] Furthermore, because it allows for evaluation even under conditions where access to medical care is limited, it can also be applied to the fields of telemedicine or smart healthcare.

[0031] The effects of this disclosure are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawing]

[0032] [Figure 1] This figure schematically illustrates a musculoskeletal system evaluation system using images and algorithms according to one embodiment of the present disclosure. [Figure 2] This is a flowchart of a musculoskeletal system evaluation method using images and an algorithm according to one embodiment of the present disclosure. [Figure 3] This figure illustrates the generation of a first image from video footage of daily activities according to one embodiment of the present disclosure. [Figure 4] This figure illustrates how to generate a second image from video footage of an inspection operation according to one embodiment of the present disclosure. [Figure 5] This figure illustrates how body coordinate information is acquired by a first camera according to one embodiment of the present disclosure. [Figure 6] This figure illustrates how body coordinate information is acquired by a second camera according to one embodiment of the present disclosure. [Figure 7] This figure schematically illustrates a musculoskeletal system evaluation system including a cloud according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0033] Throughout this disclosure, the same reference numerals refer to the same component. This disclosure does not describe all elements of the embodiments, and omits information that is known in the art to which this disclosure belongs or that is redundant between embodiments. The terms “part,” “module,” “component,” and “block” as used herein can be implemented by software or hardware, and depending on the embodiment, multiple “parts,” “modules,” “components,” and “blocks” may be implemented as a single component, or a single “part,” “module,” “component,” and “block” may include multiple components. Throughout this specification, when a part is described as being “connected” to another part, this includes not only direct connection but also indirect connection, in which case connection includes connection via a wireless communication network.

[0034] Furthermore, when a part is described as "containing" a certain component, unless otherwise stated, it does not mean that other components are excluded, but rather that other components may be included.

[0035] Throughout this specification, when it is stated that one member is located "on" another member, this includes not only cases where the member is in contact with another member, but also cases where another member is present between the two members.

[0036] Terms such as "first," "second," etc., are used to distinguish one component from another, and the components are not limited by these terms.

[0037] A singular expression includes plural forms unless there is a clear exception in the context.

[0038] The identification codes assigned to each stage are for explanatory purposes only and do not indicate the order of the stages. Unless a specific order is explicitly stated in the context, the stages may be performed in a different order than that listed.

[0039] The operating principles and embodiments of this disclosure will be described below with reference to the attached drawings.

[0040] Prior to the explanation, the meanings of the terms used herein will be briefly explained. However, it should be noted that these explanations are provided to aid in the understanding of this specification and are not intended to limit the technical ideas of this disclosure unless explicitly stated to limit them.

[0041] In this specification, “device” includes various devices capable of performing computational processing and providing results to a user. For example, a device may include a computer, a server, and a mobile terminal, or any one of them.

[0042] Here, the computer may include, for example, a laptop computer, desktop computer, laptop computer, tablet PC, or slate PC equipped with a web browser.

[0043] The aforementioned server device is a server that communicates with external devices and processes information, and may include application servers, computing servers, database servers, file servers, game servers, mail servers, proxy servers, and web servers.

[0044] The aforementioned portable terminals include, for example, all kinds of handheld-based wireless communication devices that guarantee portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, and smartphones. They may also include wearable devices such as watches, rings, bracelets, anklets, necklaces, eyeglasses, contact lenses, or head-mounted devices (HMDs).

[0045] The artificial intelligence-related functions described herein operate via a processor and memory. The processor may consist of one or more processors. In this case, one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed as hardware structures specialized for processing a particular artificial intelligence model.

[0046] The predefined behavioral rules or artificial intelligence models are characterized by being created through learning. Here, "created through learning" means that a basic artificial intelligence model is trained using a learning algorithm with a large amount of training data to create predefined behavioral rules or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device on which the artificial intelligence described herein operates, or it may be performed via a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0047] An artificial intelligence model can consist of multiple neural network layers. Each of these neural network layers has multiple weight values, and neural network operations are performed by operations between the results of operations from the previous layer and these multiple weight values. The multiple weight values ​​of the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weight values ​​can be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks can include, but are not limited to, deep neural networks (DNNs), such as CNNs (Convolutional Neural Networks), DNNs (Deep Neural Networks), RNNs (Recurrent Neural Networks), RBMs (Restricted Boltzmann Machines), DBNs (Deep Belief Networks), BRDNNs (Bidirectional Recurrent Deep Neural Networks), or Deep Q-Networks.

[0048] A processor can generate a neural network, train (or learn) a neural network, perform calculations based on incoming input data and generate an information signal based on the results, or retrain the neural network.

[0049] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), and LSM (Liquid This may include, but is not limited to, State Machines, ELMs (Extreme Learning Machines), ESNs (Echo State Networks), DRNs (Deep Residual Networks), DNCs (Differentiable Neural Computers), NTMs (Neural Turing Machines), CNs (Capsule Networks), KNs (Kohonen Networks), and ANs (Attention Networks), and any neural network would be understood by an average engineer.

[0050] According to exemplary embodiments of this disclosure, the processor may include CNNs (Convolutional Neural Networks), R-CNNs (Region with Convolutional Neural Networks), RPNs (Region Proposal Networks), RNNs (Recurrent Neural Networks), S-DNNs (Stacking-based Deep Neural Networks), S-SDNNs (State-Space Dynamic Neural Networks), Deconvolution Networks, DBNs (Deep Belief Networks), RBMs (Restricted Boltzmann Machines), Fully Convolutional Networks, LSTMs (Long Short-Term Memory) Networks, Classification Networks, Generative Modeling, eExplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4 for natural language processing, Visual Analytics, Visual Understanding, Video Synthesis, ResNet for vision processing, Anomaly Detection, Prediction, and Time-Series for data intelligence. A wide variety of artificial intelligence structures and algorithms can be used, including but are not limited to forecasting, optimization, recommendation, and data creation.

[0051] The embodiments of this disclosure will be described in detail below with reference to the attached drawings.

[0052] Figure 1 is a diagram illustrating a musculoskeletal system evaluation system 1 (hereinafter referred to as "the system") using images and algorithms according to one embodiment of the present disclosure.

[0053] In one embodiment of this disclosure, System 1 may include a musculoskeletal system evaluation device 10, a camera 20, and a user terminal 30. However, in some embodiments, System 1 may include fewer or more components than those shown in Figure 1.

[0054] The musculoskeletal system evaluation device 10 receives video footage captured by the camera 20, processes the received video footage to generate images (or sequences of images) that include postures or movements significant for spinal and joint evaluation, calculates evaluation indices via body coordinate information for the generated images, and can evaluate the health and disease state of the spine and joints or the alignment state of the spine and joints based on the calculated evaluation indices.

[0055] The musculoskeletal system evaluation device 10 can provide evaluation results to the user terminal 30. Specifically, the user terminal 30 can install an app or program that provides musculoskeletal system evaluation services and check the evaluation results within the app.

[0056] Here, the user terminal 30 may be the subject's terminal or the medical professional's terminal. Based on the evaluation results, the subject can understand and manage the condition of their spine and joints themselves without the help of a specialist. Based on the evaluation results, the medical professional can diagnose spinal and joint deformities of the subject early and confirm objective information necessary to observe their progression.

[0057] The user terminal 30 is a device to which information processing means such as a computer may be applied, and includes all devices including a processor such as a control unit, a camera such as a camera, input / output means including a touchscreen, and communication functions. In other words, any device such as a smartphone, tablet, PDA, laptop, or desktop computer can be used.

[0058] In one embodiment, the user terminal 30 can also perform the operations of the musculoskeletal system evaluation device 10 in an on-device format. Specifically, the user terminal 30 can process images captured by the camera 20 via the app or program installed on the user terminal 30 to generate images (or sequences of images) that include postures or movements significant for spinal and joint evaluation, calculate evaluation indices based on the body coordinate information related to the generated images, and evaluate the health and disease state of the spine and joints or the alignment state of the spine and joints based on the calculated evaluation indices.

[0059] Camera 20 may include a first camera, such as an RGB camera, and a second camera, such as a Depth camera, which provides three-dimensional information.

[0060] Specifically, the first camera is a camera capable of acquiring 2D or 3D information using a pose estimation algorithm, and refers to a general video camera or smartphone camera. In other words, the first camera does not immediately provide 2D or 3D information; rather, 2D or 3D information can be acquired by applying a pose estimation algorithm to the image captured by the first camera.

[0061] The second camera could be a depth camera, a camera that calculates depth using an infrared emitter / sensor, or a smartphone camera including a LiDAR sensor. However, it is not limited to these; any camera capable of providing 3D information can be used.

[0062] Referring to Figure 1, the musculoskeletal system evaluation device 10 may include a communication unit 11, a memory 12, and a processor 13. The processor 13 may include an image analysis module 131, an index calculation module 132, and a state evaluation module 133. However, in some embodiments, the musculoskeletal system evaluation device 10 and the processor 13 may include fewer or more components than those shown in Figure 1.

[0063] The communication unit 11 may include one or more modules that enable wireless or wired communication between the musculoskeletal system evaluation device 10 and the camera 20, between the musculoskeletal system evaluation device 10 and the user terminal 30, between the musculoskeletal system evaluation device 10 and an external device (not shown), and between the musculoskeletal system evaluation device 10 and a communication network. For example, it may include at least one of a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0064] Various types of communication networks can be used, such as wireless communication methods like WLAN (Wireless LAN), Wi-Fi, Wibro, WiMAX, and HSDPA (High Speed ​​Downlink Packet Access), or wired communication methods like Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coax), FTTC (Fiber to The Curb), and FTTH (Fiber to The Home).

[0065] On the other hand, the communication network is not limited to the communication methods exemplified above, but may include other widely known communication methods and any form of communication method that may be developed in the future.

[0066] Wired communication modules may include a variety of cable communication modules, such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service), as well as various other wired communication modules such as Local Area Network (LAN) modules, Wide Area Network (WAN) modules, or Value Added Network (VAN) modules.

[0067] Wireless communication modules may include not only Wi-Fi modules and WiBro (Wireless broadband) modules, but also wireless communication modules that support a variety of wireless communication methods such as GSM (Global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (Universal Mobile Telecommunications System), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G.

[0068] The short-range communication module is for short-range communication and uses Bluetooth. TM Short-range communication can be supported by utilizing at least one of the following technologies: RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus).

[0069] Memory 12 may store at least one process for performing a musculoskeletal assessment using images and algorithms.

[0070] Memory 12 can store data supporting the various functions of the musculoskeletal system evaluation device 10, programs for the operation of the processor 13, input / output data (e.g., music files, still images, videos, etc.), and numerous application programs (applications) driven by the musculoskeletal system evaluation device 10, as well as data and instructions for the operation of the musculoskeletal system evaluation device 10. At least some of these application programs can be downloaded from an external server via wireless communication.

[0071] Thus, memory 12 may include at least one type of recording medium from among flash memory type, hard disk type, SSD type (Solid State Disk type), SDD type (Silicon Disk Drive type), multimedia card micro type, card type memory (e.g., SD or XD memory), RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, magnetic disk, and optical disk. Furthermore, although memory 12 is separate from the musculoskeletal system evaluation device 10, it may be a database connected by wire or wireless.

[0072] The processor 13 stores data related to an algorithm or a program that reproduces the algorithm for controlling the operation of the components within the musculoskeletal evaluation device 10, and can perform the aforementioned operations using the data stored in the memory. In this case, the memory 12 and the processor 13 may be implemented as separate chips, or the memory 12 and the processor 13 may be implemented as a single chip.

[0073] Furthermore, the processor 13 can control any one or more of the above-mentioned components in combination in order to realize the various embodiments of this disclosure, as described in Figures 2 to 6 below, on the musculoskeletal system evaluation device 10.

[0074] The image analysis module 131 of the processor 13 can analyze images (videos) captured by the camera 20 using at least one of an individual identification algorithm and a pose estimation algorithm.

[0075] The index calculation module 132 of the processor 13 can calculate an evaluation index for evaluating at least one of the spine and joints based on the body coordinate information obtained by the analysis.

[0076] The state evaluation module 133 of the processor 13 can determine the state of at least one of the subject's spine and joints based on the calculated evaluation index.

[0077] The first embodiment of a musculoskeletal system evaluation method using images and algorithms will be described in detail below with reference to Figures 2 to 6.

[0078] Figure 2 is a flowchart of a musculoskeletal system evaluation method using images and an algorithm according to one embodiment of the present disclosure.

[0079] Figure 3 is a diagram illustrating the generation of a first image from video footage of everyday activities according to one embodiment of the present disclosure.

[0080] Figure 4 is a diagram illustrating the generation of a second image from video footage of an inspection operation according to one embodiment of the present disclosure.

[0081] Figure 5 is a diagram illustrating how body coordinate information is acquired by a first camera according to one embodiment of the present disclosure.

[0082] Figure 6 is a diagram illustrating how body coordinate information is acquired by a second camera according to one embodiment of the present disclosure.

[0083] For the sake of explanation, each step will be described below as being performed by the processor 13, but it can be understood that each step is performed by one of the following modules included in the processor 13: the image analysis module 131, the index calculation module 132, the state evaluation module 133, and any module not shown in Figure 1.

[0084] Referring to Figure 2, the processor 13 of the musculoskeletal system evaluation device 10 can acquire images captured by the camera 20 (S210).

[0085] Camera 20 may include a first camera, such as an RGB camera, and a second camera, such as a Depth camera, which provides three-dimensional information.

[0086] Images (videos) captured by camera 20 (first camera or second camera) may include first images related to daily activities and second images related to examination activities.

[0087] Here, the first image may be video data acquired when the evaluation mode is the daily operation mode, and the second image may be video data acquired when the evaluation mode is the inspection operation mode.

[0088] The processor 13 may use the first image to extract information significant for evaluation from the subject's daily living activities, or it may use the second image to obtain accurate information when the subject consciously performs the test actions.

[0089] Specifically, the first image may be video data capturing the subject's daily activities. The first image may include specific postures or movements from the subject's daily activities that are relevant to the evaluation.

[0090] Here, "posture" refers to a static position of the body (e.g., standing posture), while "activity" refers to dynamic body movement (e.g., walking).

[0091] The first image, which captures the subject's daily activities, may be saved and stored in a cloud connected to the communications unit. In this case, the entire first image may be saved in the cloud, or it may be saved after being converted to a lower resolution, or after being saved with a reduced fps (frames per second), or only the skeleton information of the first image may be saved.

[0092] In other words, according to the embodiment, the first image is stored in the cloud, but it can be stored in at least one of the following forms: the entire first image, the first image converted to a lower resolution, the first image with reduced fps (frames per second), and the skeleton information of the first image.

[0093] Furthermore, the first image stored in the cloud can be deleted after a certain period of time to protect privacy.

[0094] According to one embodiment, the processor 13 may provide only the moving portion of the first image to the cloud. Alternatively, the processor 13 may provide the entire first image along with time information relating to the moving portion of the first image to the cloud.

[0095] According to one embodiment, if multiple objects (people) appear in the first image, the processor 13 may recognize the subject based on pre-stored identification information and provide only the portion in which the subject appears to the cloud. Alternatively, the processor 13 may provide the cloud with the entire first image along with time information relating to the portion of the first image in which the subject appears.

[0096] An activity classifier can determine specific postures or movements contained in the first image stored in the cloud. The activity classifier has pre-stored information on postures or movements to be judged and can detect postures or movements in the first image that are significant for evaluation.

[0097] According to the embodiment, posture or movement information stored in the activity classifier can be updated. Specifically, additional posture or movement information for evaluating existing diseases can be updated, or posture or movement information for evaluating new diseases can be updated.

[0098] The processor 13 may analyze the alignment and presence or absence of deformation of the subject's joints and / or spine based on specific postures or movements determined by the activity classifier.

[0099] For example, if the activity classifier detects an upright posture from the first image, the processor 13 may analyze the sagittal plane alignment of the subject's spine and coronal plane deformation of the knees based on the subject's upright posture.

[0100] As another example, if the activity classifier detects an arm-raising motion from the first image, the processor 13 may analyze the subject's shoulder joint movement (movement pattern and range of motion) based on the subject's arm-raising motion.

[0101] As another example, if the activity classifier detects a seated posture from the first image, the processor 13 may analyze the deformation of the subject's cervical and thoracic spine based on the subject's seated posture.

[0102] As another example, if the activity classifier detects a sitting motion from the first image, the processor 13 may analyze the sagittal plane movement (movement pattern and range of motion) of the subject's knee joint based on the subject's sitting motion.

[0103] As another example, if the activity classifier detects walking motion from the first image, the processor 13 may analyze the movement (movement pattern and range of motion) of the subject's spine, knee joints, and ankle joints based on the subject's walking motion.

[0104] Furthermore, the second image may be generated when the subject performs a pre-set test action related to a specific posture or movement associated with the evaluation. Here, the test action may be set as a posture or movement associated with the object being evaluated.

[0105] The second image, which captures the subject's examination movements, may be saved and stored in a cloud connected to the communications unit. In this case, the entire second image may be saved in the cloud, or it may be saved in a lower resolution or with a reduced fps (frames per second), or only the skeleton of the second image may be saved.

[0106] In other words, according to the embodiment, the second image is stored in the cloud, but it can be stored in at least one of the following forms: the entire second image, the second image converted to a lower resolution, the second image with reduced fps (frames per second), and the skeleton information of the second image.

[0107] Furthermore, the second image stored in the cloud may be deleted after a certain period of time for privacy protection.

[0108] For example, if the object of evaluation is the spine and joints, the examination action may be set to maintain an upright posture in the front or side view for a predetermined time. Based on the second image taken of the subject in the upright posture, the processor 13 may analyze the sagittal plane alignment of the subject's spine and coronal plane deformation of the knee.

[0109] As another example, if the object of evaluation is the shoulder joint, the test movement may be set to have the subject perform the movements necessary for evaluating the function of the shoulder joint. The processor 13 may analyze the range of motion and movement pattern of the subject's shoulder joint based on the second image in which the test movement was captured.

[0110] As another example, if the object of evaluation is the knee joint, the test movement may be set to have the subject perform the movement necessary for evaluating the function of the knee joint. The processor 13 may analyze the range of motion and movement pattern of the subject's knee joint based on the second image in which the test movement was captured.

[0111] As another example, if the object of evaluation is the spine, the test movement may be set to have the subject perform movements that involve bending the body forward or backward, which are necessary for evaluating the function of the spine. The processor 13 may analyze the range of motion and movement pattern of the subject's spine based on the second image in which the test movement is captured.

[0112] The processor 13 of the musculoskeletal system evaluation device 10 can analyze the image using at least one of the personal identification algorithm and the posture estimation algorithm (S220).

[0113] The processor 13 may decide whether or not to apply the personal identification algorithm depending on the evaluation mode. The personal identification algorithm may only be applied if the evaluation mode is the normal operation mode.

[0114] In other words, the first image obtained by photographing the subject's daily life may be difficult to identify objects in (as mentioned above, if there are multiple objects in the first image), and a process to identify the subject may be necessary. Therefore, the processor 13 can identify the subject from the first image using an individual identification algorithm.

[0115] Specifically, the processor 13 can identify the subject in the first image based on the identification information (e.g., a facial image) entered by the subject when they register for the musculoskeletal system assessment service.

[0116] If the processor 13 identifies a face detected in the first image as belonging to a user registered with the service, it may provide the first image to the cloud for storage.

[0117] If the processor 13 determines that the face detected in the first image is not that of a user registered with the service, it may choose not to provide the first image to the cloud.

[0118] Referring to Figure 3, a first image can be generated by capturing daily activities using an RGB camera or a depth camera. An individual identification algorithm can be applied to the generated first image to determine whether or not the object detected in the first image is a subject.

[0119] According to one embodiment, when a subject is detected from the first image by the personal identification algorithm, the processor 13 may provide only the portion of the first image in which the subject is moving to the cloud. Alternatively, the processor 13 may provide the entire first image along with temporal information relating to the portion of the first image in which the subject is moving to the cloud.

[0120] According to the embodiment, if multiple objects (people) appear in the first image, and the subject is detected from among the multiple objects by the personal identification algorithm, the processor 13 may provide only the portion in which the subject appears to the cloud. Alternatively, the processor 13 may provide the cloud with the entire first image along with time information relating to the portion of the first image in which the subject appears.

[0121] While the above description assumes the use of personal identification algorithms for object identification, it is not limited to this; facial recognition or gait pattern recognition algorithms can also be used.

[0122] Conversely, the second image obtained by the subject performing the examination action does not require the application of an individual identification algorithm because the subject being photographed is clearly defined.

[0123] Referring to Figure 4, when video footage of the inspection operation is acquired using an RGB camera or a depth camera, that video footage can be generated as a second image.

[0124] Furthermore, the processor 13 can determine whether or not to apply the pose estimation algorithm depending on whether the camera is the first camera or the second camera.

[0125] In other words, the processor 13 can apply a machine learning-based pose estimation algorithm to the image captured by the first camera to acquire two-dimensional or three-dimensional information, and based on this, acquire body coordinate information.

[0126] Referring to Figure 5, when the video captured by the RGB camera (first camera) is processed / analyzed to obtain the first or second image (S51), a posture estimation algorithm is applied to the first or second image (S52), and body coordinate information for the first or second image can be obtained (S53).

[0127] Conversely, for images captured by the second camera, which is capable of providing depth information, the processor 13 can immediately acquire body coordinate information without needing to apply a machine learning-based pose estimation algorithm.

[0128] Referring to Figure 6, when the video captured by the Depth camera (second camera) is processed / analyzed to obtain the first or second image (S61), 3D body coordinate information for the first or second image can be immediately obtained (S62).

[0129] However, although Figure 6 explains that 3D body coordinate information can be obtained immediately from images captured by the Depth camera without applying a pose estimation algorithm, this is not the only option. According to the embodiment, a pose estimation algorithm can also be applied to images captured by the Depth camera. Specifically, 3D body coordinate information can be extracted by integrating depth information obtained by the Depth camera with 2D body coordinate information obtained by applying a pose estimation algorithm to images captured by the Depth camera.

[0130] In steps S51 and S61, if the video captured by the camera is of a routine operation, a first image may be generated based on the operation described with reference to Figure 3, and if it is of an inspection operation, a second image may be generated based on the operation described with reference to Figure 4.

[0131] The processor 13 of the musculoskeletal system evaluation device 10 can calculate an evaluation index for evaluating at least one of the spine and joints based on the body coordinate information obtained by the analysis (S230).

[0132] Here, the evaluation indicators may be set differently depending on the mechanism and characteristics of the degeneration and disease of each vertebra or joint being evaluated.

[0133] For example, when the target of evaluation is the shoulder joint, it is important to evaluate the kinematics of the shoulder complex, so an index for evaluating the range of motion of the shoulder joint may be set as a primary indicator. In addition, an index may be set for the workspace, which quantifies the movement of the shoulder complex in three dimensions in the situation in which a specific movement is performed.

[0134] When the lumbar spine is the target of evaluation, indices can be set to assess the sagittal vertical axis, lumbar lordosis angle, and scoliosis in the coronal plane. When the cervical spine is the target of evaluation, the cervical lordosis angle, chin-brow vertical axis (CBVA), and sagittal vertical axis of the cervical spine are important, and indices can be set to assess these.

[0135] When the knee joint is the target of evaluation, the degree of genu varum / genu valgus deformity and the flexion contracture angle are important, and therefore, indicators can be established to evaluate these. In addition, indicators for lower limb muscle strength can be established by analyzing information related to knee joint movement during specific movements of the subject.

[0136] When the ankle joint is the target of evaluation, the range of motion and dynamic balance of the ankle joint are important, and therefore, indicators can be established to evaluate these.

[0137] Specifically, the processor 13 can determine what kind of evaluation a particular posture or movement contained in the image is for, and calculate evaluation indices for the spine or joints associated with the evaluation posture or movement based on the body coordinate information for the image.

[0138] According to one embodiment, the processor 13 may calculate an evaluation index for evaluating at least one of the spine and joints associated with a particular posture or movement, based on the changes in movement and body coordinate information when the subject takes a specific posture or movement in the first image.

[0139] According to one embodiment, the processor 13 may calculate an evaluation index for evaluating at least one of the spine and joints associated with the posture, based on the changes in movement when the subject takes a specific test action in the second image and the body coordinate information. As described above, the test action may be set as a posture or action associated with the object to be evaluated.

[0140] For example, in the case of the shoulder joint, as mentioned above, an index for evaluating the range of motion of the shoulder joint and an index for the workspace that quantifies the movement of the shoulder complex in three dimensions can be set as evaluation indices. Therefore, the processor 13 can calculate the actual value of the evaluation index for the subject's shoulder joint (e.g., the actual range of motion) based on the body coordinate information that has changed in response to changes in movement in posture or movements related to the shoulder joint during the subject's daily activities, or changes in movement during shoulder joint examination movements.

[0141] In the case of the lumbar spine, as mentioned above, indicators for evaluating the sagittal vertical axis, lumbar lordosis angle, and scoliosis in the coronal plane can be set as evaluation indicators. Therefore, the processor 13 can calculate the actual values ​​(e.g., the actual lumbar lordosis angle) of the subject's lumbar spine evaluation indicators based on body coordinate information that has changed in response to changes in movement in postures or movements related to the lumbar spine during the subject's daily activities, or changes in movement during lumbar spine examinations.

[0142] In the case of the cervical spine, as mentioned above, the cervical lordosis angle, chin-brow vertical axis (CBVA), and sagittal vertical axis of the cervical spine can be set as evaluation indices. Therefore, the processor 13 can calculate the actual values ​​(e.g., the actual cervical lordosis angle) of the subject's cervical spine evaluation indices based on the changes in body coordinate information corresponding to the changes in posture or movement related to the cervical spine in the subject's daily activities, and the changes in movement at c.

[0143] In the case of the knee joint, as mentioned above, the degree of genu varum / genu valgus deformity and the flexion contracture angle can be set as evaluation indicators. Therefore, the processor 13 can calculate the actual values ​​for the evaluation indicators of the subject's knee joint (for example, the actual degree of genu varum / genu valgus deformity) based on body coordinate information that has changed in response to changes in movement in postures or movements related to the knee joint during the subject's daily activities, or changes in movement during lumbar spine examinations.

[0144] In the case of the ankle joint, as mentioned above, the range of motion and dynamic balance of the ankle joint can be set as evaluation indicators. Therefore, the processor 13 can calculate the actual value of the evaluation indicator for the subject's ankle joint (for example, the actual range of motion of the ankle joint) based on the body coordinate information that has changed in response to changes in movement in posture or movements related to the ankle joint during the subject's daily activities, or changes in movement during lumbar spine examination movements.

[0145] The processor 13 of the musculoskeletal system evaluation device 10 can determine the condition of at least one of the subject's spine and joints based on the calculated evaluation index (S240).

[0146] The processor 13 can quantitatively evaluate and report the degree of joint degeneration (aging) based on the calculated evaluation index.

[0147] According to one embodiment, the processor 13 can compare the calculated evaluation index with a previous evaluation index to determine whether there is a deterioration or improvement in at least one of the vertebrae and joints.

[0148] For example, if the actual range of motion of the shoulder joint is used as an evaluation index during shoulder joint assessment, and the currently calculated actual range of motion of the shoulder joint is smaller than the previously calculated actual range of motion of the shoulder joint, it may be concluded that the condition of the subject's shoulder joint has worsened compared to before.

[0149] According to one embodiment, the processor 13 can determine whether the subject of evaluation falls under a pathological condition based on the calculated evaluation index.

[0150] For example, when lumbar lordosis is used as an evaluation index during lumbar spine assessment, if the currently calculated actual lumbar lordosis falls outside a predetermined reference range, the subject's lumbar spine may be judged to be in a pathological state. Here, the reference range may be set for each subject based on the subject's characteristics, such as age, smoking status, weight, previous evaluation results, and presence or absence of related diseases.

[0151] Based on these evaluation results, processor 13 may provide rehabilitation methods for the abnormal spine and / or joints, or suggest a visit to a medical institution.

[0152] According to one embodiment, if there are multiple evaluation indices for a spine or joint associated with a single evaluation posture, the processor 13 may calculate an integrated evaluation score based on pre-set weight values ​​for each of the multiple evaluation indices. Here, the weight values ​​may be set differently for each evaluation indice depending on the degree to which it contributes to the accuracy of the evaluation result.

[0153] For example, the condition of the lumbar spine and knee joint can be evaluated based on an upright posture. In this case, as mentioned above, the sagittal vertical axis of the lumbar spine, the lumbar lordosis angle, the degree of genu varum / genu valgus deformity of the knee joint, and the flexion contracture angle can be set as evaluation indicators. At this time, higher weight values ​​can be assigned to each of the four evaluation indicators in order of their contribution to the accuracy of the evaluation.

[0154] The processor 13 can calculate an integrated evaluation score for the subject's lumbar spine and knee joint condition by applying weight values, which are set differently for the sagittal vertical axis of the lumbar spine, the lumbar lordosis angle, the degree of genu varum / genu valgus deformity of the knee joint, and the flexion contracture angle, respectively, which are calculated based on body coordinate information.

[0155] The processor 13 may determine that there is an abnormality (pathological condition) in the subject being evaluated if the calculated integrated evaluation score is smaller than a pre-set baseline score.

[0156] In such cases, the processor 13 can calculate each of the multiple evaluation metrics without reapplying weight values, compare the calculated values ​​to their respective preset reference ranges, and determine that evaluation metrics outside the reference range are factors causing anomalies. The processor 13 can provide a training method for the evaluation metrics determined to be factors causing anomalies.

[0157] Specifically, if the integrated evaluation score for the subject's lumbar spine and knee joint condition is lower than a predetermined baseline score, the processor 13 may compare each of the evaluation indices—the sagittal vertical axis of the lumbar spine, the lumbar lordosis angle, the degree of genu varum / genu valgus deformity of the knee joint, and the flexion contracture angle—with the baseline range. If any one of the four evaluation indices (for example, the lumbar lordosis angle) falls outside the baseline range, the processor 13 may determine that the lumbar lordosis angle is the cause of the abnormality and provide the subject with intensive training methods for that evaluation indice.

[0158] The following section will describe a system that includes the cloud, as explained in Figure 1, with reference to Figure 7.

[0159] Figure 7 is a diagram illustrating a musculoskeletal system evaluation system 2 including a cloud according to one embodiment of the present disclosure.

[0160] Referring to Figure 7, System 2 may include a musculoskeletal system evaluation device 10, a camera 20, a user terminal 30, and a cloud 40. However, in some embodiments, System 2 may include fewer or more components than those shown in Figure 7.

[0161] The operation of the musculoskeletal system evaluation device 10, camera 20, and user terminal 30 shown in Figure 7 is the same as that described with reference to Figures 1 to 6, so a detailed explanation is omitted.

[0162] The cloud 40 added in Figure 7 can work in conjunction with the communication unit 11 of the musculoskeletal system evaluation device 10 to save and store the first or second image transmitted from the communication unit 11.

[0163] Cloud 40 may store the entire first image, or it may store the first image converted to a lower resolution or with a reduced fps (frames per second), or only the skeleton of the first image may be stored.

[0164] In other words, according to the embodiment, the first image is stored in the cloud 40, but it can be stored in at least one of the following forms: the entire first image, the first image converted to a lower resolution, the first image with reduced fps (frames per second), and the skeleton information of the first image.

[0165] Furthermore, the first image stored in the cloud may be volatile after a certain period of time for privacy protection.

[0166] Although only the storage method for the first image was described above, the second image can also be transmitted and stored in various forms using a similar method.

[0167] According to one embodiment, the processor 13 may provide only the moving portion of the first image to the cloud 40. Alternatively, the processor 13 may provide the cloud 40 with the entire first image along with time information relating to the moving portion of the first image.

[0168] According to one embodiment, if multiple objects (people) appear in the first image, the processor 13 may recognize the subject based on stored identification information and provide only the portion in which the subject appears to the cloud 40. Alternatively, the processor 13 may provide the cloud 40 with the entire first image along with time information relating to the portion of the first image in which the subject appears.

[0169] As mentioned above, the cloud 40 can also serve the role of saving and storing images, and according to the embodiment, the cloud 40 can also perform the analysis / evaluation operation of the processor 13 and the detection / decision operation of the activity classifier as described above.

[0170] Although Figure 2 shows the steps being performed sequentially, this is merely an illustrative explanation of the technical concept of this embodiment. Anyone with ordinary skill in the technical field to which this embodiment belongs can modify and adapt the steps shown in Figure 2 in various ways, such as changing the order or performing them in parallel, as long as it does not deviate from the essential characteristics of this embodiment. Therefore, Figure 2 is not limited to a chronological order.

[0171] On the other hand, in the description above, the steps shown in Figure 2 can be further divided into additional steps or combined into fewer steps according to embodiments of the present disclosure. In addition, some steps can be omitted as needed, and the order between steps can be changed.

[0172] On the other hand, the disclosed embodiments may be implemented in the form of a recording medium for storing computer-executable instructions. The instructions are stored in the form of program code and, when executed by a processor, can generate program modules that perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0173] Computer-readable recording media include all types of recording media that store instructions that can be deciphered by a computer. Examples include ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0174] Embodiments of this disclosure have been described above with reference to the accompanying drawings. A person with ordinary skill in the art to which this disclosure pertains will understand that this disclosure may be carried out in forms different from those disclosed without altering the technical idea or essential features of this disclosure. The disclosed embodiments are illustrative and should not be constrained.

Claims

1. Memory containing at least one process for performing musculoskeletal system assessment using images and algorithms; and Includes a processor that performs operations according to the process described above, The aforementioned processor, The system acquires images captured by a camera, calculates an evaluation index for evaluating at least one of the subject's spine and joints based on the body coordinate information obtained from the images, and determines the state of at least one of the spine and joints based on the calculated evaluation index. The aforementioned image is a musculoskeletal system evaluation device using images and algorithms, which includes a first image acquired when the evaluation mode is the daily activity mode and a second image acquired when the evaluation mode is the examination activity mode.

2. The musculoskeletal system evaluation device using images and algorithms according to claim 1, wherein the camera includes a first camera capable of acquiring three-dimensional information by a posture estimation algorithm and a second camera capable of providing depth information.

3. The aforementioned processor, Based on the three-dimensional information obtained by applying the posture estimation algorithm to the image captured by the first camera, three-dimensional body coordinate information is acquired. A musculoskeletal system evaluation device using images and an algorithm according to claim 2, wherein two-dimensional body coordinate information obtained by applying the posture estimation algorithm to an image captured by the second camera and depth information obtained by the second camera are integrated to obtain three-dimensional body coordinate information.

4. The first image includes a specific posture or movement from the subject's daily activities that is relevant to the evaluation. The aforementioned specific posture or movement is determined by an activity classifier. The musculoskeletal system evaluation device using the image and algorithm according to claim 1, wherein the second image is an image taken when the subject performs a pre-set test action for the specific posture or movement.

5. The aforementioned processor, Based on the changes in movement when the subject assumes the specific posture or movement in the first image, and the changes in body coordinate information corresponding to those changes in movement, an evaluation index is calculated to evaluate at least one of the spine and joints associated with the movement. A musculoskeletal system evaluation device using images and an algorithm according to claim 4, which compares the calculated evaluation index with a previous evaluation index to determine whether there is deterioration or improvement in at least one of the vertebrae and joints.

6. The aforementioned processor, Based on the changes in movement when the subject performs the examination action in the second image and the body coordinate information that changes in response to those changes in movement, an evaluation index is calculated to evaluate at least one of the spine and joints associated with the relevant action. A musculoskeletal system evaluation device using images and an algorithm according to claim 4, which compares the calculated evaluation index with a previous evaluation index to determine whether there is deterioration or improvement in at least one of the vertebrae and joints.

7. The aforementioned evaluation indicators are: A musculoskeletal system evaluation device using images and algorithms according to claim 1, which are set differently according to the mechanism and characteristics of degeneration and disease for each vertebra or joint being evaluated.

8. When the target of evaluation is the shoulder joint, the evaluation index includes an index for evaluating the range of motion of the shoulder joint and an index for the workspace that quantifies the movement of the shoulder complex in three dimensions. When the subject of evaluation is the lumbar spine, the evaluation index includes an index for evaluating the sagittal vertical axis of the lumbar spine, the lumbar lordosis angle, and the scoliosis in the coronal plane. When the subject of evaluation is the cervical spine, the evaluation index includes an index for evaluating the cervical lordosis angle, the chin-brow vertical axis (CBVA), and the sagittal vertical axis of the cervical spine. When the subject of evaluation is the knee joint, the evaluation index includes an index for evaluating the degree of deformation of genu varum and genu valgus and the flexion contracture angle. The musculoskeletal system evaluation device using images and algorithms according to claim 7, wherein the evaluation index when the target of evaluation is the ankle joint includes an index for evaluating the range of motion and dynamic balance of the ankle joint.

9. A musculoskeletal system evaluation device using an image and algorithm according to claim 1, wherein the cloud, which is linked to the communication unit of the device, stores at least one of the following forms: the entire first image, the first image converted to a low resolution, the first image with reduced fps (frames per second), and skeleton information of the first image.

10. In a method performed by the apparatus, The stage of acquiring images captured by a camera; A step of calculating an evaluation index for evaluating at least one of the subject's spine and joints based on the body coordinate information obtained from the aforementioned image; and The step includes determining the condition of at least one of the spine and the joints based on the calculated evaluation index, The aforementioned image is a musculoskeletal system evaluation method using an image and an algorithm, the image being a first image acquired when the evaluation mode is the daily activity mode and a second image acquired when the evaluation mode is the examination activity mode.

11. The musculoskeletal system evaluation method using images and algorithms according to claim 10, wherein the cameras include a first camera capable of acquiring three-dimensional information by a posture estimation algorithm and a second camera capable of providing depth information.

12. The step of analyzing the aforementioned image is: Based on the three-dimensional information obtained by applying the posture estimation algorithm to the image captured by the first camera, three-dimensional body coordinate information is acquired. A method for evaluating the musculoskeletal system using an image and algorithm according to claim 11, wherein two-dimensional body coordinate information obtained by applying the posture estimation algorithm to an image captured by the second camera and depth information obtained by the second camera are integrated to obtain three-dimensional body coordinate information.

13. The first image includes a specific posture or movement from the subject's daily activities that is relevant to the evaluation. The aforementioned specific posture or movement is determined by an activity classifier. The musculoskeletal system evaluation method using the image and algorithm according to claim 10, wherein the second image is an image taken when the subject performs a pre-set test action for the specific posture or movement.

14. The step of calculating the aforementioned evaluation index is: Based on the changes in movement when the subject assumes the specific posture or movement in the first image, and the changes in body coordinate information corresponding to those changes in movement, an evaluation index is calculated to evaluate at least one of the spine and joints associated with the movement. The step of determining the aforementioned state is: A method for evaluating the musculoskeletal system using images and an algorithm according to claim 13, wherein the calculated evaluation index is compared with a previous evaluation index to determine whether there is deterioration or improvement in at least one of the vertebrae and joints.

15. The step of calculating the aforementioned evaluation index is: Based on the changes in movement when the subject performs the examination action in the second image and the body coordinate information that changes in response to those changes in movement, an evaluation index is calculated to evaluate at least one of the spine and joints associated with the relevant action. The step of determining the aforementioned state is: A method for evaluating the musculoskeletal system using images and an algorithm according to claim 13, wherein the calculated evaluation index is compared with a previous evaluation index to determine whether there is deterioration or improvement in at least one of the vertebrae and joints.

16. The aforementioned evaluation indicators are: A method for evaluating the musculoskeletal system using images and an algorithm according to claim 10, which are set differently according to the mechanism and characteristics of degeneration and disease for each vertebra or joint being evaluated.

17. When the target of evaluation is the shoulder joint, the evaluation index includes an index for evaluating the range of motion of the shoulder joint and an index for the workspace that quantifies the movement of the shoulder complex in three dimensions. When the subject of evaluation is the lumbar spine, the evaluation index includes an index for evaluating the sagittal vertical axis of the lumbar spine, the lumbar lordosis angle, and the scoliosis in the coronal plane. When the subject of evaluation is the cervical spine, the evaluation index includes an index for evaluating the cervical lordosis angle, the chin-brow vertical axis (CBVA), and the sagittal vertical axis of the cervical spine. When the subject of evaluation is the knee joint, the evaluation index includes an index for evaluating the degree of deformation of genu varum and genu valgus and the flexion contracture angle. The musculoskeletal system evaluation method using images and algorithms according to claim 16, wherein the evaluation index when the target of evaluation is the ankle joint includes an index for evaluating the range of motion and dynamic balance of the ankle joint.

18. A method for evaluating the musculoskeletal system using an image and algorithm according to claim 10, wherein the cloud, which is linked to the communication unit of the device, stores at least one of the following forms: the entire first image, the first image converted to a low resolution, the first image with reduced fps (frames per second), and skeleton information of the first image.

19. A computer program stored on a computer-readable recording medium, wherein, when executed by one or more processors, the computer program performs an operation for performing a musculoskeletal system evaluation method using images and algorithms, the operation being: The action of acquiring an image captured by a camera; The process of calculating an evaluation index for evaluating at least one of the subject's spine and joints based on the body coordinate information obtained from the aforementioned image; and This includes an action to determine the state of at least one of the spine and the joints based on the calculated evaluation index, The aforementioned image is a computer program stored on a computer-readable recording medium, which includes a first image acquired when the evaluation mode is the normal operation mode and a second image acquired when the evaluation mode is the inspection operation mode.

Images