Apparatus and method for evaluating thoracic kyphosis and lumbar lordosis based on deep neural network

Abstract

The present invention relates to a machine learning-based method and apparatus for determining spinal anterior-posterior curvature, and more specifically, to a machine learning-based method and apparatus for determining spinal anterior-posterior curvature that allows a patient or user to easily determine whether the spine is lordotic or kyphotic based on an image captured by a standard camera without the risk of radiation exposure.

Description

Technology Field

[0001] The present invention relates to a machine learning-based method and apparatus for determining spinal anterior-posterior curvature, and more specifically, to a machine learning-based method and apparatus for determining spinal anterior-posterior curvature that allows a patient or user to easily determine whether the spine is lordotic or kyphotic based on an image captured by a general camera (a camera based on an RGB image sensor) without the risk of radiation exposure. Background Technology

[0002] Generally, in clinical practice, a lateral view of the spine is taken to check the curvature of the spine. Key points representing the sacrum, lumbar spine, and thoracic spine are identified in the X-ray image, vertical lines are drawn, and the angles between them are calculated to evaluate the degree of anterior-posterior curvature of the lumbar and thoracic spines.

[0003] However, alternative methods are needed because X-rays and CT scans pose a problem where imaging is impossible for subjects with dangerous or fatal radiation exposure, such as pregnant women.

[0004] For example, the prior art (Republic of Korea Registered Patent No. 10-2389067) is a technology for evaluating spinal diseases based on radiographic images, and is a technology for determining scoliosis rather than lordosis or kyphosis of the spine based on radiographic images taken from the front or back of the patient. Prior art literature

[0005] Republic of Korea Registered Patent No. 10-2389067 The problem to be solved

[0006] Accordingly, the present invention is proposed to allow a general person to easily determine whether the spine is lordotic or kyphotic based on an image acquired primarily by a general two-dimensional camera (a camera based on an RGB image sensor) before visiting a hospital, and aims to provide a device, system, method, computer-readable recording medium, and computer program for determining lordotic or kyphotic spine.

[0007] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0008] A machine learning-based method for determining anterior-posterior only of a spine according to an embodiment of the present invention for achieving the above-mentioned purpose is a method in which at least a portion of each step is performed by a processor, and may include the step of acquiring a side image of a subject's body; the step of inputting the side image into a machine learning model trained to determine key points for a spine for determining anterior-posterior only of the spine, and estimating key points for at least some vertebrae of the subject's spine in the side image; the step of determining a plurality of vectors and a plurality of orthogonal lines perpendicular to the vectors based on the key points; and the step of determining anterior-posterior only of the subject's spine based on the plurality of orthogonal lines.

[0009] A machine learning-based spine anterior-posterior determination device according to another embodiment of the present invention includes a memory unit that stores a learned machine learning model trained to determine key points for spine anterior-posterior determination of a side image of a subject's body, a communication unit that transmits and receives information to and from an external device, and a processor that controls the memory and the communication unit. The processor acquires the side image, estimates key points for at least some vertebrae of the subject's spine in the side image through the machine learning model, and can determine the spine anterior-posterior of the subject based on the key points. Effects of the invention

[0010] According to the machine learning-based method and apparatus for determining spinal anterior or posterior curvature according to an embodiment of the present invention, a side image of a person is input into a machine learning model to estimate key points for determining spinal lordosis or kyphosis from the side image, and the anterior or posterior curvature of the subject's spine can be determined based on vector information based on the estimated key points.

[0011] Accordingly, the user can determine spinal scoliosis or spinal kyphosis simply and quickly by using a terminal such as a smartphone and the machine learning-based spinal anterior-posterior determination method according to the present invention, and by simply capturing and inputting a side image.

[0012] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0013] FIG. 1 is a block diagram illustrating the configuration of a machine learning-based spine anterior-posterior determination system according to one embodiment of the present invention. FIG. 2 is a block diagram illustrating the configuration of a machine learning-based spine anterior-posterior only determination device according to an embodiment of the present invention. FIG. 3 is a conceptual diagram illustrating a series of processes for determining only the anterior and posterior aspects of the spine according to one embodiment of the present invention. FIG. 4 is an illustrative diagram showing an example of a key point, a vector, and an orthogonal line determined according to an embodiment of the present invention. FIG. 5 illustrates an example of a spinal anterior-posterior determination display shown on a side image according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for determining only the anterior and posterior aspects of the spine according to an embodiment of the present invention. Figure 7 is a conceptual diagram of a pose evaluation solution for acquiring a lateral image using a pose estimation model. Specific details for implementing the invention

[0014] The objectives and effects of the present invention, and the technical configurations for achieving them, will become clear by referring to the embodiments described in detail below in conjunction with the accompanying drawings. In describing the present invention, if it is determined that a detailed description of known functions or configurations may unnecessarily obscure the essence of the invention, such detailed description will be omitted. Furthermore, the terms described below are defined considering the structure, role, and function, etc., in the present invention, and these may vary depending on the intentions or conventions of the user or operator.

[0015] However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims set forth in the patent claims. Therefore, such definition must be based on the content throughout this specification.

[0016] Throughout the specification, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0017] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings.

[0019] FIG. 1 is a block diagram illustrating the configuration of a machine learning-based spine anterior-posterior determination system according to an embodiment of the present invention, FIG. 2 is a block diagram illustrating the configuration of a machine learning-based spine anterior-posterior determination device according to an embodiment of the present invention, FIG. 3 is a conceptual diagram illustrating a series of processes for determining spine anterior-posterior only according to an embodiment of the present invention, FIG. 4 illustrates an example of a key point, vector, and orthogonal line determined according to an embodiment of the present invention, and FIG. 5 illustrates an example of a spine anterior-posterior only display shown on a side image according to an embodiment of the present invention.

[0021] Referring to FIG. 1, a machine learning-based spine anterior-posterior determination system (1) according to one embodiment of the present invention may include a camera device (100), a spine anterior-posterior determination device (200), and an output device (300).

[0022] Each of the camera device (100), the spine anterior-posterior determination device (200), and the output device (300) may be equipped with a communication module or an input / output interface for connecting to a communication network for mutual communication. Additionally, the camera device (100), the spine anterior-posterior determination device (200), and the output device (300) may be mounted together on a single smart device or may be mounted separately on different devices.

[0024] The camera device (100) may be provided as a device that acquires images based on an RGB image sensor, such as a general RGB camera, a camera sensor equipped in a smart device, or a general two-dimensional camera.

[0025] The camera device (100) can obtain a side image by photographing the side of the subject. At this time, unlike X-ray imaging, the subject can obtain a side image through the camera device (100) even while wearing clothes.

[0027] The spine anterior-posterior only determination device (200) may include a memory unit (210), a communication unit (220), and a processor (230) as shown in FIG. 2.

[0029] The memory unit (210) can store a machine learning model trained to determine key points for the spine in a side image.

[0030] The communication unit (220) can transmit and receive information to and from external devices, such as a camera device (100) or an output device (300).

[0031] The processor (230) can control the overall operation of the spine front-and-back only judgment device (200).

[0032] The processor (230) can load a machine learning model and information necessary to execute the machine learning model from the memory unit (210). Additionally, the processor (230) can load information necessary to execute the machine learning model from the memory unit (210).

[0033] In one embodiment, the processor (230) may include a filtering unit (232), a judgment unit (234), and a learning unit (236), and may additionally include an evaluation unit (238).

[0035] The filtering unit (232) receives a side image from the camera device (100) through the communication unit (220) and can filter out noise in the side image.

[0036] In one embodiment, the filtering unit (232) can detect unnecessary noise images based on a learned machine learning model and filter the detected noise images.

[0038] The judgment unit (234) acquires a side image of the subject's body with the noise image filtered, inputs the side image into a machine learning model trained to determine key points for determining the anterior-posterior only of the spine, estimates key points for at least some of the vertebrae of the subject's spine in the side image, and can determine the anterior-posterior only of the subject's spine based on the estimated key points.

[0039] To this end, first, a machine learning model can be trained by the learning unit (236).

[0040] Below, the machine learning model is described as being trained in the spine anterior-posterior only determination device (200) as an example, but the machine learning model is trained in a separate training device and the trained machine learning model is not excluded from being distributed to and utilized in the spine anterior-posterior only determination device (200).

[0041] In one embodiment, the learning unit (236) can train a machine learning model using a plurality of lateral images in which specific points of the spine are labeled as training data. Here, the specific points of the spine can be pre-read by a specialist and set on the lateral images.

[0043] Here, specific points of the spine can be set by referring to the vertebra numbers according to accredited medical information defined for the spine in the medical industry, for example.

[0044] In one embodiment, the learning unit (236) can store the network structure and initial parameters of the machine learning model before it is learned by specific learning data in a database.

[0045] Initial parameters may include initial inter-node weights, initial inter-node biases, mini-batch size, number of training iterations, and learning rate. Additionally, model parameters may include inter-node weights and inter-node biases. When performing transfer learning, the network structure prior to the transfer learning may be stored in a database. Furthermore, information regarding patient details, body parts, and diseases may be included for each lateral image.

[0046] Subsequently, the learning unit (236) inputs learning data in which key points for determining the anterior-posterior relationship of the spine are labeled on a human side image, and can perform learning based on a learning optimization algorithm such as Gradient Descent (GD), Stochastic Gradient Descent (SGD), Momentum, Nesterov Accelerate Gradient (NAG), Adagrad, AdaDelta, RMSProp, Adam, Nadam, etc., using a loss function such as Mean Squared Error (MSE) or Cross Entropy Error (CEE). The present invention is not limited to the loss function or learning optimization algorithm given as an example above, but can use various types of loss function or learning optimization algorithm.

[0047] In addition, the learning unit (236) can use a transfer learning technique to further train a deep neural network (e.g., ResNet, etc.) that has already learned data from numerous images on key points representing specific bones of the spine according to one embodiment of the present invention, thereby creating a deep neural network dedicated to determining only the anterior and posterior aspects of the spine.

[0049] The judgment unit (234) can estimate key points corresponding to pre-set vertebrae in the side image through the machine learning model learned in this way. That is, it can estimate key points for specific vertebra numbers.

[0050] In one embodiment, the judgment unit (234) can estimate a pair of key points located vertically relative to the side, corresponding to a preset vertebral bone of the subject's spine as shown in FIG. 4 (a).

[0051] Key points are information corresponding to the location of at least some bones of the subject's spine on a lateral image, as in FIG. 3 (a), and can be indicated by a pair of points corresponding to the location. For example, markings representing the key points can be displayed by overlapping them on the lateral image.

[0053] Next, the judgment unit (234) can determine multiple vectors based on key points estimated through a machine learning model.

[0054] That is, the judgment unit (234) can determine a vector connecting key points located on the same vertebra as illustrated in (b) of FIG. 4.

[0055] In one embodiment, a vector can be determined as shown in FIG. 3(b) by a straight line connecting each pair of key points corresponding to the positions of the 1st thoracic vertebra, the 12th thoracic vertebra, and the 1st sacral vertebra. Accordingly, it is decided to name the first vector (VT1) corresponding to the 1st thoracic vertebra, the second vector (VT2) corresponding to the 12th thoracic vertebra, and the third vector (VT3) corresponding to the 1st sacral vertebra.

[0057] Next, the judgment unit (234) can determine an orthogonal line perpendicular to each vector, as shown in (c) of FIG. 4.

[0058] That is, the judgment unit (234) can determine an orthogonal line that is perpendicular to the vector and passes through the midpoint of a key point located on the same vertebra.

[0059] In one embodiment, the judgment unit (234) can determine a first orthogonal line (OR1) perpendicular to the first vector (VT1), a second orthogonal line (OR2) perpendicular to the second vector (VT2), and a third orthogonal line (OR3) perpendicular to the third vector (VT3), as shown in (c) of FIG. 3.

[0060] Next, the judgment unit (234) can calculate the angle formed by the plurality of orthogonal lines and determine only the anterior and posterior aspects of the subject's spine based on the calculated angle.

[0061] Specifically, the judgment unit (234) can determine thoracic kyphosis based on the first angle formed by the first orthogonal line (OR1) and the second orthogonal line (OR2), and determine lumbar lordosis based on the second angle formed by the second orthogonal line (OR2) and the third orthogonal line (OR3). FIG. 5 illustrates an example of the first angle and the second angle displayed on a side image.

[0062] At this time, the judgment unit (234) may use a table in which diagnostic criteria and a range for distinguishing the severity (degree) of thoracic kyphosis based on the range of the first angle are stored in advance, and likewise may use a table in which diagnostic criteria and a range for distinguishing the severity (degree) of lumbar lordosis based on the range of the second angle are stored in advance. The tables may be stored in the memory unit (210).

[0063] The information stored in the tables for each of these thoracic kyphosis and lumbar lordosis may be information pre-set by the opinions of experts or medical staff or medical information.

[0064] In this way, the mark generated by processing on the image to determine whether it is only the anterior-posterior spine and the result of determining only the anterior-posterior spine can be output through the output device (300) and provided to the user.

[0065] Since the present invention utilizes a first angle and a second angle as parameters for determining only the anterior-posterior aspect of the spine, it becomes possible to precisely determine only the anterior-posterior aspect of the spine regardless of the subject's height, volume, or weight. In other words, when determining only the anterior-posterior aspect of the subject's spine, errors caused by differences in relative height, volume, and weight compared to the subjects used for training can be eliminated.

[0067] The evaluation unit (238) evaluates the artificial intelligence model generated from the learning unit (236) and the result image derived through the artificial intelligence model to determine how generalizable the artificial intelligence model is to new data.

[0068] The level of prediction accuracy can be evaluated using metrics including accuracy and loss, and whether it is optimized only for the dataset used in machine learning can be evaluated. The evaluation method may include known or to be known evaluation methods for machine learning.

[0069] In one embodiment, the range of the metric is set by segmenting only the area containing key points corresponding to the vertebrae of the target number, and the number of pixels in that area, and the pixel error can be determined according to the accuracy of the key point location.

[0070] Based on this, the accuracy of the artificial intelligence model was calculated by dividing the data of 17 subjects into training (15 people) and testing (2 people) in a 9:1 ratio, and the training error was found to be 2.36 pixels and the test error was 4.04 pixels.

[0071] The output device (300) can display and output information such as a side image of a subject captured through a camera device (100), an analysis process by a spine anterior-posterior only determination device (200), and analysis results on the side image.

[0072] The output device (300) may be a display of a smart device or an electronic device equipped with a display connected to a smart device, but is not limited thereto.

[0074] As such, the spine anterior-posterior only determination system (1) of the present invention can be a system that estimates the location of a specific bone number of the spine and determines only the anterior-posterior only of the spine when the input image is an image related to the spine.

[0075] In addition, the physician's opinion regarding the subject's lateral image can be stored and managed as class labels for each lateral image for the machine learning of the present invention. By training the machine learning model using lateral images of specific vertebra numbers and their corresponding class labels, it becomes possible to determine only the anterior-posterior relationship of the subject's spine based solely on the lateral images.

[0076] Therefore, a patient or user can determine the anterior-posterior spine of a subject simply by using a terminal they possess to access the anterior-posterior spine system according to the present invention or a platform using the same, and inputting a side image of the subject captured through a standard camera.

[0078] The spine anterior-posterior determination system (1) according to the present invention can provide a service that determines the anterior-posterior of a subject's spine by inputting the corresponding side image into a prediction model when a specific user connects via a wired or wireless communication network and transmits a query (a query regarding the anterior-posterior of a subject's spine and the degree thereof by inputting a side image). Alternatively, as described above, it can be implemented on a user's smart device (i.e., a smart device on which a spine anterior-posterior determination application including a trained machine learning model is distributed), and can determine whether the user's spine is anterior-posterior based on a photo taken directly by the user with the smart device.

[0080] The spinal anterior-posterior determination system (1) provided in the present invention is equipped with a program that performs various functions for image reading according to the present invention, and provides an Application Program Interface (API) to provide various services through an image reading platform.

[0081] From this, each service developer can develop desired services using the aforementioned API. For example, each developer can develop an app that estimates and provides images. It is also possible to configure the system to provide various types of health experiences, body shape management services, clinical decision support services, and experience and play services through the API for determining the anterior-posterior spine provided by the anterior-posterior spine determination system of the present invention.

[0082] In other words, the spine anterior-posterior only determination system (1) according to one embodiment of the present invention is a platform that enables a service developer to develop a desired service and to generate revenue by providing the developed service to a user.

[0083] The spine anterior-posterior determination system (1) installs an app for determining spine anterior-posterior only on a PC, tablet, mobile terminal, etc., and by running the app to operate a service through spine anterior-posterior determination, the user can estimate whether there is spine anterior-posterior only and the severity from their own image.

[0084] In addition, the spine anterior-posterior determination system (1) according to one embodiment of the present invention can not only read images of each part of the spine but also analyze time-series changes in the corresponding spine part based on the read images, predict future conditions, and suggest the most suitable treatment method for the current condition.

[0086] FIG. 6 is a flowchart illustrating a method for determining only the anterior and posterior aspects of the spine according to an embodiment of the present invention.

[0087] A method for determining only the anterior-posterior aspect of a spine according to one embodiment of the present invention may be carried out in substantially the same configuration as the anterior-posterior aspect determination device (200) of FIG. 1. Accordingly, components identical to the anterior-posterior aspect determination device (200) of FIG. 1 are given the same reference numerals, and repetitive descriptions are omitted.

[0088] In addition, the method for determining only the anterior-posterior aspect of the spine according to the present embodiment can be executed by software (application) for performing anterior-posterior aspect determination of the spine using machine learning.

[0089] First, the spine anterior-posterior only determination device can acquire a side image (S110). For example, a side image can be acquired by taking a general RGB camera, a camera sensor equipped in a smart device, a general 2D camera, etc.

[0090] Next, a lateral image is input into a machine learning model trained to determine key points for determining the anterior-posterior relationship of the spine, thereby allowing the estimation of key points for at least some of the vertebrae of the subject's spine in the lateral image (S120).

[0091] To do this, a machine learning model can first be trained.

[0092] In one embodiment, a machine learning model can be trained using a plurality of lateral images labeled with specific points of the spine as training data. Here, the specific points of the spine may be set by prior reading by a specialist, set according to accredited medical information defined for the spine in the medical industry, or set based on bone numbers.

[0093] Key points for specific vertebra numbers can be established through a machine learning model trained in this manner. For example, key points can be marked as a pair of points at each location of at least a portion of the subject's spine on a lateral image, as shown in FIG. 3(a) and FIG. 4(a). That is, a pair of key points located vertically relative to the lateral side on a pre-set vertebra of the subject's spine can be estimated. For example, markings representing the key points can be displayed by overlapping them on the lateral image.

[0095] Next, based on the key points above, a plurality of vectors and a plurality of orthogonal lines orthogonal to each of the above vectors can be determined (S130).

[0096] In other words, based on key points extracted through a machine learning model, a vector connecting key points located on the same vertebra can be determined.

[0097] In one embodiment, a vector can be determined as shown in FIG. 3(b) by a straight line connecting each pair of key points corresponding to the positions of the 1st thoracic vertebra, the 12th thoracic vertebra, and the 1st sacral vertebra. Accordingly, it is decided to name the first vector (VT1) corresponding to the 1st thoracic vertebra, the second vector (VT2) corresponding to the 12th thoracic vertebra, and the third vector (VT3) corresponding to the 1st sacral vertebra. Then, an orthogonal line perpendicular to each vector can be generated.

[0098] In one embodiment, an orthogonal line can be determined that passes through the midpoint of a key point located on the same vertebra and is perpendicular to the vector.

[0099] In one embodiment, as shown in FIG. 3 (c), a first orthogonal line (OR1) perpendicular to the first vector (VT1), a second orthogonal line (OR2) perpendicular to the second vector (VT2), and a third orthogonal line (OR3) perpendicular to the third vector (VT3) can be determined.

[0101] Next, the angle formed by multiple orthogonal lines can be calculated (S140).

[0102] In one embodiment, a first angle formed by a first orthogonal line (OR1) and a second orthogonal line (OR2), and a second angle formed by a second orthogonal line (OR2) and a third orthogonal line (OR3) can be calculated. FIG. 5 illustrates an example of the first angle and the second angle displayed on a side image of a subject.

[0104] Next, based on the angle calculated above, only the anterior and posterior aspects of the subject's spine can be determined (S150).

[0105] Specifically, referring to FIG. 5, thoracic kyphosis can be determined based on the first angle formed by the first orthogonal line (OR1) and the second orthogonal line (OR2), and lumbar lordosis can be determined based on the second angle formed by the second orthogonal line (OR2) and the third orthogonal line (OR3).

[0106] At this time, to determine the anterior-posterior aspect of the spine, a table in which diagnostic criteria and a range for distinguishing the severity (degree) can be determined based on the range of the first angle may be used, and likewise, a table in which diagnostic criteria and a range for distinguishing the severity (degree) can be determined based on the range of the second angle may be used. The table may be stored in the memory unit (210).

[0107] The information stored in the tables for each of these thoracic kyphosis and lumbar lordosis may be information pre-set by the opinions of experts or medical staff or medical information.

[0108] Since the first and second angles are used as parameters to determine only the anterior-posterior aspect of the spine, it becomes possible to precisely determine only the anterior-posterior aspect of the spine regardless of the subject's height, volume, or weight. In other words, when determining only the anterior-posterior aspect of the spine, errors caused by differences in relative height, volume, and weight compared to the subjects used for training can be eliminated.

[0109] Next, the result of determining only the anterior-posterior aspect of the spine can be output to the user. In one embodiment, the mark generated by processing on the image to determine whether it is only the anterior-posterior aspect of the spine and the result of determining only the anterior-posterior aspect of the spine can be output through an output device and provided to the user.

[0111] FIG. 7 is a conceptual diagram of a posture evaluation solution that determines whether a side image for determining anterior-posterior spinal position was taken in an accurate posture based on a machine learning model for estimating posture. In one embodiment, the posture evaluation solution may be used together with the aforementioned program for determining anterior-posterior spinal position, and may be, for example, part of the function of an application.

[0112] Referring to FIG. 7, the present invention may be based on the following method to obtain a side image of the aforementioned subject.

[0113] First, as shown in FIG. 7 (a), the user positions the camera so that the subject's entire body is included in the field of view (FoV). If posture points corresponding to the user's entire body are not acquired in the images captured in real-time by the camera based on the previously described machine learning model for estimating posture, the processor (230) may display a message to the user to adjust the camera's field of view. Then, the user is advised to wear clothes that fit as closely to the body as possible or to remove the upper body, and is instructed to stand with their side facing the camera screen, look straight ahead, keep their knees straight, and assume a standing posture with their arms crossed or positioned in front of their chest.

[0114] The processor (230) applies a captured side image of the subject or a real-time captured image to a machine learning model that estimates the posture, and checks through the side image whether the subject's knee is extended and whether the position of the head is higher than the position of the body.

[0115] More specifically, by comparing and verifying whether the vertical height of the posture point (17) corresponding to the head among the posture points of the open pose shown in FIG. 7 (b) is located higher than the posture point (5) corresponding to the shoulder, it can be determined whether the position of the head is higher than the position of the body.

[0116] Additionally, the processor (230) can determine whether the subject's knee is in an extended state based on the angle between the line segments connected to the posture point corresponding to the knee among the posture points corresponding to the lower body.

[0117] After determining whether the subject's shooting posture is appropriate through this series of processes, a side image is acquired and applied to the aforementioned deep learning-based machine learning model to determine whether the spine is posterior or posterior.

[0118] The aforementioned spinal anterior-posterior determination system may be implemented by a computing device comprising at least some of a processor, memory, a user input device, and a presentation device. Memory is a medium for storing computer-readable software, applications, program modules, routines, instructions, and / or data, etc., which are coded to perform specific tasks when executed by a processor. The processor may read and execute computer-readable software, applications, program modules, routines, instructions, and / or data, etc., stored in memory.

[0119] A user input device may be a means for a user to input commands to the processor to execute a specific task or to input data necessary for the execution of a specific task. The user input device may include a physical or virtual keyboard or keypad, key buttons, a mouse, a joystick, a trackball, a touch-sensitive input means, or a microphone, etc. A presentation device may include a display, a printer, a speaker, or a vibration device, etc.

[0120] A computing device may include various devices such as smartphones, tablets, laptops, desktops, servers, and clients. A computing device may be a single stand-alone device, or it may include multiple computing devices operating in a distributed environment composed of multiple computing devices that cooperate with each other through a communication network.

[0121] In addition, the above-described method for determining only the anterior-posterior aspect of a spine may be executed by a computing device having a processor and a memory storing computer-readable software, an application, a program module, a routine, an instruction, and / or a data structure, etc., coded to perform a method for determining only the anterior-posterior aspect of a spine utilizing an agent model when executed by the processor.

[0122] The embodiments described above may be implemented through various means. For example, the embodiments may be implemented by hardware, firmware, software, or a combination thereof.

[0123] In the case of implementation by hardware, the image diagnosis method utilizing the agent model according to the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, or microprocessors.

[0124] For example, the method for determining only the anterior-posterior aspect of the spine according to the embodiments may be implemented using a machine learning semiconductor device in which neurons and synapses of a deep neural network are implemented using semiconductor devices. In this case, the semiconductor devices may be currently used semiconductor devices, such as SRAM, DRAM, or NAND, or next-generation semiconductor devices, such as RRAM, STT MRAM, or PRAM, or a combination thereof.

[0125] When implementing the method for determining only the anterior-posterior aspect of the spine according to the embodiments using a machine learning semiconductor device, the result (weight) of training an agent model with software may be transferred to a synapse mimicking device arranged in an array, or training may be performed on the machine learning semiconductor device.

[0126] In the case of implementation by firmware or software, the method for determining only the anterior-posterior aspect of the spine according to the embodiments may be implemented in the form of a device, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory unit and executed by a processor. The memory unit may be located inside or outside the processor and may exchange data with the processor by various means already known.

[0127] Additionally, terms such as "system," "processor," "controller," "component," "module," "interface," "model," or "unit" described above may generally refer to computer-related entities, hardware, combinations of hardware and software, software, or running software. For example, the aforementioned components may be, but are not limited to, processes driven by a processor, processors, controllers, control processors, objects, execution threads, programs, and / or computers. For example, both the application running on the controller or processor and the controller or processor may be components. One or more components may reside within a process and / or execution thread, and the components may be located on a single device (e.g., a system, a computing device, etc.) or distributed across two or more devices.

[0128] Meanwhile, another embodiment provides a computer program stored on a computer recording medium that performs the aforementioned method for determining only the anterior-posterior aspect of the spine. Additionally, another embodiment provides a computer-readable recording medium that records a program for realizing the aforementioned capsule endoscope image interpretation method.

[0129] A program recorded on a recording medium can execute the aforementioned steps by being read, installed, and executed on a computer. In this way, in order for a computer to read a program recorded on a recording medium and execute functions implemented by the program, the program may include code encoded in a computer language such as C, C++, JAVA, or machine language, which can be read by the computer's processor (CPU) through the computer's device interface.

[0130] Such code may include functional code related to functions that define the aforementioned functions, and may also include control code related to execution procedures necessary for a computer processor to execute the aforementioned functions according to a predetermined procedure.

[0131] In addition, this code may further include memory reference-related code regarding where (address) in the computer's internal or external memory additional information or media required for the computer's processor to execute the aforementioned functions should be referenced.

[0132] In addition, if the computer processor needs to communicate with any other computer or server located remotely in order to execute the aforementioned functions, the code may further include communication-related code regarding how the computer processor should communicate with any other computer or server located remotely using the computer's communication module, and what information or media should be transmitted or received during communication.

[0133] Computer-readable recording media that record programs as described above include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage device, etc.

[0134] In addition, computer-readable recording media are distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner.

[0135] Furthermore, the functional program for implementing the present invention, and the related code and code segments, etc., may be easily inferred or modified by programmers skilled in the art to which the present invention belongs, taking into account the system environment of a computer that reads a recording medium and executes the program.

[0136] The method for determining the anterior-posterior relationship of the spine described through FIG. 6 may also be implemented in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, as well as removable and inseparable media. Additionally, a computer-readable medium may include all computer storage media. Computer storage media include both volatile and non-volatile, removable and inseparable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0137] The aforementioned method for determining the anterior-posterior orientation of the spine may be executed by an application installed by default on the terminal (which may include a program included in a platform or operating system installed by default on the terminal), or by an application (i.e., a program) directly installed by the user on the master terminal through an application provider server, such as an application store server, an application, or a web server related to the service. In this sense, the aforementioned method for determining the anterior-posterior orientation of the spine may be implemented as an application (i.e., a program) that is installed by default on the terminal or directly installed by the user, and may be recorded on a computer-readable recording medium such as the terminal.

[0138] Specific embodiments of the present invention have been described above. However, those skilled in the art will understand that the spirit and scope of the present invention are not limited to these specific embodiments, and that various modifications and variations are possible within the scope of not altering the essence of the invention.

[0139] Accordingly, the embodiments described above are provided to fully inform those skilled in the art of the scope of the invention and should be understood as illustrative in all respects and not restrictive, and the invention is defined only by the scope of the claims.

Claims

Claim 1 A method in which at least a portion of each step is performed by a processor, comprising: a step of acquiring a side image of a subject's body captured based on an RGB image sensor; a step of inputting the side image into a machine learning model trained to determine key points for the spine for determining anterior-posterior only of the spine, thereby estimating key points for at least some vertebrae of the subject's spine in the side image; a step of determining a plurality of vectors and a plurality of orthogonal lines perpendicular to the vectors based on the key points; and a step of determining anterior-posterior only of the subject's spine based on the plurality of orthogonal lines, wherein the step of estimating the key points estimates a pair of key points located in upper and lower positions relative to the side on a preset vertebra of the subject's spine in the side image, and the step of determining a plurality of orthogonal lines perpendicular to each vector determines a vector connecting the key points located on the same vertebra. A machine learning-based method for determining anterior-posterior spine only, comprising the step of determining an orthogonal line perpendicular to the vector and passing through the midpoint of the key point located on the same vertebra, wherein the machine learning model is trained to estimate a pair of key points located at the top and bottom of the same vertebra corresponding to each vertebra number, wherein the training is performed using a plurality of side images, which are images of the side of a subject wearing clothes and in which the upper and lower positions for each vertebra number are labeled, as training data, and a model dedicated to determining anterior-posterior spine only is created through transfer learning that additionally learns the upper and lower key points for each vertebra number based on a deep neural network that has already trained a plurality of image data. Claim 2 A machine learning-based method for determining only the anterior-posterior aspect of a spine according to claim 1, wherein the step of determining only the anterior-posterior aspect of a subject’s spine comprises: a step of calculating an angle between the plurality of orthogonal lines; and a step of determining only the anterior-posterior aspect of a subject’s spine based on the orthogonal lines and the angle. Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 A machine learning-based method for determining anterior-posterior spinal Claim 7 In claim 6, the step of determining the vector comprises the step of generating a first vector corresponding to the first thoracic vertebra, a second vector corresponding to the twelfth thoracic vertebra, and a third vector corresponding to the first sacral vertebra, and the step of generating the orthogonal line comprises determining a first orthogonal line perpendicular to the first vector, a second orthogonal line perpendicular to the second vector, and a third orthogonal line perpendicular to the third vector, a machine learning-based method for determining anterior-posterior spinal position. Claim 8 In claim 7, the step of determining only the anterior-posterior aspect of the subject’s spine comprises: a step of determining thoracic kyphosis based on a first angle formed by the first orthogonal line and the second orthogonal line; or a step of determining lumbar lordosis based on a second angle formed by the second orthogonal line and the third orthogonal line, a machine learning-based method for determining anterior-posterior aspect of the spine. Claim 9 In claim 1, the machine learning-based method for determining anterior-posterior only of the spine, wherein the machine learning model is trained using a plurality of lateral images labeled with specific key points of the spine as training data. Claim 10 A machine learning-based method for determining anterior-posterior spinal position, comprising, prior to the step of acquiring the side image, a posture estimation step of inputting the side image into a posture estimation model and determining whether the subject's knee is extended or whether the position of the head is above the position of the body based on the output posture point of the subject. Claim 11 In claim 10, the posture estimation step is a machine learning-based method for determining whether the position of the head is higher than the position of the body by comparing the posture point corresponding to the head and the posture point corresponding to the shoulder among the posture points based on the posture point of the subject. Claim 12 A memory unit that stores a machine learning model trained to determine key points for determining anterior-posterior spinal anteroposterior relations in a lateral image of the subject's body; a communication unit that transmits and receives information with an external device; The machine learning-based spine anterior-posterior determination method includes a processor that controls the memory and communication unit, wherein the processor acquires the side image captured based on an RGB image sensor, estimates a pair of key points located at upper and lower positions relative to the side from a preset vertebra of the subject's spine in the side image through a machine learning model, determines a vector connecting the key points located on the same vertebra, determines an orthogonal line perpendicular to the vector and passing through the midpoint of the key points located on the same vertebra, and determines only the anterior-posterior of the subject's spine based on the plurality of orthogonal lines, wherein the machine learning model is trained to estimate a pair of key points located at the upper and lower ends of the same vertebra corresponding to each vertebra number, wherein the training is performed using a plurality of side images as training data, wherein the upper and lower positions for each vertebra number are labeled and the images are taken of the side of a subject wearing clothes, and a model dedicated to determining only the anterior-posterior of the spine is created through transfer learning that additionally learns the upper and lower key points for each vertebra number based on a deep neural network that has already trained on the plurality of image data. Device. Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 In claim 12, the machine learning-based spine anterior-posterior determination device, wherein the machine learning model is a model trained to determine a pair of key points at the 1st thoracic vertebra, the 12th thoracic vertebra, and the 1st sacral vertebra, respectively. Claim 18 In claim 17, the processor determines a first vector corresponding to the first thoracic vertebra, a second vector corresponding to the twelfth thoracic vertebra, and a third vector corresponding to the first sacral vertebra, and determines a first orthogonal line perpendicular to the first vector, a second orthogonal line perpendicular to the second vector, and a third orthogonal line perpendicular to the third vector, a machine learning-based spinal anterior-posterior determination device. Claim 19 In claim 18, the processor determines thoracic kyphosis based on the angle formed by the first orthogonal line and the second orthogonal line, or determines lumbar lordosis based on the angle formed by the second orthogonal line and the third orthogonal line, a machine learning-based spinal anterior-posterior curvature determination device. Claim 20 In claim 12, the machine learning-based spine anterior-posterior determination device, wherein the machine learning model is trained using a plurality of lateral images labeled with specific points of the spine as training data.

Images