Osteoporosis diagnosis system and interpretation method using x-ray images

Abstract

An osteoporosis diagnosis system according to one embodiment comprises: a classification model training device which includes a plurality of classification models, at least one of the plurality of classification models being an artificial intelligence model corresponding to each of a plurality of divided images obtained by dividing an entire image in correspondence to each anatomical part, and which trains a corresponding classification model on the basis of the plurality of divided images and a training label corresponding to each divided image, the training label being labeled as normal or osteoporosis; and a bone disease classification device for diagnosing the presence or absence of osteoporosis by inputting, into the corresponding classification model of the classification model training device, a plurality of divided images divided from an image to be interpreted.

Description

Osteoporosis diagnosis system and interpretation method using X-RAY images

[0001] The present disclosure relates to a diagnosis system and method for osteoporosis using X-ray images, and more particularly, to a diagnosis system and method for osteoporosis using chest X-ray whole images and segmented images.

[0002] South Korea entered an aging society in 2018, with 14% of the population aged 65 and older. It is projected to become a super-aged society by 2025, with the proportion exceeding 20%. Consequently, the number of patients with osteoporosis is expected to surge, and the medical and socioeconomic costs associated with osteoporosis are expected to skyrocket.

[0003] Osteoporosis is a skeletal disease that progresses throughout the body, where bone strength is weakened and the risk of fracture increases. Osteoporosis has no specific symptoms, but if a fracture occurs, the risk of secondary fractures and complications increases, so it is necessary to prevent the risk of osteoporosis through screening tests for reduced bone density. Osteoporosis is diagnosed through a bone density test such as Dual-Energy X-ray Absorptiometry (hereinafter referred to as 'DEXA'), and based on the bone index (T-score) measured after the bone density test, it is classified as normal, osteopenia, or osteoporosis.

[0004] With the recent rapid development of image analysis technology using artificial intelligence models, artificial intelligence models are also being utilized in the diagnosis of osteoporosis. U.S. Patent No. 12,033,318 describes a technique for estimating bone density from X-ray images using an artificial intelligence model. Specifically, according to U.S. Patent No. 12,033,318, a human skeletal image is used as learning data, and bone mineral density (BMD) measured by DEXA is used as supervised data to train an artificial intelligence model. Through the completed training, the artificial intelligence model can directly predict bone density or bone indices (such as T-score) with consistency from a chest X-ray image, and through this, the condition of the bone or the presence or absence of bone disease can be determined.

[0005] The technology of U.S. Patent No. 12,033,318 has the advantage of providing detailed diagnostic information, such as bone condition, through continuous prediction values ​​of bone density or bone indicators. However, in order to train a highly accurate artificial intelligence model, a large amount of medical images and paired DEXA-extracted label information (bone indicator information such as bone density or T-score) from various hospitals (or cohort groups) are required. Typically, medical data deals with individual patient records, making it difficult to access in the private sector, and it is not standardized in a structured form suitable for learning. Therefore, it is not easy to secure a large amount of medical data required for training an artificial intelligence model, and there is a problem that a lot of cost is required to secure sufficient data.

[0006] Meanwhile, X-ray images may contain unnecessary shapes or parts (hereinafter referred to as "artifacts") during the acquisition process due to factors such as the photographer's skill level, the location or movement of the subject, differences in X-ray absorption rates, living implants, and tissues other than bone. However, U.S. Patent No. 12,033,318 does not describe any processing of these artifacts, and thus has the problem of recognizing artifacts as part of bone information.

[0007] Therefore, in order to learn a highly accurate artificial intelligence model from X-ray images, a procedure to detect and remove these artifacts is required.

[0008] The technical problem to be solved by the present invention is to provide an osteoporosis diagnosis system and method that can provide accurate analysis results even when the image is incomplete by inferring bone-related information by synthesizing the output results of models learned from each of a plurality of extracted images using an ensemble algorithm.

[0009] In addition, the technical problem to be solved by the present invention is to provide an osteoporosis diagnosis system and diagnosis method that can quickly secure learning efficiency and accuracy of a learning model by applying transfer learning.

[0010] An osteoporosis diagnosis system according to one embodiment

[0011] A classification model learning device that learns a corresponding classification model based on a plurality of segmented images and a learning label corresponding to each segmented image, wherein the learning label is labeled as normal or osteoporosis; and a bone disease classification device that inputs a plurality of segmented images segmented from an image to be read into a corresponding classification model of the classification model learning device to diagnose whether or not osteoporosis exists.

[0012] The above plurality of classification models include a first classification model corresponding to the entire image, and the classification model learning device can learn the first classification model based on the entire image and the learning label corresponding thereto.

[0013] The above full image may be a chest X-ray image.

[0014] The above classification model learning device may include a first image segmentation unit that segments the entire image into anatomical parts to generate segmented images; a learning dataset generation unit that generates a plurality of learning datasets based on a whole image of a normal person or an osteoporosis patient input from the outside and a plurality of segmented images segmented by the first image segmentation unit, each learning dataset including a learning image that is the entire image or a segmented image and a learning label labeled as normal or osteoporosis; and a plurality of classification model learning units that learn corresponding classification models based on the plurality of learning datasets generated by the learning dataset generation unit.

[0015] The first image segmentation unit crops and segments the entire image into images corresponding to each anatomical part, and the learning image generated by the learning dataset generation unit may include an entire chest image and one or more segmented images of the right clavicle-scapula, the left clavicle-scapula, the cervical spine, and the thoracic / lumbar spine.

[0016] The above bone disease classification device may include a second image segmentation unit that segments the image to be read by anatomical parts to generate a plurality of segmented images; and a bone disease classification inference unit that inputs the entire image to be read and the plurality of segmented images segmented by the second image segmentation unit into a corresponding classification model among the plurality of classification models of the classification model learning device, and infers a bone disease classification result for each input image.

[0017] The above-mentioned bone disease classification device may further include a combining unit that combines a plurality of bone disease classification results inferred by the bone disease classification inference unit using an ensemble algorithm to diagnose whether the image to be read is an osteoporosis image or a normal image.

[0018] According to another embodiment, an osteoporosis diagnosis system

[0019] A classification model learning device that performs primary learning for a plurality of source classification models based on first learning images including segmented images segmented from full images of a normal person and an osteoporosis patient, and transfers the primary learning result and performs secondary learning for a plurality of target classification models based on second learning images including segmented images segmented from full images of a normal person, an osteopenia patient, and an osteoporosis patient; and a bone disease classification device that inputs a plurality of segmented images segmented from an image to be read into the corresponding target classification models to diagnose whether or not osteoporosis exists.

[0020] The above plurality of source classification models and the above plurality of target classification models may each include a first source classification model and a first target classification model corresponding to the entire image.

[0021] The above full image may be a chest X-ray image.

[0022] The classification model learning device comprises: a first image segmentation unit that receives a source learning image which is an entire image of the normal person or the osteoporosis patient and segments the source learning image into images corresponding to each anatomical region, and receives a target learning image which is an entire image of the normal person, the osteopenia patient, or the osteoporosis patient, and segments the target learning image into images corresponding to each anatomical region; a learning dataset generation unit that generates a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image which is the source learning image or an image segmented from the source learning image and a source learning label labeled as normal or osteoporosis; and a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image which is the target learning image or an image segmented from the target image and a target learning label labeled as non-osteoporosis or osteoporosis; It may include a plurality of source classification model learning units that perform the first learning for the corresponding source classification model based on the plurality of source learning datasets; and a plurality of target classification model learning units that perform the second learning for the corresponding target classification model based on the plurality of target learning datasets.

[0023] The classification model learning device comprises: a first image segmentation unit that receives a source learning image which is an entire image of the normal person or the osteoporosis patient and segments the source learning image into images corresponding to each anatomical region, and receives a target learning image which is an entire image of the normal person, the osteopenia patient, or the osteoporosis patient, and segments the target learning image into images corresponding to each anatomical region; a learning dataset generation unit that generates a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image which is the source learning image or an image segmented from the source learning image and a source learning label labeled as normal or osteoporosis; and a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image which is the target learning image or an image segmented from the target image and a target learning label labeled as normal, osteopenia, or osteoporosis; It may include a plurality of source classification model learning units that perform the first learning for the corresponding source classification model based on the plurality of source learning datasets; and a plurality of target classification model learning units that perform the second learning for the corresponding target classification model based on the plurality of target learning datasets.

[0024] The above classification model learning device may further include a transfer learning unit that performs transfer learning for a corresponding target classification model learning unit based on hidden layer parameters, which are learning results of the source classification model learning unit.

[0025] The first image segmentation unit crops and segments the entire image into images corresponding to each anatomical part, and the learning image generated by the learning dataset generation unit may include an entire chest image and one or more segmented images of the right clavicle-scapula, the left clavicle-scapula, the cervical spine, and the thoracic / lumbar spine.

[0026] The above bone disease classification device may include a second image segmentation unit that segments the image to be read by anatomical parts to generate a plurality of segmented images; and a bone disease classification inference unit that inputs the entire image to be read and the plurality of segmented images segmented by the second image segmentation unit into a corresponding classification model among the plurality of target classification models, and infers a bone disease classification result for each input image.

[0027] The above bone disease classification device may further include a combining unit that combines a plurality of bone disease classification results inferred by the bone disease classification inference unit using an ensemble algorithm to diagnose whether the target image for reading is an osteoporosis image.

[0028] The above bone disease classification device can diagnose the above target image as an osteoporosis image or a non-osteoporosis image.

[0029] The above bone disease classification device can diagnose the above target image as an osteoporosis image, an osteopenia image, or a normal image.

[0030] According to one embodiment, a bone disease classification device is a bone disease classification device that diagnoses whether or not osteoporosis is present by connecting to a classification model learning device including a plurality of classification models each corresponding to a whole image or a plurality of segmented images obtained by segmenting the whole image corresponding to each anatomical part.

[0031] The above bone disease classification device can diagnose whether the target image for reading has osteoporosis by inputting the target image for reading or a plurality of segmented images segmented from the target image for reading into a corresponding classification model among the plurality of classification models.

[0032] The above bone disease classification device may include an image segmentation unit that segments the image to be read by anatomical parts to generate a plurality of segmented images; and a bone disease classification inference unit that inputs the entire image to be read and the plurality of segmented images segmented by the image segmentation unit into a corresponding classification model among the plurality of classification models, and infers a bone disease classification result for each input image.

[0033] The above bone disease classification device may further include a combining unit that combines a plurality of bone disease classification results inferred by the bone disease classification inference unit using an ensemble algorithm to diagnose whether the target image for reading is an osteoporosis image.

[0034] The above bone disease classification device can diagnose the above target image as an osteoporosis image or a non-osteoporosis image.

[0035] The above bone disease classification device can diagnose the above target image as an osteoporosis image, an osteopenia image, or a normal image.

[0036] The above multiple classification learning models may be classification learning models that are transferred from the learning results of a first classification model that is first learned with learning data of normal people and osteoporosis patients, and are secondarily learned with learning data of normal people, osteopenia, and osteoporosis patients.

[0037] According to one embodiment, a classification model learning device can perform primary learning for a plurality of source classification models based on first learning images including segmented images segmented from full images of a normal person and an osteoporosis patient, and transfer the first learning result and perform secondary learning for a plurality of target classification models based on second learning images including segmented images segmented from full images of a normal person, an osteopenia patient, and an osteoporosis patient.

[0038] The above plurality of source classification models and the above plurality of target classification models may each include a first source classification model and a first target classification model corresponding to the entire image.

[0039] The above classification model learning device may further include a transfer learning unit that performs transfer learning for a corresponding target classification model learning unit based on hidden layer parameters, which are learning results of the source classification model learning unit.

[0040] According to one embodiment, a method for diagnosing osteoporosis is a method for diagnosing osteoporosis from an image to be read,

[0041] The method comprises the steps of: dividing the above-mentioned target image into a plurality of images corresponding to each anatomical part; inputting the above-mentioned target image and the divided plurality of divided images into a corresponding classification model among a plurality of classification models, at least one of which is an artificial intelligence model corresponding to each of the plurality of divided images obtained by dividing the entire image into each anatomical part; and inferring a bone disease classification result for each input image; and combining the plurality of inferred bone disease classification results using an ensemble algorithm to diagnose whether the above-mentioned target image has osteoporosis.

[0042] The method may further include a step of learning the plurality of classification models based on a whole image of a normal person or a patient with osteoporosis, wherein the classification model learning step may include a step of generating a segmented image by dividing the whole image by anatomical region; a step of generating a plurality of learning data sets based on the whole learning image and the plurality of segmented segmented images, each learning data set including a learning image that is a whole image or a segmented image and a learning label labeled as normal or osteoporosis; and a step of learning a corresponding classification model based on each of the plurality of learning data sets.

[0043] The above full image is a chest X-ray image, and the above full image can be cropped into images corresponding to each anatomical part to create multiple segmented images.

[0044] It further includes a step of learning the plurality of classification models based on the entire images of a normal person, a patient with osteoporosis, or a patient with osteoporosis, and the plurality of classification learning models may be classification learning models that are transferred from the learning results of the plurality of source classification models that are first learned with the first learning images of the normal person and the patient with osteoporosis, and are secondarily learned with the second learning images of the normal person, the patient with osteopenia, or the patient with osteoporosis.

[0045] A step of segmenting a source learning image, which is an entire image of a normal person or an osteoporosis patient, into images corresponding to each anatomical region; A step of generating a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis; A step of performing the first learning on the corresponding source classification model based on the plurality of source learning datasets; A step of performing transfer learning on the corresponding target classification model based on hidden layer parameters, which are learning results of the source classification model; A step of segmenting the target learning image, which is an entire image of a normal person, a non-osteoporosis patient, or an osteoporosis patient, into images corresponding to each anatomical region; The method may include: generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as osteoporosis or non-osteoporosis; and performing the second learning on the corresponding target classification model based on the plurality of target learning datasets.

[0046] A step of segmenting a source learning image, which is an entire image of a normal person or an osteoporosis patient, into images corresponding to each anatomical region; A step of generating a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis; A step of performing the first learning on the corresponding source classification model based on the plurality of source learning datasets; A step of performing transfer learning on the corresponding target classification model based on hidden layer parameters, which are learning results of the source classification model; A step of segmenting the target learning image, which is an entire image of a normal person, a non-osteoporosis patient, or an osteoporosis patient, into images corresponding to each anatomical region; The method may include: generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as normal, osteopenia, or osteoporosis; and performing the second learning on the corresponding target classification model based on the plurality of target learning datasets.

[0047] A classification model learning method according to one embodiment is a method of learning multiple classification models based on the entire images of a normal person, a patient with osteopenia or osteoporosis,

[0048] A method for classifying a plurality of source classification models, comprising: performing primary learning on the basis of first learning images including segmented images segmented from full images of a normal person and an osteoporosis patient; and transferring the primary learning results and performing secondary learning on the basis of second learning images including segmented images segmented from full images of a normal person, an osteopenia patient, and an osteoporosis patient.

[0049] The step of performing the first learning may include the step of segmenting a source learning image, which is an entire image of the normal person or the osteoporosis patient, into images corresponding to each anatomical part; the step of generating a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis; and the step of performing the first learning on the corresponding source classification model based on the plurality of source learning datasets.

[0050] The step of performing the secondary learning may include the step of performing transfer learning on a corresponding target classification model based on a hidden layer parameter which is a learning result of the source classification model; the step of segmenting a target learning image which is an entire image of a normal person, a non-osteoporosis patient, or the osteoporosis patient into images corresponding to each anatomical part; the step of generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image which is the target learning image or an image segmented from the target image, and a target learning label labeled as non-osteoporosis or osteoporosis; and the step of performing the secondary learning on the corresponding target classification model based on the plurality of target learning datasets.

[0051] The step of performing the secondary learning may include: performing transfer learning on a corresponding target classification model based on a hidden layer parameter which is a learning result of the source classification model; segmenting a target learning image which is an entire image of a normal person, a non-osteoporosis patient, or an osteoporosis patient into images corresponding to each anatomical region; generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image which is the target learning image or an image segmented from the target image, and a target learning label labeled as normal, osteopenia, or osteoporosis; and performing the secondary learning on the corresponding target classification model based on the plurality of target learning datasets.

[0052] A computer according to one embodiment comprises at least one processor configured to execute computer-readable instructions, wherein by the at least one processor,

[0053] A target image for interpretation is divided into multiple images corresponding to each anatomical part, and the target image for interpretation and the multiple divided images are input into a corresponding classification model among multiple classification models, at least one of which is an artificial intelligence model corresponding to each of the multiple divided images obtained by dividing the entire image into each anatomical part, so as to infer a bone disease classification result for each input image, and the multiple inferred bone disease classification results are combined using an ensemble algorithm to diagnose whether the target image for interpretation has osteoporosis.

[0054] According to the embodiments, the output results of models learned from multiple extracted images are synthesized by an ensemble algorithm to infer goal-related information, thereby increasing prediction accuracy and providing accurate analysis results even when the image is incomplete.

[0055] According to the examples, there is an effect of quickly securing learning efficiency and accuracy of a learning model by applying transfer learning.

[0056] FIG. 1 is a diagram showing the configuration of a medical imaging system according to an embodiment of the present invention.

[0057] FIG. 2 is a drawing showing a computing device that implements an osteoporosis diagnosis system according to an embodiment of the present invention.

[0058] FIG. 3 is a drawing showing an osteoporosis diagnosis system according to the first embodiment of the present invention.

[0059] FIG. 4 is a diagram showing a classification model learning device according to the first embodiment of the present invention.

[0060] FIG. 5 is a drawing showing an example of a segmented CXR image according to an embodiment of the present invention.

[0061] FIG. 6 is a diagram showing an example of generating multiple learning datasets according to the first embodiment of the present invention.

[0062] Figure 7 is a diagram showing a deep neural network model according to an embodiment of the present invention.

[0063] Figure 8 is a diagram showing a classification model learning method according to the first embodiment of the present invention.

[0064] FIG. 9 is a drawing showing a bone disease classification device according to the first embodiment of the present invention.

[0065] Fig. 10 is a drawing showing a bone disease classification method according to the first embodiment of the present invention.

[0066] FIG. 11 is a drawing showing an osteoporosis diagnosis system according to a second embodiment of the present invention.

[0067] Fig. 12 is a diagram showing a classification model learning device according to a second embodiment of the present invention.

[0068] Figure 13 is a diagram showing an example of transfer learning according to the second embodiment of the present invention.

[0069] Figure 14 is a diagram showing a classification model learning method according to the second embodiment of the present invention.

[0070] Fig. 15 is a drawing showing a bone disease classification device according to a second embodiment of the present invention.

[0071] Fig. 16 is a drawing showing a bone disease classification method according to a second embodiment of the present invention.

[0072] Figure 17 is a drawing showing a bonding process performed in a bonding part according to the second embodiment of the present invention.

[0073] Figure 18 is a diagram showing a classification model learning method according to a third embodiment of the present invention.

[0074] Fig. 19 is a drawing showing a bone disease classification method according to a third embodiment of the present invention.

[0075] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement the invention. However, the present invention can be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted for clarity of description, and similar parts are designated with similar reference numerals throughout the specification.

[0076] Throughout the specification, whenever a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0077] Additionally, terms such as “……part”, “……unit”, and “……module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0078] In this specification, “transmitting or providing” may include not only direct transmission or providing, but also indirect transmission or providing via another device or by using a bypass route.

[0079] In this specification, expressions described in the singular may be interpreted as singular or plural, unless explicit expressions such as “one” or “single” are used.

[0080] In describing embodiments of the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description is omitted.

[0081] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

[0082] FIG. 1 is a diagram showing the configuration of a medical imaging system according to an embodiment of the present invention.

[0083] Referring to FIG. 1, a medical imaging system (1) may include at least one user terminal (30), a medical image storage device (20), and an osteoporosis diagnosis system (10).

[0084] The user terminal (30) is comprised of hardware and software that installs programs executed by a processor and provides a computing environment and network environment for performing the operations of the present invention. The user terminal (30) can be implemented in various types, such as, for example, a computing device within a workstation, a mobile device, etc.

[0085] The user terminal (30) can display medical image-related data stored in the medical image storage device (20) by linking with the medical image storage device (20). In addition, the user terminal (30) can display the results of bone disease diagnosed by the osteoporosis diagnosis system (10). Such a function can be provided through a viewer, which is a dedicated program installed in the user terminal (30). The viewer can be installed and executed, for example, in a computing device within a workstation, and is implemented to connect to the medical image storage device (20) and the osteoporosis diagnosis system (10), and can display medical image-related data stored in the medical image storage device (20) or bone disease information (e.g., osteoporosis) diagnosed by the osteoporosis diagnosis system (10).

[0086] The medical image storage device (20) can store and manage captured medical images. In addition, the medical image storage device (20) can store and manage analysis results for the medical images. The medical image storage device (20) may include a database (PACS database) of a Picture Archiving and Communications System (PACS; hereinafter also referred to as 'PACS'). The medical image storage device (20) may store data according to a designated data format. For example, the medical image storage device (20) may store medical images captured by medical imaging devices and analysis results of the medical images according to the DICOM (Digital Imaging and Communications in Medicine) standard, and may communicate with a user terminal (30) to provide data for image interpretation. In the embodiment of the present invention, the DICOM standard is described as an example of a standard used for storing medical images, but the present invention is not limited thereto and may be provided in various other formats.

[0087] The medical image storage device (20) can obtain bone disease diagnosis information (e.g., osteoporosis / non-osteoporosis) from the osteoporosis diagnosis system (10). The medical images stored in the medical image storage device (20) may be X-ray images, magnetic resonance imaging (MRI) images, ultrasound images, computed tomography (CT) images, digital mammography (MMG) images, digital breast tomosynthesis (DBT) images, etc.

[0088] In the embodiment of the present invention, a chest X-ray image (hereinafter also simply referred to as an 'X-ray image') is described as an example of a medical image, but the present invention is not limited thereto, and the present invention can be applied to any type of medical image.

[0089] The osteoporosis diagnosis system (10) uses multiple artificial intelligence (AI) models to analyze chest X-ray (hereinafter referred to as 'CXR') images and diagnoses whether there is a bone disease (e.g., whether there is osteoporosis) based on the CXR analysis results.

[0090] The osteoporosis diagnosis system (10) according to an embodiment of the present invention may be equipped with a plurality of artificial intelligence models (hereinafter also referred to as 'learning models' or 'classification models') each specialized for a full CXR image and a plurality of segmented CXR images, and after segmenting an input full CXR image into a plurality of segmented CXR images by reflecting anatomical features, the full image and each segmented image may be input into a corresponding artificial intelligence model to determine whether there is a bone disease (whether there is osteoporosis). The artificial intelligence model (AI model) according to an embodiment of the present invention is generated to make medical inferences from input medical images, and the model structure, training data configuration, training method, and medical inference target may be designed in various ways.

[0091] The osteoporosis diagnosis system (10) or a detailed device within the osteoporosis diagnosis system according to an embodiment of the present invention can be implemented as a computing device as shown in FIG. 2.

[0092] Referring to FIG. 2, the osteoporosis diagnosis system or a detailed device (computing device, 10) within the osteoporosis diagnosis system may include one or more processors (11), a memory (13) for loading a program executed by the processor (11), a storage (15) for storing the program and various data, a communication interface (17), and a bus (19) connecting them. In addition, the osteoporosis diagnosis system (10) may further include various components.

[0093] The program may include instructions that cause the processor (11) to perform methods / operations according to various embodiments of the present invention when loaded into the memory (13). That is, the processor (11) may perform methods / operations according to various embodiments of the present invention by executing the instructions. The program is composed of a series of computer-readable instructions grouped based on function, and refers to something that is executed by the processor.

[0094] The processor (11) controls the overall operation of each component of the computing device (10). The processor (11) may be configured as a single physical entity, but may also be configured as multiple entities. The processor (11) configured as multiple entities may divide and process a single execution element or divide and process multiple execution elements.

[0095] The processor (11) may be configured to include at least one of a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), or any other type of processor well known in the art of the present invention. In addition, the processor (11) may perform operations for at least one application or program for executing methods / operations according to various embodiments of the present invention.

[0096] Memory (13) stores various data, commands, and / or information. Memory (13) can load one or more programs from storage (15) to execute methods / operations according to various embodiments of the present invention. Memory (13) may be implemented as volatile memory such as RAM, but the technical scope of the present invention is not limited thereto.

[0097] Storage (15) can store a program non-temporarily. Storage (15) can be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any type of computer-readable recording medium well known in the art to which the present invention pertains.

[0098] According to an embodiment of the present invention, a plurality of classification models and learned parameters of each classification model may be stored in the storage. In this case, each classification model may be a plurality of artificial intelligence models (learning models) specialized for an entire X-ray image or a plurality of segmented X-ray images, and may be an artificial intelligence model having a neural network structure that learns the relationship between an input image and a label.

[0099] The communication interface (17) supports wired and wireless Internet communication of the computing device (10). Furthermore, the communication interface (17) may support various communication methods other than Internet communication. To this end, the communication interface (17) may be configured to include a communication module well known in the technical field of the present invention.

[0100] The bus (19) provides a communication function between components of the computing device (10). The bus (19) can be implemented as various types of buses such as an address bus, a data bus, and a control bus.

[0101] A computer program includes instructions executed by a processor (11) and is stored in a non-transitory computer-readable storage medium, the instructions causing the processor (11) to perform operations according to an embodiment of the present invention. The computer program may be downloaded through a network or sold in product form.

[0102] A computer program according to an embodiment of the present invention may include instructions for collecting a dataset for pre-training an artificial intelligence model and training the artificial intelligence model using the collected dataset. The computer program may also include instructions for preprocessing input data and training the artificial intelligence model using the preprocessed input data.

[0103] The functional blocks (indicated as “...parts”) of the detailed devices of the osteoporosis diagnosis system (10) described below can be implemented through cooperation of programs, etc. included in the processor (11) and memory (13) (or storage (15)), which are the detailed components of the computing device illustrated in FIG. 2.

[0104] Hereinafter, with reference to FIGS. 3 to 10, an osteoporosis diagnosis system (10) according to the first embodiment of the present invention will be described.

[0105] Referring to FIG. 3, the osteoporosis diagnosis system (10) according to the first embodiment of the present invention may include a classification model learning device (100) and a bone disease classification device (200).

[0106] The classification model learning device (100) includes a plurality of classification models. Each classification model may be an artificial intelligence model corresponding to a whole CXR image (hereinafter also simply referred to as a “whole image”) or a plurality of segmented CXR images (hereinafter also simply referred to as “segmented images”) in which the whole CXR image is segmented according to each anatomical region (e.g., whole chest, left clavicle-scapula, right clavicle-scapula, cervical spine, thoracic / lumbar spine).

[0107] According to the first embodiment of the present invention, the classification model learning device (100) receives the entire CXR image (entire image) of a normal person or an osteoporosis patient as training images. That is, the classification model learning device (100) receives the entire image of a normal person or an osteoporosis patient and the corresponding training label (osteoporosis / normal) as training data.

[0108] As described below, the classification model learning device (100) divides the entire input image into multiple segmented images corresponding to anatomical parts, creates a learning dataset based on the entire image or segmented images and the input learning labels (osteoporosis / normal), and then learns the corresponding classification model using each learning dataset.

[0109] The classification model (artificial intelligence model) used in the embodiment of the present invention may include a deep neural network structure. The deep neural network may have a structure including multiple hidden layers between the input layer and the output layer. Deep neural network algorithms that may be used include a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBM), a Q network, a long short-term memory (LSTM), a gated recurrent unit (GRU), a transformer, a capsule network, etc., and the present invention is not limited thereto, and other artificial intelligence models may be used.

[0110] According to an embodiment of the present invention, the classification model learning device (100) uses multiple classification models having the same neural network structure. However, the present invention is not limited to this, and classification models having different neural network structures can be used depending on the characteristics of the entire image or segmented images.

[0111] The bone disease classification device (200) receives a CXR image to be read (hereinafter also referred to as a 'target image') and inputs the received target image into a plurality of classification models that have completed learning performed by the classification model learning device (100) to predict whether there is a bone disease (whether there is osteoporosis). Specifically, as described below, the bone disease classification device (200) divides the input target image into a plurality of segmented images corresponding to anatomical parts, inputs each of the entire image or the segmented images into a classification model that has completed learning, and infers a bone disease classification for each of the plurality of models. Then, the bone disease classification device (200) diagnoses whether there is a final bone disease using an ensemble algorithm for the plurality of inferred bone disease classification results.

[0112] Referring to FIG. 4, the classification model learning device (100) according to the first embodiment of the present invention may include an image segmentation unit (110), a learning data set generation unit (120), and multiple classification model learning units (130).

[0113] The image segmentation unit (110) receives the entire CXR image of a normal or osteoporotic patient as a learning target and the learning label (normal / osteoporosis) information related thereto, and segments the entire CXR image into images corresponding to each anatomical part.

[0114] FIG. 5 is a drawing showing an example of a CXR image segmented by an image segmentation unit (110) and an entire CXR image according to an embodiment of the present invention.

[0115] Referring to FIG. 5, the image segmentation unit (110) according to an embodiment of the present invention generates segmented CXR images by cropping the entire CXR image by anatomical region. Specifically, the image segmentation unit (110) can generate segmented images corresponding to the entire chest image (E1), the right clavicle-scapula (E2), the left clavicle-scapula (E3), the cervical spine (E4), and the thoracic / lumbar spine (E5) by cropping the entire CXR image by anatomical region. In this way, according to an embodiment of the present invention, since the original CXR image is divided into equal parts and each segmented image including each anatomical region is used as a learning image, there is an advantage of saving preprocessing time and computing resources of data used in the classification model, without using a separate artificial intelligence model that detects bone regions in the entire CXR image or performs segmentation.

[0116] However, the present invention is not limited to a method of cropping by dividing the entire image into images corresponding to each anatomical part, and various image segmentation techniques, such as use of a segmentation model, can be used in the present invention.

[0117] In addition, in the embodiment of the present invention, the entire chest image (E1), right clavicle-scapula (E2), left clavicle-scapula (E3), cervical spine (E4), and thoracic / lumbar spine (E5) were used as learning images, but the present invention is not limited thereto, and the entire CXR image and images corresponding to some anatomical regions (e.g., right clavicle-scapula image and left clavicle-scapula image) may be used, or only segmented images may be used instead of the entire image.

[0118] The learning dataset generation unit (120) generates multiple learning datasets based on the entire CXR image input from the outside, the learning label (normal / osteoporosis) related thereto, and multiple segmented images segmented by the image segmentation unit (110).

[0119] FIG. 6 is a diagram showing an example of generating multiple learning datasets according to the first embodiment of the present invention.

[0120] Referring to FIG. 6, the learning data set generation unit (120) generates a plurality of learning data sets (1-1 learning data set, 1-2 learning data set, 1-3 learning data set, …, 1-n learning data set) based on the first X-Ray learning image and the first learning label, which are input first learning data, and a plurality of segmented images segmented by the image segmentation unit (110). Specifically, the learning dataset generation unit (120) generates a 1-1 learning dataset based on the input 1st X-Ray learning image (1-1 entire learning image) and the corresponding first learning label, generates a 1-2 learning dataset based on the 1-2 segmented learning image (e.g., right clavicle-scapula image) segmented by the image segmentation unit (110) and the first learning label (label corresponding to the entire image), and generates a 1-3 learning dataset based on the 1-3 segmented learning image (e.g., left clavicle-scapula image) segmented by the image segmentation unit (110) and the first learning label. In the same manner, a 1-n learning dataset can be generated based on the 1-n segmented learning image segmented by the image segmentation unit (110) and the first learning label.

[0121] Referring to FIG. 6, it can be seen that all learning labels included in the multiple learning data sets generated by the learning data set generation unit (120) are generated identically to the learning label (first learning label) corresponding to the input CXR image (first X-Ray learning image).

[0122] The multiple learning datasets generated by the learning dataset generation unit (120) are used as learning data for the corresponding classification model. For example, the 1-1 learning dataset, the 1-2 learning dataset, and the 1-3 learning dataset are used as learning data for the 1st classification model, the 2nd classification model, and the 3rd classification model, respectively.

[0123] The classification model learning unit (130) may include a plurality of classification model learning units (130-1, 130-2, 130-3, ..., 103_n) that learn each of the plurality of learning datasets. For example, the first classification model learning unit (130_1) learns the first classification model (artificial intelligence model) using the 1-1 entire learning image and the first learning label, which are the 1-1 datasets generated by the learning dataset generation unit (120). The second classification model learning unit (130_2) learns the second classification model (artificial intelligence model) using the 1-2 segmented learning image and the first learning label, which are the 1-2 datasets generated by the learning dataset generation unit (120). Likewise, the n-th classification model learning unit (130_n) uses the 1-n-th segmented learning image and the first learning label, which are the 1-n-th datasets generated by the learning dataset generation unit (120), to train the n-th classification model (artificial intelligence model).

[0124] Figure 7 is an exemplary diagram illustrating a deep neural network model (50) used in a classification model according to an embodiment of the present invention. The neural network model (50) is an example of a machine learning model, and in machine learning technology and cognitive science, it is a statistical learning algorithm implemented based on the structure of a biological neural network or a structure that executes the algorithm.

[0125] According to one embodiment, a deep neural network model (50) may represent a machine learning model having problem-solving capabilities by learning that nodes, which are artificial neurons that form a network by combining synapses like in a biological neural network, repeatedly adjust the weights of synapses so that the error between the correct output corresponding to a specific input and the inferred output is reduced. For example, the neural network model (50) may include any probability model, neural network model, etc. used in artificial intelligence learning methods such as machine learning and deep learning.

[0126] The deep neural network model (50) is implemented as a multilayer perceptron (MLP) composed of multiple layers of nodes and connections between them. The deep neural network model (50) according to one embodiment can be implemented using one of various artificial neural network model structures including MLP.

[0127] As illustrated in FIG. 7, the neural network model (50) is composed of an input layer (51) that receives an input signal or data (40) from the outside, an output layer (53) that outputs an output signal or data (60) corresponding to the input data, and n hidden layers (52_1 to 52_n) located between the input layer (51) and the output layer (53) that receive signals from the input layer (51), extract characteristics, and transmit them to the output layer (53) (where n is a positive integer). Here, the output layer (53) receives signals from the hidden layers (52_1 to 52_n) and outputs them to the outside.

[0128] In this way, a plurality of input variables and corresponding plurality of output variables are respectively matched in the input layer (51) and the output layer (53) of the neural network model (50), and the synaptic values ​​(weights) between the nodes included in the input layer (51), the hidden layers (52_1 to 52_n), and the output layer (53) are adjusted, so that learning can be performed so that the correct output corresponding to a specific input can be extracted. Through this learning process, the characteristics hidden in the input variables of the neural network model (50) can be identified, and the synaptic values ​​(or weights) between the nodes of the neural network model (50) can be adjusted so that the error between the output variables calculated based on the input variables and the target output is reduced.

[0129] According to an embodiment of the present invention, each classification model learning unit (130) can learn a corresponding classification model by inputting an input learning image (whole learning image or segmented image) corresponding to the input layer (51) and inputting a learning label (normal / osteoporosis) through the output layer (53). That is, each classification model learning unit (130) can perform learning based on the input learning image (whole learning image or segmented image) and the learning label (normal / osteoporosis) and adjust weights indicating the connection strength between a specific node of the current layer and a node of the previous layer.

[0130] The following describes a classification model learning method according to a first embodiment of the present invention with reference to FIG. 8. FIG. 8 is a diagram illustrating a classification model learning method according to a first embodiment of the present invention.

[0131] The classification model learning device (100) receives the entire CXR image (entire image) of a normal person or an osteoporosis patient as training images. That is, the classification model learning device (100) receives the entire image of a normal person or an osteoporosis patient and the corresponding training label (osteoporosis / normal) as training data. (S10)

[0132] Thereafter, the classification model learning device (100) divides the entire input image into a plurality of segmented images corresponding to anatomical regions. (S20) According to an embodiment of the present invention, the image segmentation unit (110) crops the entire input CXR image by anatomical region to generate segmented CXR images. Specifically, the image segmentation unit (110) can crop the entire CXR image by anatomical region to generate segmented images corresponding to the entire chest image (E1), right clavicle-scapula (E2), left clavicle-scapula (E3), cervical spine (E4), and thoracic / lumbar spine (E5).

[0133] The learning dataset generation unit (120) generates multiple learning datasets based on the entire image or segmented images and the learning labels (osteoporosis / normal) input in relation to the entire image. (S30)

[0134] The classification model learning unit (130) uses the multiple learning datasets generated by the learning dataset generation unit (120) as learning data for the corresponding classification model, and learns the corresponding classification model. (S40)

[0135] The classification model learning unit (130) stores learning parameters (weights, biases, etc.), which are learning results of a classification model based on a learning dataset, in memory or storage. (S50)

[0136] The following describes a bone disease classification device and a bone disease classification method according to the first embodiment of the present invention with reference to FIGS. 9 and 10.

[0137] Referring to FIG. 9, a bone disease classification device (200) according to the first embodiment of the present invention may include an image segmentation unit (210), a bone disease classification inference unit (220), and a combination unit (230).

[0138] The image segmentation unit (210) receives a CXR image to be read (hereinafter also referred to as a 'target image') and (S110) segments the entire target image into images corresponding to each anatomical part. (S120) At this time, the image segmentation unit (210) of the bone disease classification device (200) segments the entire image into images corresponding to each anatomical part using the same segmentation method as the image segmentation unit (210) of the classification model learning device (100). According to an embodiment of the present invention, the image segmentation unit (210) applies a simple method of cropping the entire CXR image by anatomical part to generate a segmented CXR image, but the present invention is not limited thereto, and segmentation may be performed using a separate artificial intelligence model that performs segmentation.

[0139] The bone disease classification inference unit (220) inputs the entire target image and the plurality of segmented images segmented by the image segmentation unit (210) into a corresponding classification model among the plurality of classification models for which learning has been completed performed by the classification model learning device (100), and infers a bone disease classification result for each image (entire image or segmented image). (S130) At this time, the classification result inferred by the bone disease classification inference unit (220) may be a logit or probability value.

[0140] The combining unit (230) combines multiple bone disease classification results inferred by the bone disease classification inference unit (220) using an ensemble algorithm to diagnose whether there is a final bone disease (normal or osteoporosis). (S140) At this time, the ensemble algorithm according to an embodiment of the present invention can combine (synthesize) multiple bone disease classification results using an average technique. For example, the combining unit (230) can average logit or probability values, which are bone disease classification result values ​​inferred from each classification model, and compare the average value with a reference value to determine whether there is a bone disease (normal / osteoporosis). At this time, according to an embodiment of the present invention, when combining multiple bone disease classification results, the combining unit (230) can determine whether there is a bone disease (normal / osteoporosis) by excluding one or more results and calculating the average value only for the remaining results. For example, the presence or absence of a bone disease (normal / osteoporosis) can be determined by excluding the largest or smallest logit or probability values ​​among the bone disease classification results inferred from each classification model and taking the average only for the remaining classification results.

[0141] Meanwhile, according to embodiments of the present invention, various techniques other than averaging can be applied as ensemble techniques. For example, any one or a combination of voting, bagging, boosting, and stacking can be used as ensemble techniques.

[0142] According to the first embodiment of the present invention described above, the classification model learning device (100) uses osteoporosis or normal CXR images as learning input images, and the bone disease classification device (200) diagnoses the target image by classifying it as osteoporosis or normal as the final bone disease classification result for the target image. In this way, since the first embodiment of the present invention performs learning using only osteoporosis and normal images (i.e., images with a clear difference), learning can be performed relatively easily even with a small amount of learning data.

[0143] However, bone disease classification may include osteopenia in addition to osteoporosis and normal, as shown in Table 1 below. This bone disease classification can be categorized based on the T-score measured from DEXA.

[0144] Bone Disease Classification T-Score: Normal - 1.0 or higher, Osteopenia - 2.5 to -1.0, Osteoporosis - Less than 2.5

[0145] In order to train a classification model (artificial intelligence model) using data (image data) of osteopenia patients as learning data in addition to data of normal and osteoporotic patients, a large amount of learning data is required, but it is not easy to secure a large amount of medical data required for training an artificial intelligence model, and there is a problem that a lot of cost is required to secure sufficient data. Accordingly, in the second embodiment of the present invention, an osteoporosis diagnosis system is disclosed, which performs first learning for a first classification model (hereinafter also referred to as a 'source classification model') using learning data of normal people and osteoporosis patients, which are relatively easy to learn, and transfers the results of the first learning and performs second learning for a second classification model (hereinafter also referred to as a 'target classification model') using learning data of normal people, osteopenia, and osteoporosis patients, which are difficult to learn.

[0146] Hereinafter, with reference to FIGS. 11 to 17, an osteoporosis diagnosis system (10) according to a second embodiment of the present invention will be described. In the following, descriptions of overlapping parts with the first embodiment of the present invention will be omitted, and differences will be primarily described.

[0147] Referring to FIG. 11, the osteoporosis diagnosis system (10) according to the second embodiment of the present invention may include a classification model learning device (300) and a bone disease classification device (200).

[0148] The classification model learning device (300) may include a source classification model group (303) composed of a plurality of source classification models (artificial intelligence models) and a target classification model group (304) composed of a plurality of target classification models.

[0149] Each source classification model and each target classification model may be an artificial intelligence model corresponding to the entire CXR image or multiple segmented CXR images that are segmented from the entire CXR image corresponding to each anatomical region.

[0150] According to the second embodiment of the present invention, the classification model learning device (300) receives the entire CXR image (entire image) of a normal person, osteopenia, or osteoporosis patient as a learning image.

[0151] As described below, the classification model learning device (300) trains a source classification model group (303) based on learning images and learning labels (normal / osteoporosis) of normal people and osteoporosis patients, which are relatively easy to learn, and transfers the hidden layer parameters, which are the learning results of the source classification model group (303), to a target classification model group (304), and then trains a target classification model group (304) based on learning images and learning labels (non-osteoporosis / osteoporosis) of normal people, osteopenia, and osteoporosis patients, which are difficult to learn.

[0152] In this way, according to the second embodiment of the present invention, the classification model learning device (300) receives the entire CXR image (entire image) of a normal person or an osteoporosis patient and the corresponding learning label (normal / osteoporosis) as source learning data for training the source classification model group (303). In addition, the classification model learning device (300) receives the entire CXR image (entire image) of a normal person, an osteopenia patient, or an osteoporosis patient and the corresponding learning label (non-osteoporosis / osteoporosis) as target learning data for training the target classification model group (304). That is, the classification model learning device (300) according to the second embodiment of the present invention receives learning labels (non-osteoporosis / osteoporosis) divided into two categories for the entire CXR image of a normal person, an osteopenia patient, or an osteoporosis patient as target learning labels for training the target classification model group (304). Here, the target learning label of “osteoporosis” is a learning label corresponding to normal and osteopenia images as a non-osteoporosis classification, and the target learning label of “osteoporosis” is a learning label corresponding to osteoporosis images.

[0153] According to the second embodiment of the present invention, since the target classification model group (304) is trained through target learning labels (osteoporosis / osteoporosis) divided into two categories, there is an advantage in that the target classification model group (304) can be sufficiently trained even with a relatively small number of images of normal people, osteopenia patients, and osteoporosis patients.

[0154] The source classification model and target classification model used in the second embodiment of the present invention may include a deep neural network structure. Deep neural network algorithms that may be used include a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBM), a Q network, a long short-term memory (LSTM), a gated recurrent unit (GRU), a transformer, a capsule network, etc., and the present invention is not limited thereto, and other artificial intelligence models may be used.

[0155] At this time, according to the second embodiment of the present invention, the multiple source classification models and target classification models included in each of the source classification model group and the target classification model group may use artificial intelligence models having the same neural network structure. However, the present invention is not limited thereto, and the source classification model group and the target classification model group may use classification models having different neural network structures.

[0156] The bone disease classification device (200) receives a CXR image to be read (hereinafter also referred to as a 'target image') and inputs the received target image into a group (304) of target classification models for which training has been completed performed by a classification model learning device (300) to predict whether there is a bone disease (osteoporosis / osteophytosis). Specifically, the bone disease classification device (200) divides the input target image into a plurality of segmented images corresponding to anatomical parts, and inputs each of the entire image or the segmented images into a corresponding target classification model for which training has been completed, thereby inferring a bone disease classification for each of the plurality of target classification models. Then, the bone disease classification device (200) diagnoses whether there is a final bone disease using an ensemble algorithm for the plurality of inferred bone disease classification results.

[0157] Referring to FIG. 12, a classification model learning device (300) according to a second embodiment of the present invention may include an image segmentation unit (310), a learning data set generation unit (320), a plurality of source classification model learning units (330), a plurality of target classification model learning units (340), and a transfer learning unit (350).

[0158] The image segmentation unit (310) receives a source learning image, which is a full CXR image of a normal person and an osteoporosis patient, and a source learning label (normal / osteoporosis), and segments the source learning image into images corresponding to each anatomical region. In addition, the image segmentation unit (310) receives a target learning image, which is a full CXR image of a normal person, an osteopenia patient, and an osteoporosis patient, and a corresponding target learning label (non-osteoporosis / osteoporosis), and segments the target learning image into images corresponding to each anatomical region.

[0159] The learning dataset generation unit (320) generates a plurality of source learning datasets based on the entire CXR image input as a source learning image, the source learning label (normal / osteoporosis) related thereto, and the plurality of segmented images segmented by the image segmentation unit (310). In addition, the learning dataset generation unit (320) generates a plurality of target learning datasets based on the entire CXR image input as a target learning image, the target learning label (non-osteoporosis / osteoporosis) related thereto, and the plurality of segmented images segmented by the image segmentation unit (310).

[0160] The source classification model learning unit (330) trains the source classification model group (303) based on the source learning images and source learning labels (normal / osteoporosis) of normal people and osteoporosis patients, which are relatively easy to learn. The source classification model learning unit (330) includes a plurality of source classification model learning units (330-1, 330-2, 330-3,., 330_n) that each train a plurality of source learning datasets. For example, the first source classification model learning unit (330_1) trains the first source classification model (artificial intelligence model) using the entire source learning images and source learning labels generated by the learning dataset generation unit (320). The second source classification model learning unit (330_2) trains the second source classification model (artificial intelligence model) using the first segmented images and source learning labels generated by the learning dataset generation unit (320).

[0161] The transfer learning unit (350) transfers the hidden layer parameters, which are the learning results of the source classification model group (303), to the target classification model group (304). Figure 13 is a diagram schematically illustrating the transfer learning unit (350) transferring the hidden layer parameters, which are the learning results of the source classification model group (303), to the target classification model group (304).

[0162] Referring to FIG. 13, the transfer learning unit (350) transfers and sets at least some of the parameters of the source classification model to the parameters of the corresponding target classification model. Specifically, the transfer learning unit (350) transfers and sets at least some of the parameters of the first source classification model to the parameters of the first target classification model, and transfers and sets at least some of the parameters of the second source classification model to the parameters of the second target classification model. In a similar manner, the transfer learning unit (350) transfers and sets at least some of the parameters of the n-th source classification model to the parameters of the n-th target classification model.

[0163] The target classification model learning unit (340) trains the target classification model group (304) in which the hidden layer parameters, which are the learning results of the source classification model group (303), are transferred and learned based on the target learning images and target learning labels (non-osteoporosis / osteoporosis) of normal people, osteopenia patients, and osteoporosis patients.

[0164] The target classification model learning unit (340) includes a plurality of target classification model learning units (340-1, 340-2, 340-3, …, 340_n) that each learn a plurality of target learning datasets. For example, the first target classification model learning unit (340_1) learns the first target classification model (artificial intelligence model) using the entire target learning image and target learning labels generated by the learning dataset generation unit (320). The second target classification model learning unit (340_2) learns the second target classification model (artificial intelligence model) using the first segmented image and target learning labels generated by the learning dataset generation unit (320).

[0165] The following describes a classification model learning method according to a second embodiment of the present invention with reference to FIG. 14. FIG. 14 is a diagram illustrating a classification model learning method according to a second embodiment of the present invention.

[0166] The classification model learning device (300) receives source learning images / source learning labels and target learning images / target learning labels as learning data. (S210) That is, the classification model learning device (300) receives source learning images, which are entire CXR images of a normal person and an osteoporosis patient, and corresponding source learning labels (normal / osteoporosis), and receives target learning images, which are entire CXR images of a normal person, an osteopenia patient, and an osteoporosis patient, and corresponding target learning labels (non-osteoporosis / osteoporosis).

[0167] Thereafter, the image segmentation unit (310) of the classification model learning device (300) segments the received source learning image and target learning image into a plurality of segmented images corresponding to anatomical regions, respectively. (S220) According to an embodiment of the present invention, the image segmentation unit (310) crops the entire input CXR image by anatomical region to generate segmented CXR images. Specifically, the image segmentation unit (310) crops the entire CXR image by anatomical region to generate segmented images corresponding to the entire chest image, right clavicle-scapula, left clavicle-scapula, cervical spine, and thoracic / lumbar spine.

[0168] The learning dataset generation unit (320) generates multiple source learning datasets based on the entire CXR image input as a source learning image, the corresponding source learning label (normal / osteoporosis), and multiple segmented images segmented by the image segmentation unit (310). (S230)

[0169] The learning dataset generation unit (320) generates multiple target learning datasets based on the entire CXR image input as a target learning image, the corresponding target learning label (osteoporosis / osteoporosis), and multiple segmented images segmented by the image segmentation unit (310). (S240)

[0170] The source classification model learning unit (330) uses multiple source learning datasets, each consisting of learning images of relatively easy-to-learn normal people and osteoporosis patients and source learning labels (normal / osteoporosis), as learning data for the corresponding source classification model, to learn the corresponding source classification model. (S250)

[0171] The transfer learning unit (350) performs transfer learning on a target classification model based on a source classification model for which training has been completed. (S260) That is, the transfer learning unit (350) performs transfer learning on a corresponding target classification model based on the artificial intelligence neural network parameters of the source classification model for which training has been completed. (S260)

[0172] The target classification model learning unit (340) uses multiple target learning datasets consisting of target learning images and target learning labels (non-osteoporosis / osteoporosis) of normal individuals, osteopenia patients, and osteoporosis patients for the transfer-learned target classification model as learning data for the corresponding target classification model, thereby learning the corresponding target classification model. (S270)

[0173] The target classification model learning unit (340) stores learning parameters (weights, biases, etc.), which are learning results of the target classification model based on the target learning dataset, in memory or storage. (S280)

[0174] The following describes a bone disease classification device and a bone disease classification method according to a second embodiment of the present invention with reference to FIGS. 15 to 17.

[0175] Referring to FIG. 15, a bone disease classification device (200) according to a second embodiment of the present invention may include an image segmentation unit (210), a bone disease classification inference unit (220), and a combination unit (230).

[0176] The image segmentation unit (210) receives the CXR image to be read (hereinafter also referred to as the 'target image') and (S310) segments the entire target image into images corresponding to each anatomical part. (S320) At this time, the image segmentation unit (210) of the bone disease classification device (200) segments the entire image into images corresponding to each anatomical part using the same segmentation method as the image segmentation unit (210) of the classification model learning device (300).

[0177] The bone disease classification inference unit (220) inputs the entire target image and the plurality of segmented images segmented by the image segmentation unit (210) into a corresponding target classification model among the plurality of target classification model groups (304) for which learning has been completed by the classification model learning device (300), and infers a bone disease classification result for each image (entire image or segmented image). (S330) At this time, the classification result inferred by the bone disease classification inference unit (220) may be a logit or probability value.

[0178] The combining unit (230) combines multiple bone disease classification results inferred by the bone disease classification inference unit (220) using an ensemble algorithm to diagnose whether the final bone disease is present (osteoporosis / non-osteoporosis). (S340)

[0179] Fig. 17 is a drawing showing a bonding process performed in a bonding part (230) according to a second embodiment of the present invention.

[0180] Referring to FIG. 17, according to the second embodiment of the present invention, the combining unit (230) can combine (synthesize) multiple bone disease classification results using an ensemble algorithm. For example, the combining unit (230) can average logit or probability values, which are bone disease classification result values ​​inferred from each classification model, and compare the average value with a reference value to determine whether there is a bone disease (osteoporosis / non-osteoporosis). At this time, according to the embodiment of the present invention, when combining multiple bone disease classification results, the combining unit (230) can determine whether there is a bone disease (osteoporosis / non-osteoporosis) by calculating an average value only for the remaining results excluding one or more results. Meanwhile, according to the embodiment of the present invention, various techniques other than averaging can be applied as the ensemble technique. For example, any one or a combination of voting, bagging, boosting, and stacking can be used as the ensemble technique.

[0181] In this way, according to an embodiment of the present invention, since the final classification result is derived by integrating the multiple inference results obtained by applying various bone regions included in the segmented image to multiple classification models (artificial intelligence models) using an ensemble algorithm, it is possible to sufficiently reflect the feature information of various anatomical regions while effectively excluding specific regions that may cause a decline in classification performance. Therefore, even if the segmented image (especially, the image segmented by cropping) contains artifacts, there is an advantage in that performance degradation can be prevented without introducing a separate auxiliary model for artifact recognition.

[0182] In addition, in order to learn a classification model with higher accuracy and infer classification results, it is necessary to use not only multiple segmented images but also the entire image, for the following reasons.

[0183] Chest X-ray images (CXR images) may have differences in anatomical locations within the subject due to factors such as the skill of the photographer, equipment, and race. In the embodiment of the present invention, instead of using a separate artificial intelligence model to detect bone areas or perform segmentation, the original CXR image is divided into equal parts (cropped) and the segmented images including each anatomical area are used as learning images or target images. In this way, if multiple classification models (artificial intelligence models) are used only for the segmented images that determine only each individual piece area without using a classification model that analyzes the entire image, there may be a possibility of performance degradation depending on the quality of the CXR image. Therefore, it is important to input the entire image as a learning image or target image and to perform a classification result by ensembling the judgment results for each individual anatomical area based on the judgment result considering the entire area in the CXR.

[0184] The following describes the experimental results compared to a comparative example to demonstrate the effectiveness of the second embodiment of the present invention.

[0185]

[0186] 1. Preparing the dataset

[0187] We received 57,589 anonymized post-anterior view chest X-ray images and label data from the Asan Medical Center, South Korea (IRB No. 2019-1226). Based on these data, we prepared datasets (training data, validation data) for learning the classification models of the second embodiment of the present invention and the comparative example, as well as datasets for internal performance evaluation, as shown in Table 2.

[0188] Training Data Validation Data Test Data Normal Osteopenia Osteoporosis Normal Osteopenia Osteoporosis Normal Osteopenia Osteoporosis 28,68117,3672,7314,0402,458323673652664

[0189] 2. Creation of a classification model

[0190] 2.1. Creation of a model (ensemble application model of multiple individual classification models of the entire image and multiple segmented images) for implementation example 1_corresponding to the second embodiment of the present invention.

[0191] Based on the prepared training data set and validation data set in Table 2, the entire chest X-ray image (CXR image) (input image resolution: 1024 X 1024) and the left shoulder, right shoulder, cervical spine, and spine each part (segmented) images extracted from the entire image (input image resolution: spine image - 512 X 1024, cervical spine / left and right shoulders - 512 X 512) were prepared.

[0192] Using the InceptionV3 architecture, a neural network model, a source classification model for transfer learning was created by training with images labeled “normal” and “osteoporosis” among the entire CXR image and the extracted images (segmented images) by region.

[0193] Among the whole image and the segmented images (segmented images), the images corresponding to “normal” and “osteopenia” were categorized as “non-osteoporosis” and labeled, and the images corresponding to “osteoporosis” were labeled as “osteoporosis”. The whole image / segmented images (segmented images) and their labels were transferred to the source classification model to create a target classification model (expert model) for the whole image and each segmented image. Using an ensemble algorithm, a model was created that can output the “final inference classification result” by excluding the “preliminary inference classification result (label and probability value)” with the lowest probability (or logit) among the “preliminary inference classification result” of the 5 target classification models learned for each segment (whole image, left shoulder, right shoulder, neck, spine) and calculating the average.

[0194]

[0195] 2.2. Comparative Example 1 Creation of Model (Single Classification Model)_ Some modifications of the second embodiment of the present invention

[0196] Based on the prepared learning dataset and validation dataset in Table 2, the entire chest X-ray image (CXR image) (input image resolution: 1024 X 1024) and the left shoulder, right shoulder, cervical spine, and spine each region (segmented) images extracted from the entire image (input image resolution: spine image - 512 X 1024, cervical spine / left and right shoulders - 512 X 512) were prepared.

[0197] Among the above regional images (segmented images), the images corresponding to “normal” and “osteopenia” were labeled as “non-osteoporosis,” and the images corresponding to “osteoporosis” were labeled as “osteoporosis.” The above regional images (segmented images) and labels were trained based on the InceptionV3 architecture, and four individual classification models specialized for each anatomical region were created.

[0198]

[0199] 2.3 Implementation Example 2: Creation of a model (an ensemble application model of multiple individual classification models excluding the classification model of the entire image)

[0200] Based on the prepared learning dataset and validation dataset in Table 2, the entire chest X-ray image (CXR image) (input image resolution: 1024 X 1024) and the left shoulder, right shoulder, cervical spine, and spine each region (segmented) images extracted from the entire image (input image resolution: spine image - 512 X 1024, cervical spine / left and right shoulders - 512 X 512) were prepared.

[0201] Using the InceptionV3 architecture, a neural network model, a source classification model for transfer learning was created by training with images labeled “normal” and “osteoporosis” among the extracted images (segmented images) by region.

[0202] Among the above-mentioned regional images (segmented images), the images corresponding to “normal” and “osteopenia” were categorized as “non-osteoporosis” and labeled, and the images corresponding to “osteoporosis” were labeled as “osteoporosis”, and the segmented images and their labels were transferred to the source classification model to create a target classification model for each regional image (segmented image). By applying an ensemble algorithm, a model capable of outputting a “final inference classification result” by averaging the “preliminary inference classification results (labels and probability values)” of the target classification models for two types of anatomical regions (“cervical spine-spine”, “cervical spine-left shoulder”, “cervical spine-right shoulder”, “spine-left shoulder”, “spine-right shoulder”, or “left shoulder-right shoulder”) was created.

[0203]

[0204] 3. Experimental Results_Internal Performance Evaluation (Internal Test) Results of Implementation Examples 1 and 2 and Comparative Example 1

[0205] In order to compare the performance of the classification model (source classification model-target classification model) according to the second embodiment of the present invention and a single classification model for each anatomical region, the sensitivity, specificity, positive predictive value, and negative predictive value were evaluated based on classification accuracy (AUC) and True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN).

[0206] As can be seen from Tables 3 and 4, the classification model according to Implementation Example 1 exhibited a high level of classification accuracy (AUC) compared to the classification model of Comparative Example 1, and a high level of negative predictive value (NPV) compared to Comparative Example 1. It is noteworthy that a high level of negative predictive value was achieved, as a patient classified as negative for osteoporosis may actually have the disease, which may lead to worsening of osteoporosis or fracture.

[0207] Comparative Example 1 Left shoulder Right shoulder Cervical spine Accuracy (AUC) 0.89860.90770.90120.9092TP504549513496TN1141108111331148FP185245193178FN159114150167 Sensitivity, %76.0282.8177.3874.81 Specificity, %86.0581.5285.4486.58 Positive predictive value (PPV), %73.1569.1472.6673.59 Negative predictive value (NPV), NPV 87.7790.4688.3187.30

[0208] Implementation Example 1 Implementation Example 2 Full image + partial image (cervical spine + spine + left shoulder + right shoulder) Partial image 1 (cervical spine + spine) Partial image 2 (cervical spine + left shoulder) Partial image 3 (cervical spine + right shoulder) Partial image 4 (spine + left shoulder) Partial image 5 (spine + left shoulder) Partial image 6 (left shoulder + right shoulder) Accuracy (AUC) 0.93880.92410.91660.920.92090.92550.9139TP576483491520467500501TN1146120211931174120711921155FP180124133152119134171FN87180172143196163162 Sensitivity, %86.8872.8574.0678.4370.4475.4175.57 Specificity, %86.4390.6589.9788.5491.0389.8987.10 Positive Predictive Value (PPV), %76.1979.5778.6977.3879.6978.8674.55 Negative Predictive Value (NPV), NPV92.9486.9887.4089.1486.0387.9787.70

[0209] In addition, the classification model of Implementation Example 2 showed a high level of classification accuracy (AUC) compared to the classification model of Comparative Example 1, and a high level of negative predictive value (NPV) compared to Comparative Example 1. It is noteworthy that a high level of negative predictive value was achieved because a patient classified as negative for osteoporosis actually has the disease, which can cause aggravation of osteoporosis or fracture. However, the classification model of Implementation Example 2 (using only segmented images, not the entire image) showed a slightly lower level of classification accuracy (AUC) and a slightly lower level of negative predictive value (NPV) compared to the classification model of Implementation Example 1.

[0210] According to the second embodiment of the present invention described above, since the target classification model is trained through target learning labels (osteoporosis / osteoporosis) divided into two categories, there is an advantage in that the target classification model can be sufficiently trained even with a relatively small number of images of normal people, osteopenia patients, and osteoporosis patients.

[0211] Meanwhile, unlike the second embodiment, in the present invention, if sufficient learning image data of normal people, osteopenia patients, and osteoporosis patients are provided, a target classification model can be trained with target learning labels (normal / osteopenia / osteoporosis) divided into three more detailed categories, and a more detailed bone disease (normal / osteopenia / osteoporosis) can be classified for the target image using the trained target classification model.

[0212] Below, a classification model learning method and a bone disease classification method according to a third embodiment of the present invention are described.

[0213] Fig. 18 is a diagram illustrating a classification model learning method according to a third embodiment of the present invention. The classification model learning method according to the third embodiment of the present invention is almost similar to the operation of the classification model learning device (300) according to the second embodiment of the present invention described with reference to Figs. 11 and 12. Accordingly, the classification model learning method according to the third embodiment of the present invention will be described below with reference to the classification model learning device (300) illustrated in Figs. 11 and 12.

[0214] The classification model learning device (300) receives source learning images / source learning labels and target learning images / target learning labels as learning data. (S410) That is, the classification model learning device (300) receives source learning images and source learning labels (normal / osteoporosis), which are entire CXR images of a normal person and an osteoporosis patient, and receives target learning images and target learning labels (normal / osteoporosis / osteoporosis), which are entire CXR images of a normal person, an osteopenia patient, and an osteoporosis patient.

[0215] Thereafter, the image segmentation unit (310) of the classification model learning device (300) segments the received source learning image and target learning image into multiple segmented images corresponding to anatomical parts. (S420)

[0216] The learning dataset generation unit (320) generates multiple source learning datasets based on the entire CXR image input as a source learning image, the source learning label (normal / osteoporosis) related thereto, and multiple segmented images segmented by the image segmentation unit (310). (S430)

[0217] The learning dataset generation unit (320) generates multiple target learning datasets based on the entire CXR image input as a target learning image, the target learning labels (normal / osteopenia / osteoporosis) related thereto, and multiple segmented images segmented by the image segmentation unit (310). (S440)

[0218] The source classification model learning unit (330) uses multiple source learning datasets, each consisting of relatively easy-to-learn learning images of normal people and osteoporosis patients and source learning labels (normal / osteoporosis), as learning data for the corresponding source classification model, to learn the corresponding source classification model. (S450)

[0219] The transfer learning unit (350) performs transfer learning on a target classification model based on a source classification model for which training has been completed. (S460) That is, the transfer learning unit (350) performs transfer learning on a corresponding target classification model based on the artificial intelligence neural network parameters of the source classification model for which training has been completed. (S460)

[0220] The target classification model learning unit (340) uses a plurality of target learning datasets, each consisting of target learning images and target learning labels (normal / osteopenia / osteoporosis) of normal, osteopenia, and osteoporosis patients, as learning data for the corresponding target classification model for the transfer-learned target classification model, thereby learning the corresponding target classification model. (S470)

[0221] The target classification model learning unit (340) stores learning parameters (weights, biases, etc.), which are learning results of the target classification model based on the target learning dataset, in memory or storage. (S480)

[0222] Fig. 19 is a drawing showing a bone disease classification method according to a third embodiment of the present invention. The bone disease classification method according to the third embodiment of the present invention is almost similar to the operation of the bone disease classification device according to the second embodiment of the present invention described with reference to Figs. 11, 15, and 17. Accordingly, the bone disease classification method according to the third embodiment of the present invention will be described below with reference to the bone disease classification device (200) illustrated in Figs. 11, 15, and 17.

[0223] The image segmentation unit (210) receives the CXR image to be read (hereinafter also referred to as the 'target image') and (S510) segments the entire target image into images corresponding to each anatomical part. (S520) At this time, the image segmentation unit (210) of the bone disease classification device (200) segments the entire image into images corresponding to each anatomical part using the same segmentation method as the image segmentation unit (210) of the classification model learning device (300).

[0224] The bone disease classification inference unit (220) inputs the entire target image and the plurality of segmented images segmented by the image segmentation unit (210) into a corresponding target classification model among the plurality of target classification model groups (304) for which learning has been completed by the classification model learning device (300), and infers a bone disease classification result for each image (entire image or segmented image). (S530) At this time, the classification result inferred by the bone disease classification inference unit (220) may be a logit or probability value.

[0225] The combining unit (230) combines multiple bone disease classification results inferred by the bone disease classification inference unit (220) using an ensemble algorithm to diagnose whether a final bone disease exists (osteoporosis / osteopenia / normal). (S540)

[0226] The embodiments described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. At this time, the medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, a RAM, a flash memory, etc.

[0227] Unless the steps constituting the method according to an embodiment of the present invention are explicitly described in a specific order or contrary to the description, the steps may be performed in any appropriate order, and the present invention is not limited to the order in which the steps are described. Any use of examples or exemplary terms herein is merely intended to illustrate the present invention in detail and is not intended to limit the scope of the present invention. Furthermore, those skilled in the art will appreciate that various modifications, combinations, and variations can be made within the scope of the claims or their equivalents.

[0228] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by a person having ordinary skill in the art to which the present invention pertains also fall within the scope of the present invention.

Claims

1. A classification model learning device that learns a corresponding classification model based on a plurality of segmented images and a learning label corresponding to each segmented image, wherein at least one of the plurality of classification models is an artificial intelligence model corresponding to each segmented image obtained by segmenting the entire image into corresponding anatomical parts; and An osteoporosis diagnosis system including a bone disease classification device that inputs a plurality of segmented images segmented from an image to be read into a corresponding classification model of the classification model learning device to diagnose osteoporosis.

2. In paragraph 1, The above multiple classification models include a first classification model corresponding to the entire image, The above classification model learning device is an osteoporosis diagnosis system that learns the first classification model based on the entire image and the corresponding learning label.

3. In paragraph 2, The above full image is an osteoporosis diagnosis system that is a chest X-ray image.

4. In paragraph 2, The above classification model learning device A first image segmentation unit that segments the entire image into anatomical parts to generate segmented images; A learning dataset generation unit that generates a plurality of learning datasets based on a full image of a normal person or an osteoporosis patient input from the outside and a plurality of segmented images segmented by the first image segmentation unit, each learning dataset including a learning image that is a full image or a segmented image and a learning label labeled as normal or osteoporosis; and An osteoporosis diagnosis system comprising a plurality of classification model learning units that each learn a corresponding classification model based on a plurality of learning datasets generated by the learning dataset generation unit.

5. In paragraph 4, The above first image segmentation unit crops and segments the entire image into images corresponding to each anatomical part, The above learning image generated by the above learning dataset generation unit is an osteoporosis diagnosis system including a full chest image and at least one segmented image of the right clavicle-scapula, left clavicle-scapula, cervical spine, and thoracic / lumbar spine.

6. In any one of paragraphs 1 to 5, The above bone disease classification device A second image segmentation unit that segments the above-mentioned target image into anatomical parts and generates multiple segmented images; An osteoporosis diagnosis system including a bone disease classification inference unit that inputs the entire target image for reading and the plurality of segmented images segmented by the second image segmentation unit into a corresponding classification model among the plurality of classification models of the classification model learning device, and infers a bone disease classification result for each input image.

7. In paragraph 6, The above bone disease classification device An osteoporosis diagnosis system further comprising a combining unit that combines multiple bone disease classification results inferred by the above-mentioned bone disease classification inference unit using an ensemble algorithm to diagnose whether the image to be read is an osteoporosis image or a normal image.

8. A classification model learning device that performs primary learning for multiple source classification models based on first learning images including segmented images segmented from the entire images of normal people and osteoporosis patients, and transfers the primary learning results and performs secondary learning for multiple target classification models based on second learning images including segmented images segmented from the entire images of normal people, osteopenia patients, and osteoporosis patients; and An osteoporosis diagnosis system including a bone disease classification device that diagnoses osteoporosis by inputting a plurality of segmented images segmented from a target image to a corresponding target classification model.

9. In paragraph 8, An osteoporosis diagnosis system wherein the plurality of source classification models and the plurality of target classification models each include a first source classification model and a first target classification model corresponding to the entire image.

10. In paragraph 9, The above full image is an osteoporosis diagnosis system that is a chest X-ray image.

11. In paragraph 9, The above classification model learning device A first image segmentation unit that receives a source learning image, which is an entire image of the normal person or the osteoporosis patient, and segments the source learning image into images corresponding to each anatomical region, and receives a target learning image, which is an entire image of the normal person, the osteopenia patient, or the osteoporosis patient, and segments the target learning image into images corresponding to each anatomical region; A learning dataset generation unit that generates a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis, and generates a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as non-osteoporosis or osteoporosis; A plurality of source classification model learning units that perform the first learning for the corresponding source classification model based on the plurality of source learning datasets; and An osteoporosis diagnosis system comprising a plurality of target classification model learning units that perform secondary learning on the corresponding target classification models based on the plurality of target learning datasets.

12. In paragraph 9, The above classification model learning device A first image segmentation unit that receives a source learning image, which is an entire image of the normal person or the osteoporosis patient, and segments the source learning image into images corresponding to each anatomical region, and receives a target learning image, which is an entire image of the normal person, the osteopenia patient, or the osteoporosis patient, and segments the target learning image into images corresponding to each anatomical region; A learning dataset generation unit that generates a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis, and generates a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as normal, osteopenia, or osteoporosis; A plurality of source classification model learning units that perform the first learning for the corresponding source classification model based on the plurality of source learning datasets; and An osteoporosis diagnosis system comprising a plurality of target classification model learning units that perform secondary learning on the corresponding target classification models based on the plurality of target learning datasets.

13. In paragraph 11 or 12, An osteoporosis diagnosis system wherein the above classification model learning device further includes a transfer learning unit that performs transfer learning for a corresponding target classification model learning unit based on hidden layer parameters, which are learning results of the source classification model learning unit.

14. In paragraph 13, The above first image segmentation unit crops and segments the entire image into images corresponding to each anatomical part, The above learning image generated by the above learning dataset generation unit is an osteoporosis diagnosis system including a full chest image and at least one segmented image of the right clavicle-scapula, left clavicle-scapula, cervical spine, and thoracic / lumbar spine.

15. In any one of paragraphs 8 to 12, The above bone disease classification device A second image segmentation unit that segments the above-mentioned target image into anatomical parts to generate multiple segmented images; and An osteoporosis diagnosis system including a bone disease classification inference unit that inputs the entire target image for reading and the plurality of segmented images segmented by the second image segmentation unit into a corresponding classification model among the plurality of target classification models, and infers a bone disease classification result for each input image.

16. In paragraph 15, The above-mentioned bone disease classification device is an osteoporosis diagnosis system further including a combining unit that combines a plurality of bone disease classification results inferred by the bone disease classification inference unit using an ensemble algorithm to diagnose whether the image to be read is an osteoporosis image.

17. In paragraph 16, The above bone disease classification device is an osteoporosis diagnosis system that diagnoses the above-described target image as an osteoporosis image or a non-osteoporosis image.

18. In paragraph 16, The above bone disease classification device is an osteoporosis diagnosis system that diagnoses the above-described target image as an osteoporosis image, an osteopenia image, or a normal image.

19. In a bone disease classification device that diagnoses osteoporosis by connecting to a classification model learning device that includes multiple classification models each corresponding to a whole image or multiple segmented images corresponding to each anatomical part of the whole image, The above bone disease classification device is a bone disease classification device that inputs a target image for reading or a plurality of segmented images segmented from the target image for reading into a corresponding classification model among the plurality of classification models, thereby diagnosing whether or not there is osteoporosis in the target image for reading.

20. In paragraph 19, The above bone disease classification device An image segmentation unit that segments the above-mentioned target image into anatomical parts to generate multiple segmented images; and A bone disease classification device including a bone disease classification inference unit that inputs the entire target image for reading and the plurality of segmented images segmented by the image segmentation unit into a corresponding classification model among the plurality of classification models and infers a bone disease classification result for each input image.

21. In paragraph 20, The above bone disease classification device further includes a combining unit that combines a plurality of bone disease classification results inferred by the bone disease classification inference unit using an ensemble algorithm to diagnose whether the image to be read is an osteoporosis image.

22. In paragraph 21, The above bone disease classification device is a bone disease classification device that diagnoses the above target image as an osteoporosis image or a non-osteoporosis image.

23. In paragraph 21, The above bone disease classification device is a bone disease classification device that diagnoses the above target image as an osteoporosis image, an osteopenia image, or a normal image.

24. In any one of paragraphs 19 to 23, The above multiple classification learning models are transferred from the learning results of the first classification model learned for the first time with learning data of normal people and osteoporosis patients, and are a bone disease classification device that is a classification learning model learned for the second time with learning data of normal people, osteopenia, and osteoporosis patients.

25. A classification model learning device that performs primary learning for multiple source classification models based on first learning images including segmented images segmented from full images of normal people and osteoporosis patients, and transfers the first learning results and performs secondary learning for multiple target classification models based on second learning images including segmented images segmented from full images of normal people, osteopenia patients, and osteoporosis patients.

26. In paragraph 25, A classification model learning device, wherein the plurality of source classification models and the plurality of target classification models each include a first source classification model and a first target classification model corresponding to the entire image.

27. In paragraph 26, The above classification model learning device A first image segmentation unit that receives a source learning image, which is an entire image of the normal person or the osteoporosis patient, and segments the source learning image into images corresponding to each anatomical region, and receives a target learning image, which is an entire image of the normal person, the osteopenia patient, or the osteoporosis patient, and segments the target learning image into images corresponding to each anatomical region; A learning dataset generation unit that generates a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis, and generates a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as non-osteoporosis or osteoporosis; A plurality of source classification model learning units that perform the first learning for the corresponding source classification model based on the plurality of source learning datasets; and A classification model learning device including a plurality of target classification model learning units that perform the secondary learning for the corresponding target classification model based on the plurality of target learning datasets.

28. In paragraph 26, The above classification model learning device A first image segmentation unit that receives a source learning image, which is an entire image of the normal person or the osteoporosis patient, and segments the source learning image into images corresponding to each anatomical region, and receives a target learning image, which is an entire image of the normal person, the non-osteoporosis patient, or the osteoporosis patient, and segments the target learning image into images corresponding to each anatomical region; A learning dataset generation unit that generates a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis, and generates a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as normal, non-osteoporosis, or osteoporosis; A plurality of source classification model learning units that perform the first learning for the corresponding source classification model based on the plurality of source learning datasets; and A classification model learning device including a plurality of target classification model learning units that perform the secondary learning for the corresponding target classification model based on the plurality of target learning datasets.

29. In paragraph 27 or 28, The above classification model learning device further includes a transfer learning unit that performs transfer learning for a corresponding target classification model learning unit based on hidden layer parameters, which are learning results of the source classification model learning unit.

30. In a method for diagnosing osteoporosis by diagnosing osteoporosis from an image to be read, A step of dividing the above-mentioned target image into multiple images corresponding to each anatomical part; A step of inputting the above-described target image and the divided plurality of segmented images into a corresponding classification model among a plurality of classification models, wherein at least one model among the plurality of classification models is an artificial intelligence model corresponding to each of the plurality of segmented images obtained by dividing the entire image into respective anatomical parts, and inferring a bone disease classification result for each input image; and A method for diagnosing osteoporosis, comprising a step of diagnosing whether the target image for reading is osteoporosis by combining the inferred results of classification of multiple bone diseases using an ensemble algorithm.

31. In paragraph 30, It further includes a step of learning the plurality of classification models based on the entire image of a normal person or an osteoporosis patient, The above classification model learning step is A step of dividing the entire image into anatomical parts to create a segmented image; A step of generating a plurality of learning datasets based on the entire learning image and the plurality of segmented segmented images, each learning dataset including a learning image that is the entire image or a segmented image and a learning label labeled as normal or osteoporosis; and A method for diagnosing osteoporosis, comprising a step of learning a corresponding classification model based on each of the plurality of learning datasets, 32. In paragraph 31, The above image is a chest X-ray image. A method for diagnosing osteoporosis, wherein the entire image is cropped into images corresponding to each anatomical part to create multiple segmented images.

33. In paragraph 30, It further includes a step of learning the plurality of classification models based on the whole images of a normal person, a non-osteoporotic patient, or an osteoporotic patient, The above-mentioned multiple classification learning models are transferred from the learning results of multiple source classification models that are first learned with first learning images of normal people and osteoporosis patients, and are multiple target classification learning models that are second learning models that are second learning images of normal people, osteopenia, and osteoporosis patients. A method for diagnosing osteoporosis.

34. In paragraph 33, A step of dividing a source learning image, which is an entire image of the above normal person or the above osteoporosis patient, into images corresponding to each anatomical part; A step of generating a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis; A step of performing the first learning for the corresponding source classification model based on the plurality of source learning datasets; A step of performing transfer learning for a corresponding target classification model based on the hidden layer parameters, which are the learning results of the above source classification model; A step of dividing a target learning image, which is an entire image of a normal person, a non-osteoporotic patient, or a patient with osteoporosis, into images corresponding to each anatomical part; A step of generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as osteoporosis or non-osteoporosis; and A method for diagnosing osteoporosis, comprising a step of performing the secondary learning for the corresponding target classification model based on the plurality of target learning datasets.

35. In paragraph 33, A step of dividing a source learning image, which is an entire image of the above normal person or the above osteoporosis patient, into images corresponding to each anatomical part; A step of generating a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis; A step of performing the first learning for the corresponding source classification model based on the plurality of source learning datasets; A step of performing transfer learning for a corresponding target classification model based on the hidden layer parameters, which are the learning results of the above source classification model; A step of dividing a target learning image, which is an entire image of a normal person, a non-osteoporotic patient, or a patient with osteoporosis, into images corresponding to each anatomical part; A step of generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as normal, osteopenia, or osteoporosis; and A method for diagnosing osteoporosis, comprising a step of performing the secondary learning for the corresponding target classification model based on the plurality of target learning datasets.

36. A method for learning multiple classification models based on the whole images of a normal person, a patient with osteopenia or osteoporosis, A step of performing primary learning for a plurality of source classification models based on first learning images including segmented images segmented from the entire images of normal people and osteoporosis patients; and A classification model learning method comprising a step of transferring the first learning result and performing second learning for multiple target classification models based on second learning images including segmented images segmented from the entire images of normal people, osteopenia patients, and osteoporosis patients.

37. In paragraph 36, The steps to perform the above first learning are A step of dividing a source learning image, which is an entire image of the above normal person or the above osteoporosis patient, into images corresponding to each anatomical part; A step of generating a plurality of source learning datasets based on the source learning image and a plurality of segmented images segmented from the source learning image, each source learning dataset including the first learning image, which is the source learning image or an image segmented from the source learning image, and a source learning label labeled as normal or osteoporosis; A classification model learning method comprising a step of performing the first learning for the corresponding source classification model based on the plurality of source learning datasets.

38. In paragraph 37, The steps for performing the above second learning are A step of performing transfer learning for a corresponding target classification model based on the hidden layer parameters, which are the learning results of the above source classification model; A step of dividing a target learning image, which is an entire image of a normal person, a non-osteoporotic patient, or a patient with osteoporosis, into images corresponding to each anatomical part; A step of generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as osteoporosis or non-osteoporosis; and A classification model learning method comprising a step of performing the second learning for the corresponding target classification model based on the plurality of target learning datasets.

39. In paragraph 37, The steps for performing the above second learning are A step of performing transfer learning for a corresponding target classification model based on the hidden layer parameters, which are the learning results of the above source classification model; A step of dividing a target learning image, which is an entire image of a normal person, a non-osteoporotic patient, or a patient with osteoporosis, into images corresponding to each anatomical part; A step of generating a plurality of target learning datasets based on the target learning image and a plurality of segmented images segmented from the target learning image, each target learning dataset including the second learning image, which is the target learning image or an image segmented from the target image, and a target learning label labeled as normal, osteopenia, or osteoporosis; and A classification model learning method comprising a step of performing the second learning for the corresponding target classification model based on the plurality of target learning datasets.

40. Comprising at least one processor implemented to execute computer-readable instructions, By at least one processor, The target image is divided into multiple images corresponding to each anatomical part, The above-mentioned target image for reading and the above-mentioned divided images are input into a corresponding classification model among a plurality of classification models - at least one of the plurality of classification models is an artificial intelligence model corresponding to each of the plurality of divided images obtained by dividing the entire image into respective anatomical parts - and the bone disease classification result is inferred for each input image. A computer that diagnoses whether the target image for reading is osteoporosis by combining the results of multiple bone disease classifications using an ensemble algorithm.

Images