Diagnosis assistance device, diagnosis assistance method, and computer program

Abstract

A diagnosis assistance device 20 according to the present invention comprises: an image acceptance unit 501 that accepts a simple X-ray image of a head and neck portion of a first subject; an image generation unit 502 that extracts an image of a vertebral region from the simple X-ray image of the head and neck portion of the first subject accepted by the image acceptance unit to generate a determination object image; a storage unit 40 in which a first trained model is stored, the first trained model being created through training by using, as teacher data, a set of a vertebral region image that is a part of a simple X-ray image of a head and neck portion of a second subject different from the first subject and an index value of bone mass or bone mineral density obtained from a result of a bone mineral density test through dual-energy X-ray absorptiometry performed on a tested portion other than the head and neck portion of the second subject, and outputting the index value of the bone mass or the bone mineral density with respect to an input of a vertebral region image that is a part of a simple X-ray image of a head and neck portion; a determination unit 505 that inputs the determination object image generated by the image generation unit to the first trained model and determines whether or not there is a possibility of osteoporosis in the first subject, on the basis of the index value of the bone mass or the bone mineral density outputted from the first trained model; and an output unit 505 that outputs the determination result by the determination unit.

Description

Diagnostic Support Device, Diagnostic Support Method, and Computer Program

[0001] The present invention relates to a device, a method, and a computer program for assisting a doctor in diagnosing osteoporosis.

[0002] Osteoporosis is a disease that frequently occurs in the elderly. Patients suffering from osteoporosis have brittle bones due to decreased bone density and are prone to fractures. When a fracture occurs, walking becomes difficult, leading to a decline in the quality of life (QOL) in daily life. In addition, depending on the location of the fracture, long-term hospitalization may be required. In modern society where the aging process is advancing, early detection and early treatment of osteoporosis are demanded.

[0003] For the diagnosis of osteoporosis, generally, the results of measuring the bone density (Bone Mineral Density; BMD) of the lumbar spine and femur using a bone density measuring device (Dual-energy X-ray absorptiometry; DXA device) are used. However, since the DXA device is a relatively large device and is expensive, small-scale hospitals have problems in terms of securing installation space and maintenance management costs.

[0004] On the other hand, a device for assisting a doctor in diagnosing osteoporosis using simple X-ray images of the chest and pelvis obtained during general health examinations and medical examinations has been proposed (Patent Document 1). In this device, simple X-ray images of abnormal chests and abnormal pelvises and simple X-ray images of normal chests and normal pelvises are learned by machine learning, and based on the results, a learning algorithm for determining the possibility of the presence of abnormal sites in the simple X-ray images of the patient's chest and pelvis is used. When diagnosing whether the patient has osteoporosis or not, a doctor can refer to the determination result by the device.

[0005] In the device described in Patent Document 1, when there are osteoporosis sites or fracture sites in the bones included in the simple X-ray images of the chest and pelvis, the simple X-ray images are used as simple X-ray images of abnormal chests and abnormal pelvises, and when there are no osteoporosis sites or fracture sites, the simple X-ray images are used as simple X-ray images of normal chests and normal pelvises for learning by machine learning.

[0006] Japanese Unexamined Patent Application Publication No. 2023-072479

[0007] Plain X-ray images of the chest and pelvis are largely composed of non-bone tissues such as the lungs, heart, intestines, and bladder. Furthermore, in plain chest X-rays, the ribs, clavicle, and thoracic vertebrae are superimposed, and in plain pelvic X-rays, the ilium and lumbar vertebrae are superimposed, making the morphological findings of the bones inconsistent and often unclear. For this reason, conventional methods that learn from the entire plain X-ray image as input data may find "apparent correlations" (shortcut learning) with background patterns unrelated to osteoporosis, and with bone shape and color due to differences in imaging conditions and body position, potentially reducing the contribution of bone indicators that should be learned. As a result, there is a risk that sufficient sensitivity and generalization performance for osteoporosis diagnosis may not be obtained.

[0008] The problem that this invention aims to solve is to enable accurate determination of whether or not there is a possibility of osteoporosis.

[0009] To solve the above problems, the diagnostic support device according to the present invention comprises: an image receiving unit that receives a plain X-ray image of the head and neck of a first subject; an image generation unit that extracts an image of the vertebral region from the plain X-ray image of the head and neck of the first subject received by the image receiving unit and generates an image to be judged; a storage unit that stores a first trained model which outputs an index value of bone mass or bone density in response to an input of an image of the vertebral region, which is part of a plain X-ray image of the head and neck of a second subject different from the first subject, and a set of index values ​​of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of the second subject, as training data; a determination unit which inputs the image to be judged generated by the image generation unit to the first trained model and determines whether or not the first subject has osteoporosis based on the index value of bone mass or bone density output from the first trained model; and an output unit which outputs the determination result from the determination unit.

[0010] In the present invention, the first subject and the second subject are typically human, but may be other mammals. However, the first subject and the second subject are mammals of the same species.

[0011] In the present invention, the first trained model is created by training it using a pair of training data: a vertebral region image, which is a part of a simple X-ray image of the head and neck of a second subject, and an index value of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry (DXA) on an examination site other than the head and neck of the second subject. The first trained model outputs an index value of bone mass or bone density in response to an input of a vertebral region image, which is a part of a simple X-ray image of the head and neck.

[0012] The vertebral region image extracted from the plain X-ray image of the head and neck of the first subject is, for example, an image that includes part or all of the first to seventh cervical vertebrae. Similarly, the vertebral region image that is part of the plain X-ray image of the head and neck of the second subject is, for example, an image that includes part or all of the first to seventh cervical vertebrae. Examples of images that include part or all of the first to seventh cervical vertebrae include an image that includes the entirety of any one or more of the first to seventh cervical vertebrae, and an image that includes only the vertebral bodies of the cervical vertebrae. In order to improve the accuracy of the determination unit's determination of whether or not the first subject has osteoporosis, it is preferable that the vertebral region image extracted from the plain X-ray image of the head and neck of the first subject corresponds to the vertebral region image of the second subject used to create the first trained model. That is, for example, if one is an image that includes only the vertebral bodies of the cervical vertebrae, it is preferable that the other is also an image that includes only the vertebral bodies of the cervical vertebrae.

[0013] The first to seventh cervical vertebrae (C1-C7), which are part of the head and neck region, are areas that relatively well reflect bone density in the hip joint and lumbar spine. Furthermore, in simple X-ray images of the head and neck, the first to seventh cervical vertebrae (C1-C7) (especially the third to sixth cervical vertebrae (C3-C6)) are less affected by the skull and clavicle, and their visualization is stable. Therefore, by using images of the cervical spine region as the main source of information for creating trained models and determining the presence or absence of osteoporosis, it is possible to accurately determine the presence or absence of osteoporosis.

[0014] The examination site other than the head and neck of the second subject is preferably the proximal femur (hip joint area) and / or lumbar spine, which are recommended as target sites for bone density testing using dual-energy X-ray absorptiometry (DXA). However, any site other than the proximal femur and lumbar spine is acceptable as long as bone density testing by DXA is possible. The examination site other than the head and neck of the second subject may be one or multiple sites. If there are multiple examination sites, the sum, mean, maximum, minimum, median, etc., of the bone mass or bone density index values ​​obtained from the results of bone density testing by DXA at ​​each of these multiple sites will be used as training data.

[0015] Indicators of bone mass or bone density obtained from bone density tests include bone mass or bone density, YAM (Young Adult Mean) value, T-score, etc. While bone mass or bone density, YAM value, and T-score can be used directly as indicators, the results of binary classification of bone mass or bone density, YAM value, and T-score can also be used. Furthermore, if multiple sites are tested, the sum, average, maximum, minimum, and median values ​​of the bone density tests performed at each of these sites can be used as indicators.

[0016] In this invention, when the image receiving unit receives a simple X-ray image of the head and neck of a first subject, the image generation unit extracts an image of the vertebral region from the simple X-ray image received by the image receiving unit and generates an image to be judged. The judgment unit then inputs the image to be judged into the first trained model and determines whether or not the first subject has osteoporosis based on the index value of bone mass or bone density output from the first trained model.

[0017] The present invention also relates to a diagnostic support method and a diagnostic support program. Specifically, the diagnostic support method according to the present invention is a diagnostic support method for supporting a physician's diagnosis of osteoporosis, and includes: receiving a plain X-ray image of the head and neck of a first subject; extracting an image of the vertebral region from the plain X-ray image of the head and neck of the first subject to generate an image to be judged; obtaining a dataset including an image of the vertebral region which is part of a plain X-ray image of the head and neck of a second subject different from the first subject, and an index value of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of the second subject; creating a trained model that outputs an index value of bone mass or bone density in response to an input of an image of the vertebral region which is part of a plain X-ray image of the head and neck, by training the set of the vertebral region image and the index value included in the dataset as training data; and inputting the image to be judged into the trained model and determining whether or not the first subject has a possibility of osteoporosis based on the index value of bone mass or bone density output from the trained model.

[0018] Furthermore, the diagnostic support program according to the present invention is a computer program for supporting a physician's diagnosis of osteoporosis, and comprises a computer equipped with a storage unit that stores a trained model, which was created by learning a pair of bone mass or bone density index values ​​obtained from a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of a second subject as training data, and which outputs a bone mass or bone density index value in response to an input of a vertebral region image, which is part of a simple X-ray image of the head and neck of a first subject different from the second subject; an image receiving unit that receives a simple X-ray image of the head and neck of a first subject different from the second subject; an image generation unit that extracts an image of the vertebral region from the simple X-ray image of the head and neck of the first subject received by the image receiving unit as a judgment image and generates a judgment target image; and a judgment unit that inputs the judgment target image generated by the image generation unit to the trained model and determines whether or not the first subject has osteoporosis based on the bone mass or bone density index value output from the trained model.

[0019] According to the present invention, it is possible to accurately determine whether or not a first subject has osteoporosis.

[0020] An overall configuration diagram of a diagnostic support system, which is one embodiment of the present invention. A flowchart for the process of determining whether or not a first subject has osteoporosis or osteopenia. A plain X-ray image of the head and neck of the first subject (a), a plain X-ray image with a rectangular frame indicating the vertebral region superimposed (b), and a target image for determination extracted from the plain X-ray image (c). A flowchart for the process of determining whether or not there is a possibility of osteoporosis. A diagram showing an example of a plain X-ray image of the head and neck with visual annotations indicating the determination result (a), and another example (b). A diagram showing an example of a display screen for the determination result. A diagram explaining the procedure for creating a target image for determination in a modified example.

[0021] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a block diagram showing the overall configuration of the diagnostic support system 1 according to this embodiment. In this embodiment, the diagnostic support system 1 will be described as a system that supports the diagnosis of osteoporosis in humans. Therefore, in the following description, "subject" always refers to a human being. The diagnostic support system 1 mainly consists of an X-ray imaging device 10 and a computer 20. The computer 20 functions as the diagnostic support device of the present invention. Hereinafter, the computer 20 will be referred to as the diagnostic support device 20.

[0022] <X-ray imaging device 10> The X-ray imaging device 10 is a device that irradiates various parts of a subject, such as the chest, abdomen, head, and limbs, with X-rays and images the internal structure of each part based on the amount of X-rays that pass through the subject's body. Images acquired by the X-ray imaging device 10 without the use of contrast agents are called plain X-ray images. In this embodiment, the X-ray imaging device 10 is used to acquire plain X-ray images of the subject's head and neck. The plain X-ray image data (X-ray image data) acquired by the X-ray imaging device 10 is transmitted to the diagnostic support device 20 via an interface. The X-ray imaging device 10 is an important element that serves as the starting point for the entire diagnostic support system 1.

[0023] <Diagnostic support device 20> The diagnostic support device 20 includes a communication unit 30, a storage unit 40, a control / processing unit 50, a display unit 60, and an input operation unit 70.

[0024] The communication unit 30 is connected to a communication network, including the Internet, by wired or wireless connection. The communication unit 30 can communicate with other devices such as cloud servers and web servers through the communication network.

[0025] The storage unit 40 is equipped with a hard disk drive or solid-state storage device (SSD) for storing large amounts of data. The storage unit 40 may also be equipped with a USB memory or other memory that is detachable from the diagnostic support device 20. The storage unit 40 stores programs used by the control / processing unit 50, trained models, and data used to train the trained models. The trained models will be described later. The storage unit 40 also stores plain X-ray image data of the subject's head and neck acquired by the X-ray imaging device 10, along with the subject's identification number and biological information. The biological information includes the subject's sex, age, height, weight, medical history, and lifestyle. The biological information may also include input items from the FRAX® questionnaire. The FRAX questionnaire is a fracture assessment tool created by an international collaborative research group of the World Health Organization (WHO) and includes questions about 11 risk factors (excluding bone density) related to the onset of fractures due to osteoporosis. The aforementioned subject is the subject of determination for the presence or absence of osteoporosis using the diagnostic support system 1, and corresponds to the first subject of the present invention.

[0026] The display unit 60 is composed of, for example, a liquid crystal display or an organic EL display. The display unit 60 displays images shown by the simple X-ray image data stored in the storage unit 40, images shown by the image data obtained after the simple X-ray image data has been processed by the diagnostic support device 20, and judgment results.

[0027] The input operation unit 70 is an interface for operators such as doctors and technicians to perform various operations, and consists of a keyboard, a pointing device such as a mouse, or a touch panel attached to the display unit 60. By operating the input operation unit 70, it is possible to instruct the diagnostic support device 20 to perform various operations or to input image data.

[0028] The control and processing unit 50 controls the operation of the entire diagnostic support device 20 and executes various functions of the diagnostic support device 20. The control and processing unit 50 includes, as functional blocks, an image receiving unit 501, a unit for extracting areas to be judged 502, a unit for processing images to be judged 504, an osteoporosis judgment unit 505, a display information assignment unit 506, etc. The unit for processing images to be judged 504 corresponds to the image generation unit of the present invention, and the osteoporosis judgment unit 505 corresponds to the judgment unit and output unit of the present invention.

[0029] <Trained Models> The trained models stored in the memory unit 40 are used to determine whether or not a subject has osteoporosis or osteopenia, and consist of a first trained model, a second trained model, and a third trained model. All of the aforementioned trained models are created by either machine learning or deep learning. Support vector machines (SVMs), random forests, and neural networks are used to create the trained models. Examples of neural networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), and adversarial neural networks (GANs). In this embodiment, the trained models are created by training using a CNN.

[0030] The trained model may be created in the control / processing unit 50, or it may be created in a computer separate from the diagnostic support device 20. When the trained model is created in the control / processing unit 50, the storage unit 40 stores a program (algorithm) for causing the control / processing unit 50 to function as a CNN, training data to be input to the CNN, and the like.

[0031] The subject from which training data used to train a pre-trained model is collected corresponds to the second subject of this invention. In explanations where it is necessary to identify the subject, the terms "first subject" and "second subject" will be used, while in general explanations, "subject" will be used.

[0032] The training data consists of a pair of input data and ground truth data. The control / processing unit 50 compares the data output from the CNN with the ground truth data when the input data is input to the CNN's input layer, and adjusts the trained model so that the error is minimized.

[0033] The first pre-trained model is created by training it using a set of bone mass or bone density index values ​​obtained from a vertebral region image, which is part of a simple X-ray image of the head and neck of a second subject, and a bone density test performed using dual-energy X-ray absorptiometry (DXA) on a bone density test site other than the head and neck of the second subject, as training data. The pre-trained model outputs a bone mass or bone density index value in response to an input of a vertebral region image, which is part of a simple X-ray image of the head and neck.

[0034] In this embodiment, the T-score obtained from bone density measurements performed by DXA on the femoral neck (hip joint) and lumbar spine of the second subject is used as training data. The bone density of the subject is BMD １ , the average bone density of young people (YAM value) is BMD ０ If the standard deviation of bone mineral density in young people is denoted as SD, the T-score is expressed by the following formula: T-score = (BMD) １ - BMD ０ ) / SD

[0035] Typically, subjects with a T-score of -1.0 or higher are diagnosed as normal, subjects with a T-score less than -1.0 but above -2.5 are diagnosed as having osteopenia, and subjects with a T-score below -2.5 are diagnosed as having osteoporosis. In this embodiment, the second subject with a T-score less than -1.0 is defined as "possibly having osteoporosis or osteopenia," and the second subject with a T-score of -1.0 or higher is defined as "not likely to have osteoporosis or osteopenia," thus classifying the second subject into two classes (binary classification). Although binary classification was used in this embodiment, it is also possible to classify subjects into three classes: "normal," "possibly having osteoporosis," and "possibly having osteopenia," according to the general diagnostic method for T-scores.

[0036] The second trained model is created by training on pairs of bone mass or bone density index values ​​and biometric information of a second subject, and the results of the determination of whether or not the second subject has osteoporosis, as training data. It outputs a determination of whether or not the subject has osteoporosis in response to input of bone mass or bone density index values ​​and biometric information.

[0037] The third pre-trained model is created by training the model using pairs of images of each cervical vertebra from the first to the seventh cervical vertebra, included in the simple X-ray images of the head and neck of the second subject, as training data. This pre-trained model outputs an evaluation score for an individual image of any of the cervical vertebrae from the first to the seventh cervical vertebra, given as input.

[0038] <Evaluation process for the possibility of osteoporosis or osteopenia> Next, the process by which the diagnostic support system 1 determines whether the first subject has osteoporosis or osteopenia will be explained with reference to the flowchart in Figure 2.

[0039] First, the operator uses the input operation unit 70 to input the identification number of the first subject to be judged into the diagnostic support device 20. Then, the judgment process starts, and the control / processing unit 50 reads the simple X-ray image data of the head and neck of the first subject corresponding to the identification number from the storage unit 40 and inputs it into the image reception unit 501 (step 101). As a result, the simple X-ray image of the head and neck of the first subject is received by the image reception unit 501. This step is an important step that marks the starting point of the judgment process. The control / processing unit 50 then displays the simple X-ray image of the head and neck on the display unit 60 based on the read simple X-ray image data of the head and neck (step 102).

[0040] Furthermore, the plain X-ray image data of the head and neck of the first subject received by the image receiving unit 501 is not limited to that acquired by the control and processing unit 50 from the storage unit 40, but may also be acquired from the X-ray imaging device 10 via the communication unit 30.

[0041] Next, the determination target part extraction unit 502 extracts a determination target image from the simple X-ray image received by the image reception unit 501. Specifically, the determination target part extraction unit 502 identifies a vertebral region from the simple X-ray image of the head and neck by means of segmentation processing using image analysis technology (step 103). When the vertebral region is identified, the control and processing unit 50 superimposes and displays a rectangular frame representing the vertebral region on the simple X-ray image displayed on the display unit 60. Subsequently, the determination target part extraction unit 502 extracts an image of the vertebra from the vertebral region by means of segmentation processing and creates determination target image data (step 104). The determination target image data is stored in the storage unit 40.

[0042] FIG. 3(a) shows an example of an image 101 represented by the simple X-ray image data received by the image reception unit 501. This image 101 is a simple X-ray image obtained by photographing the head and neck from the left side of the first subject. As shown in FIG. 3(a), the image 101 includes at least the first to seventh cervical vertebrae (C1 to C7), which are the cervical vertebrae of the neck, and a part of the skull. The determination target part extraction unit 502 identifies a region including all or part of the first to seventh cervical vertebrae (C1 to C7) from the image 101 as the vertebral region.

[0043] Generally, in a simple X-ray image of the head and neck, the first cervical vertebra and the second cervical vertebra are hidden by the skull, and the seventh cervical vertebra is hidden by the clavicle, and often cannot be visually recognized. In contrast, the third to sixth cervical vertebrae (C3 to C6) are not hidden by the skull and the clavicle and can be clearly visually recognized. Therefore, in the present embodiment, a region including the third to sixth cervical vertebrae is identified as the vertebral region. Thereby, variations in the position, shape, etc. of the vertebral region specified from the simple X-ray image of the head and neck can be suppressed to be small.

[0044] When the vertebral region is specified, as shown in FIG. 3(b), the control and processing unit 50 superimposes and displays a rectangular frame 102 indicating the vertebral region on the image 101 displayed on the display unit 60. The position and shape of the rectangular frame 102 displayed on the image 101 of the display unit 60 can be corrected by the operator operating the input operation unit 70. By enabling the operator to manually correct the rectangular frame 102 (vertebral region), the accuracy of the process of extracting the determination target image from the vertebral region, which will be described later, can be improved.

[0045] FIG. 3(c) shows an example of the determination target image 103 extracted from the vertebral region. In the present embodiment, the determination target image 103 is composed of vertebral body images each individually including the vertebral bodies of the third to sixth cervical vertebrae included in the vertebral region. The portions highlighted in white in the determination target image 103 are vertebral body images each individually including the vertebral bodies of the third to sixth cervical vertebrae. The vertebral body image corresponds to the individual cervical vertebra image of the present invention.

[0046] In the present embodiment, the determination target image 103 is composed only of images of the vertebral bodies of the third to sixth cervical vertebrae, but it is also possible to compose it from an image of the entire cervical vertebra including the vertebral body, spinous process, and transverse process.

[0047] Also, the operator may be able to operate the input operation unit 70 to correct the determination target image 103. By enabling correction by the manual operation of the operator, the accuracy of the image processing is further improved. Furthermore, the process of extracting the vertebral body from the vertebral region is not limited to the segmentation process, and the operator may be made to set a rectangular frame by manual operation and extract the region within the rectangular frame as the vertebral region.

[0048] Once the image to be judged 103 is created, the image processing unit 504 performs a normalization process on the image to be judged 103 (step 105). Normalization is a process that adjusts the size and tilt of the vertebral images to pre-set sizes and tilts, such as rotating, shearing, enlarging, or shrinking the vertebral images. The pre-set sizes and tilts in this normalization process are set to be approximately the same size and tilt as the vertebral region images included in the training data used to generate the trained model used for osteoporosis diagnosis, which will be described later. This improves the accuracy of the osteoporosis diagnosis unit 505's determination of the possibility of osteoporosis. In this embodiment, the vertebral images are normalized by size and tilt, but normalization may also be performed by shape or color brightness.

[0049] Through normalization processing, the vertebral bodies of the third to sixth cervical vertebrae in the image 103 to be judged are aligned according to certain criteria, resulting in an image that is optimal for determining the presence or absence of osteoporosis or osteopenia, as described later. Note that normalization processing of the image 103 to be judged is not necessarily required and may be skipped.

[0050] Once the normalization process is complete, the image processing unit 504 sets a certain margin around the normalized vertebral body image and crops the area including the normalized vertebral body image and the margin (step 106). The cropping process extracts only the information necessary for the determination, and the image to be determined 103 is obtained from which unnecessary parts and background information are removed, thereby improving the accuracy of the image processing by the image processing unit 504.

[0051] Next, the osteoporosis determination unit 505 determines whether the first subject has osteoporosis or osteopenia based on the vertebral body images of the third to sixth cervical vertebrae included in the cropped area (step 107). Hereinafter, the determination of whether or not there is a possibility of osteoporosis or osteopenia will be referred to as "osteoporosis determination". The osteoporosis determination process is carried out according to the flowchart shown in Figure 4.

[0052] First, the osteoporosis determination unit 505 evaluates whether the vertebral body images of the third to sixth cervical vertebrae included in the crop region are usable for osteoporosis determination. For this evaluation, a third pre-trained model is used, which has learned whether vertebral body images can be used for osteoporosis determination using vertebral body images annotated by a specialist as training data. When a vertebral body image is input to this third pre-trained model (step 1071), the third pre-trained model outputs an evaluation score for that vertebral body image (step 1072). The evaluation score is a numerical value that represents how useful the vertebral image is for osteoporosis determination, and the output evaluation score consists of a number from "1: usable" to "N: unusable" (N is a natural number greater than or equal to 2). The evaluation score may also be a continuous value.

[0053] Next, it is determined whether the evaluation score of the vertebral body image is above a threshold (a range useful for osteoporosis diagnosis) (step 1073). If the evaluation score is above the threshold (Yes in step 1073), the osteoporosis diagnosis unit 505 inputs the vertebral body image and the information of the first subject corresponding to the vertebral body image (sex, age, medical history, etc. of the subject) into a trained model for osteoporosis diagnosis (step 1074). On the other hand, if the evaluation score is below the threshold, the processing of the vertebral body image is terminated (No in step 1073). Thus, in this embodiment, only vertebral body images with evaluation scores useful for osteoporosis diagnosis are input into the trained model for osteoporosis diagnosis, improving the reliability of determining whether or not there is a possibility of osteoporosis.

[0054] The trained model used for osteoporosis diagnosis consists of the first trained model and the second trained model. The first trained model outputs an index value of bone density or bone mass in response to a vertebral image input, and the second trained model outputs an osteoporosis diagnosis result in response to the index value of bone density or bone mass and the subject's biological information (subject's sex, age, medical history, etc.). In step 1074, the vertebral image is first input to the first trained model, and then, when the index value of bone density or bone mass is output from the first trained model, that index value and the subject's biological information (subject's sex, age, medical history, etc.) are input to the second trained model, and finally, the osteoporosis diagnosis result of the vertebral image is output from the second trained model (step 1075). The osteoporosis diagnosis result indicates the possibility of osteoporosis for each vertebra.

[0055] In step 1076, it is determined whether there are other vertebral body images included in the image to be judged. If there are other vertebral body images (Yes in step 1076), the processing in steps 1071 to 1075 is repeated for those vertebral body images. On the other hand, if there are no other vertebral body images (No in step 1076), the process proceeds to the next step 1077.

[0056] In step 1077, the overall osteoporosis assessment value for the first subject is calculated from the osteoporosis assessment values ​​of each vertebral body image. The overall assessment value can be the average or sum of the osteoporosis assessment values ​​of the vertebral body images. Alternatively, the overall assessment value can be calculated by applying a weighting (linear sum) based on the evaluation score described above to the average or sum of the osteoporosis assessment values ​​of the vertebral body images.

[0057] Once the overall osteoporosis assessment value for the first subject is calculated, this overall assessment value is then compared with a threshold to determine whether the first subject has osteoporosis or osteopenia (step 1078). Specifically, if the overall assessment value is equal to or greater than the threshold, the first subject is determined to have a possibility of osteoporosis or osteopenia (positive), and if the overall assessment value is less than the threshold, the first subject is determined to have a low possibility of osteoporosis or osteopenia (negative). The result of the osteoporosis assessment is output to the display information assignment unit 506 (step 1079). This completes the osteoporosis assessment process.

[0058] Returning to the flowchart in Figure 2, the display information assignment unit 506 displays the osteoporosis judgment result output from the osteoporosis judgment unit 505 on the display unit 60 (step 108). Specifically, the display information assignment unit 506 adds a visual annotation representing the judgment result to the simple X-ray image of the head and neck displayed on the display unit 60. A physician who sees the visual annotation displayed on the display unit 60 can easily understand the osteoporosis judgment result of the first subject.

[0059] Figure 5 shows an example in which visual annotations representing the assessment results are added to a simple X-ray image of the head and neck displayed on the display unit 60. Specifically, in Figure 5(a), as a visual annotation, rectangular frames 602 (in this example, three rectangular frames surrounding the 3rd, 5th, and 6th cervical vertebrae, respectively) are added to the simple X-ray image of the head and neck 601, surrounding the cervical vertebrae that have high evaluation scores and are subject to osteoporosis assessment. This visually distinguishes between cervical vertebrae that are subject to osteoporosis assessment and those that are not.

[0060] In Figure 5(b), as a visual annotation, a figure 603 identical in shape to the vertebral pyramid of the cervical vertebrae that have high evaluation scores and are targeted for osteoporosis assessment is placed on top of the pyramidal pyramid of the head and neck in the plain X-ray image 601. The placement of figure 603 allows for recognition of which vertebra is being used as the basis for the osteoporosis assessment.

[0061] Figure 6 shows an example of the display screen for the judgment results shown on the display unit 60. In this example, a simple X-ray image 601 of the head and neck is displayed on the left side of the display screen 605, and the judgment result 606 is displayed on the right side. Above the simple X-ray image 601 of the head and neck displayed on the left side of the display screen 605, a rectangular frame 602 is displayed as a visual annotation, surrounding the cervical vertebrae that were judged to have high evaluation scores.

[0062] In addition, in this example, a cursor 607 is displayed for selecting or excluding cervical vertebrae to be evaluated from a simple X-ray image 601 of the head and neck. When the operator operates the input unit 70, the cursor 607 moves. The location pointed to by the cursor 607 is the vertebra to be evaluated selected by the operator. The operator can also operate the input unit 70 to select vertebrae that are not needed for evaluation, and exclude the selected vertebrae from the simple X-ray image 601.

[0063] Furthermore, in Figure 6, the right side of the display screen 605 shows the judgment result 606, which includes the osteoporosis judgment value for each cervical vertebra that was evaluated, the overall judgment value for all cervical vertebrae that were evaluated, and the message "Suspected osteoporosis" as the judgment result based on the overall judgment value. In this way, by simultaneously displaying the overall judgment value for multiple cervical vertebrae that were evaluated and the judgment value for each cervical vertebra as the judgment result, it is possible to grasp the detailed judgment result of the first subject.

[0064] The diagnostic support system 1 updates and displays the final diagnostic result based on the information of the cervical vertebrae selected for evaluation and those excluded from evaluation.

[0065] Thus, the diagnostic support system 1 of this embodiment, through the cooperation of the diagnostic support device 20 and the X-ray imaging device 10, can efficiently and accurately determine the possibility of osteoporosis based on simple X-ray images of the head and neck of the first subject. A specialist can then use this determination result as a reference to make a rapid and accurate diagnosis of osteoporosis. In other words, the diagnostic support system 1 of this embodiment contributes to the early diagnosis and treatment of osteoporosis in the first subject.

[0066] <Modification> In the above embodiment, the vertebral region was identified from a simple X-ray image of the head and neck of the first subject, and then the vertebral bodies of the third to sixth cervical vertebrae were extracted from the vertebral region, and the images of the third to sixth cervical vertebral bodies were used as the image to be determined 103. In contrast, as shown in the example in Figure 7, the region of the third to sixth cervical vertebrae (C3 to C6) may be identified (cropped) from the simple X-ray image 621 of the head and neck of the second subject, and the cervical vertebrae may be extracted from the region images 623 to 626 of the third to sixth cervical vertebrae to create the image to be determined.

[0067] [Embodiments] It will be apparent to those skilled in the art that the exemplary embodiments described above are specific examples of the following embodiments.

[0068] (Section 1) A diagnostic support device according to one aspect of the present invention includes: an image receiving unit that receives a plain X-ray image of the head and neck of a first subject; an image generation unit that extracts an image of the vertebral region from the plain X-ray image of the head and neck of the first subject received by the image receiving unit and generates an image to be judged; a storage unit that stores a first trained model which outputs an index value of bone mass or bone density in response to an input of an image of the vertebral region which is part of a plain X-ray image of the head and neck of a second subject different from the first subject, and a set of index values ​​of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of the second subject, as training data; a determination unit which inputs the image to be judged generated by the image generation unit to the first trained model and determines whether or not the first subject has osteoporosis based on the index value of bone mass or bone density output from the first trained model; and an output unit which outputs the determination result from the determination unit.

[0069] (Paragraph 11) The diagnostic support method relating to Paragraph 11 is a method for supporting a physician's diagnosis of osteoporosis, and includes: receiving a plain X-ray image of the head and neck of a first subject; extracting an image of the vertebral region from the plain X-ray image of the head and neck of the first subject to generate an image to be judged; obtaining a dataset including an image of the vertebral region which is part of a plain X-ray image of the head and neck of a second subject different from the first subject, and an index value of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of the second subject; creating a trained model that outputs an index value of bone mass or bone density in response to an input of an image of the vertebral region which is part of a plain X-ray image of the head and neck, by training the set of the vertebral region image and the index value included in the dataset as training data; and inputting the image to be judged into the trained model and determining whether or not the first subject has a possibility of osteoporosis based on the index value of bone mass or bone density output from the trained model.

[0070] (Paragraph 12) The computer program relating to Paragraph 12 is a computer program for assisting a physician in diagnosing osteoporosis, comprising: a computer having a storage unit that stores a trained model created by training a set of bone mass or bone density index values ​​obtained from a pair of bone mass or bone density index values ​​obtained from a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of a second subject, which is a part of a simple X-ray image of the head and neck of the second subject, and which outputs a bone mass or bone density index value in response to an input of a bone mass or bone density index value that is a part of a simple X-ray image of the head and neck of the first subject, an image receiving unit that receives a simple X-ray image of the head and neck of a first subject different from the second subject, and an image generating unit that extracts an image of the bone mass or bone density from the simple X-ray image of the head and neck of the first subject received by the image receiving unit as a judgment image and generates a judgment target image, The image generation unit inputs the image to be judged into the first trained model, and the first trained model outputs an index value of bone mass or bone density, which is then used to determine whether or not the first subject has osteoporosis.

[0071] According to the diagnostic support device described in paragraph 1, the diagnostic support method described in paragraph 11, and the computer program described in paragraph 12, images of the cervical spine region from the plain X-ray images of the subject's head and neck are used as the main source of information for creating a trained model and determining osteoporosis, thereby enabling accurate determination of whether or not the first subject has osteoporosis. Therefore, a physician can make a correct diagnosis of osteoporosis in the first subject by referring to the determination result output by the determination unit of the diagnostic support device.

[0072] (Paragraph 2) The diagnostic support device relating to Paragraph 2 is a diagnostic support device relating to Paragraph 1 in which the vertebral region image included in the training data used for training the first trained model is an image that includes part or all of the first to seventh cervical vertebrae, and the image generation unit extracts individual cervical vertebral images that include part or all of the first to seventh cervical vertebrae included in the vertebral region to generate an image to be judged.

[0073] According to the diagnostic support device described in paragraph 2, the determination unit inputs individual cervical vertebral images, each containing part or all of the first to seventh cervical vertebrae, into the first trained model, and determines whether the first subject has a possibility of osteoporosis based on the bone mass or bone density index value output from the first trained model. Since the first trained model outputs a bone mass or bone density index value for each individual cervical vertebral image, even if the image of any of the first to seventh cervical vertebrae included in the vertebral region, which is part of the simple X-ray image of the head and neck of the first subject, is unclear, if the images of the remaining cervical vertebrae are clear, it is possible to determine whether the first subject has a possibility of osteoporosis.

[0074] (Clause 3) The diagnostic support device relating to paragraph 3 is a diagnostic support device relating to paragraph 1 or 2, wherein the memory unit further stores a second trained model which is created by learning a pair of index values ​​of bone mass or bone density and biological information of the second subject and the result of determining whether or not the second subject has osteoporosis as training data, and outputs a result of determining whether or not the first subject has osteoporosis in response to input of index values ​​of bone mass or bone density and biological information, and the determination unit inputs the index values ​​of bone mass or bone density output from the first trained model to the second trained model and determines whether or not the first subject has osteoporosis based on the determination result output from the second trained model.

[0075] In the diagnostic support device according to paragraph 3, the determination result of whether or not a second subject has osteoporosis, which is used as training data for the second trained model, can be, for example, the result of a diagnosis made by one or more specialists to the second subject based on the index value of the second subject's bone mass or bone density and biological information. Alternatively, the determination result can be the result obtained by inputting the T-score as an index value of the second subject's bone mass or bone density and biological information into the 12 items of the FRAX® questionnaire (risk of fracture within 10 years (%)). According to the diagnostic support device according to paragraph 3, the determination of whether or not a first subject has osteoporosis can be made with greater accuracy.

[0076] (Article 4) The diagnostic support device relating to Article 4 is a diagnostic support device relating to Article 2 or Article 3, wherein the memory unit further stores a third trained model which is created by training the memory unit with a pair of images of the first to seventh cervical vertebrae and an evaluation score for evaluating the images of the cervical vertebrae, for each of the first to seventh cervical vertebrae included in the simple X-ray images of the head and neck of the second subject, and outputs an evaluation score for evaluating the image of the cervical vertebrae in response to an input of an individual cervical vertebra from the first to seventh cervical vertebrae, and the image generation unit inputs the individual cervical vertebrae images into the third trained model, outputs an evaluation score for the cervical vertebrae from the third trained model, and extracts individual cervical vertebrae images that include each cervical vertebra in which the evaluation score falls within a predetermined range to generate the image to be judged.

[0077] In the diagnostic support device according to paragraph 4, the image generation unit inputs the individual cervical vertebrae images into a third trained model and extracts individual cervical vertebrae images that include each cervical vertebra whose evaluation score, output from the third trained model, falls within a predetermined range. Here, cervical vertebrae whose evaluation score falls within a predetermined range refer to cervical vertebrae whose images are generally clear and whose shape is not distorted or missing. Individual cervical vertebrae images that include such cervical vertebrae are used to determine whether or not the first subject has osteoporosis, thus enabling accurate and highly precise determination.

[0078] (Clause 5) The diagnostic support device relating to paragraph 5 is a diagnostic support device relating to paragraph 2 or paragraph 4, wherein the first trained model outputs the respective index values ​​for the first to seventh cervical vertebrae included in the vertebral region image in response to the input of the vertebral region image, and the determination unit inputs one or more of the individual cervical vertebrae images extracted as the image to be determined to the first trained model, causes the first trained model to output the index values ​​for each of the one or more individual cervical vertebrae images, and determines whether or not the first subject has osteoporosis based on the statistical values ​​of the index values ​​for the one or more individual cervical vertebrae images.

[0079] In the diagnostic support device relating to paragraph 5, the statistical values ​​of the index values ​​of the one or more individual cervical spine images refer to the total value, mean, maximum value, minimum value, median value, etc. By using the statistical values ​​of the index values ​​of each of the one or more individual cervical spine images, it is possible to reliably determine whether or not the first subject has osteoporosis.

[0080] (Clause 6) The diagnostic support device according to Clause 6 is a diagnostic support device according to Clause 2 or Clause 4, wherein the first trained model outputs the respective index values ​​for the first to seventh cervical vertebrae included in the vertebral region image in response to the input of the vertebral region image, and the determination unit inputs one or more of the individual cervical vertebrae images extracted as the image to be determined to the first trained model, causes the first trained model to output the index values ​​for each of the one or more individual cervical vertebrae images, and determines whether or not the first subject has osteoporosis based on a linear sum in which the index values ​​are weighted by the evaluation score of each of the one or more individual cervical vertebrae images.

[0081] According to the diagnostic support device described in paragraph 6, it is possible to determine whether or not the first subject has osteoporosis based on index values ​​that reflect the quality of each of the one or more individual cervical spine images.

[0082] (Clause 7) The diagnostic support device relating to paragraph 7 is a diagnostic support device relating to any one of paragraphs 1 to 6, wherein the image generation unit normalizes an image of the vertebral region extracted from a simple X-ray image of the head and neck of the first subject and generates an image to be judged.

[0083] (Clause 8) The diagnostic support device relating to paragraph 8 is a diagnostic support device relating to paragraph 2 and any one of paragraphs 4 to 5, wherein the image generation unit normalizes individual cervical vertebrae images, each containing part or all of the first to seventh cervical vertebrae extracted from a simple X-ray image of the head and neck of the first subject, and generates an image to be judged.

[0084] Depending on the conditions under which a plain X-ray image of the head and neck of a first subject is taken using an X-ray imaging device, and the physical condition of the first subject, the images of the vertebral region and cervical vertebrae included in the plain X-ray image of the head and neck may be unclear, or partially missing due to being hidden by the skull, clavicle, etc. If a target image for judgment is generated from such images of the vertebral region and cervical vertebrae, it is not possible to accurately determine whether or not there is a possibility of osteoporosis. In contrast, the diagnostic support device described in paragraphs 7 and 8 normalizes the images of the vertebral region and individual cervical vertebrae extracted from the plain X-ray image of the head and neck of the first subject to generate a target image for judgment, thereby enabling an accurate determination of whether or not there is a possibility of osteoporosis. Examples of normalizing images of the vertebral region include normalizing the image by the shape, size, or brightness of the vertebral region. Examples of normalizing individual cervical vertebrae include normalizing the individual cervical vertebrae by the shape, size, inclination, or brightness of the individual cervical vertebrae. Normalization of both vertebral region images and individual cervical vertebral images can be performed by image rotation / shearing, scaling, color correction, etc.

[0085] (Paragraph 9) The diagnostic support device according to Paragraph 9 is a diagnostic support device according to any one of Paragraphs 1 to 8, further comprising: a display unit and a display control unit that displays a plain X-ray image of the head and neck of the first subject received by the image receiving unit on the screen of the display unit.

[0086] According to the diagnostic support device described in paragraph 9, a physician can make a diagnosis of osteoporosis in the first subject by referring to the judgment result output by the output unit and the plain X-ray image of the first subject's head and neck displayed on the screen of the display unit.

[0087] (Item 10) The diagnostic support device according to Item 10 is a diagnostic support device according to Item 9, wherein the display control unit displays the determination result output by the output unit on the screen of the display unit, either alongside or superimposed on the simple X-ray image of the head and neck of the first subject.

[0088] According to the diagnostic support device described in paragraph 10, both the plain X-ray image of the head and neck of the first subject and the result of the determination of whether or not there is a possibility of osteoporosis are displayed on the display unit, so that the physician can efficiently diagnose osteoporosis in the first subject. The determination results displayed on the display unit screen may include a message and a numerical value (%) indicating the possibility of osteoporosis. In addition, visual annotations indicating the possibility of osteoporosis may be added to each cervical vertebra in the vertebral region included in the plain X-ray image of the head and neck displayed on the display unit screen.

[0089] 1...Diagnostic support system 10...X-ray imaging device 20...Computer, diagnostic support device 30...Communication unit 40...Storage unit 50...Control / processing unit 501...Image reception unit 502...Determination target area extraction unit 503...Determination target image processing unit 505...Osteoporosis determination unit 506...Display information assignment unit 60...Display unit 70...Input operation unit

Claims

1. A diagnostic support device comprising: an image receiving unit that receives a plain X-ray image of the head and neck of a first subject; an image generation unit that extracts an image of the vertebral region from the plain X-ray image of the head and neck of the first subject received by the image receiving unit and generates an image to be judged; a storage unit that stores a first trained model which outputs an index value of bone mass or bone density in response to an input of an image of the vertebral region which is part of a plain X-ray image of the head and neck of a second subject different from the first subject, and a set of index values ​​of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of the second subject, as training data; a determination unit that inputs the image to be judged generated by the image generation unit to the first trained model and determines whether or not the first subject has osteoporosis based on the index value of bone mass or bone density output from the first trained model; and an output unit that outputs the determination result from the determination unit.

2. The diagnostic support device according to claim 1, wherein the vertebral region image included in the training data used to train the first trained model is an image including part or all of the first to seventh cervical vertebrae, and the image generation unit extracts one or more individual cervical vertebrae images, each including part or all of the first to seventh cervical vertebrae included in the vertebral region, to generate the image to be judged.

3. The diagnostic support device according to claim 1, wherein the memory unit further stores a second trained model which is created by training the memory unit with a set of index values ​​for bone mass or bone density and biological information of the second subject and the result of determining whether or not the second subject has osteoporosis as training data, and which outputs a result of determining whether or not the first subject has osteoporosis in response to input of index values ​​for bone mass or bone density and biological information, and the determination unit inputs the index values ​​for bone mass or bone density output from the first trained model to the second trained model and determines whether or not the first subject has osteoporosis based on the determination result output from the second trained model.

4. The diagnostic support device according to claim 2, wherein the memory unit further stores a third trained model, which is created by training the memory unit with training data consisting of a pair of an image of the cervical vertebra and an evaluation score for evaluating the image of each of the first to seventh cervical vertebrae included in the simple X-ray image of the head and neck of the second subject, and outputs an evaluation score for evaluating the image of any of the first to seventh cervical vertebrae as input, and the image generation unit inputs the individual cervical vertebrae images to the third trained model, outputs an evaluation score for the cervical vertebrae from the third trained model, and generates the image to be judged by extracting individual cervical vertebrae images that individually include cervical vertebrae whose evaluation scores fall within a predetermined range.

5. The diagnostic support device according to claim 2, wherein the first trained model, in response to input of the vertebral region image, individually outputs the respective index values ​​for the first to seventh cervical vertebrae included in the vertebral region image, and the determination unit inputs one or more of the individual cervical vertebrae images extracted as the image to be determined to the first trained model, causes the first trained model to output the index values ​​for each of the one or more individual cervical vertebrae images, and determines whether or not the first subject has osteoporosis based on the statistical values ​​(sum, mean, maximum, minimum, and median) of the index values ​​for the one or more individual cervical vertebrae images.

6. The diagnostic support device according to claim 2, wherein the first trained model, in response to input of the vertebral region image, individually outputs the index values ​​for each of the first to seventh cervical vertebrae included in the vertebral region image, the determination unit inputs the one or more individual cervical vertebrae images extracted as the image to be determined to the first trained model, causes the first trained model to output the index values ​​for each of the one or more individual cervical vertebrae images, and determines whether or not the first subject has osteoporosis based on a linear sum weighted by the evaluation score of each of the one or more individual cervical vertebrae images.

7. The diagnostic support device according to claim 1, wherein the image generation unit normalizes the image of the vertebral region extracted from the simple X-ray image of the head and neck of the first subject and generates the image to be judged.

8. The diagnostic support device according to claim 2, wherein the image generation unit normalizes individual cervical vertebral images, each containing part or all of the first to seventh cervical vertebrae extracted from a simple X-ray image of the head and neck of the first subject, and generates the image to be judged.

9. A diagnostic support device according to claim 1, further comprising: a display unit; and a display control unit that displays a plain X-ray image of the head and neck of the first subject received by the image receiving unit on the screen of the display unit.

10. The diagnostic support device according to claim 9, wherein the display control unit displays the determination result output by the output unit on the screen of the display unit, either alongside or superimposed on a plain X-ray image of the head and neck of the first subject.

11. A method for supporting a physician's diagnosis of osteoporosis, comprising: receiving a plain X-ray image of the head and neck of a first subject; extracting an image of the vertebral region from the plain X-ray image of the head and neck of the first subject to generate an image to be judged; obtaining a dataset including an image of the vertebral region which is part of a plain X-ray image of the head and neck of a second subject different from the first subject, and an index value of bone mass or bone density obtained from the results of a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of the second subject; creating a trained model that outputs an index value of bone mass or bone density in response to an input of an image of the vertebral region which is part of a plain X-ray image of the head and neck, by training the pair of the vertebral region image and the index value included in the dataset as training data; and inputting the image to be judged into the trained model and determining whether or not the first subject has a possibility of osteoporosis based on the index value of bone mass or bone density output from the trained model.

12. A computer program for assisting a physician in diagnosing osteoporosis, comprising: a computer equipped with a storage unit that stores a trained model, which is created by training a computer using a pair of bone mass or bone density index values ​​obtained from a bone density test performed by dual-energy X-ray absorptiometry on an examination site other than the head and neck of a second subject as training data, and which outputs a bone mass or bone density index value in response to an input of a vertebral region image, which is part of a simple X-ray image of the head and neck of a first subject different from the second subject; an image receiving unit that receives a simple X-ray image of the head and neck of a first subject different from the second subject; an image generation unit that extracts an image of the vertebral region from the simple X-ray image of the head and neck of the first subject received by the image receiving unit as a judgment image and generates a judgment target image; and a judgment unit that inputs the judgment target image generated by the image generation unit to the first trained model and determines whether or not the first subject has osteoporosis based on the bone mass or bone density index value output from the first trained model.

Images