Information processing system and information processing method

Abstract

This information processing system comprises: an information acquisition unit that acquires information on a bone of a subject person obtained on the basis of a first method and information on the bone of the subject person obtained on the basis of a second method different from the first method; and an image generation unit that generates a display image that displays the first information and the second information.

Description

Information processing system and information processing method

[0001] The present disclosure relates to an information processing system and an information processing method that assist in understanding bone conditions.

[0002] In recent years, it has been proposed to estimate or predict physical information of a subject using computer calculations. For example, Patent Literature 1 proposes a method for predicting future bone density of postmenopausal women using a machine learning model.

[0003] Japanese Patent Application Publication No. 2019-200788

[0004] An information processing system according to one aspect of the present disclosure includes a first acquisition unit that acquires first information indicating information relating to a subject's bones obtained based on a first method, and a generation unit that generates a display image that displays the first information together with information relating to the first method.

[0005] Furthermore, an information processing method according to one aspect of the present disclosure includes a first acquisition step of acquiring first information indicating information about the subject's bones obtained based on a first method, a second acquisition step of acquiring second information indicating information about the subject's bones obtained based on a second method different from the first method, and a generation step of generating a display image displaying the first information and the second information.

[0006] In addition, a server device according to one aspect of the present disclosure includes a first acquisition unit that acquires first information indicating information about the subject's bones obtained based on a first method, a second acquisition unit that acquires second information indicating information about the subject's bones obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0007] The display device, first generating device, second generating device, and third generating device according to each aspect of the present disclosure may be realized by a computer. In this case, the control programs of the display device, second generating device, and third generating device that cause the computer to operate as each part (software element) of the display device, first generating device, second generating device, and third generating device, and the computer-readable recording medium on which they are recorded, also fall within the scope of the present disclosure.

[0008] 1 is a block diagram showing a configuration of a main part of an example of an information processing system according to the present embodiment 1. FIG. 2 is a flowchart showing an example of the flow of display processing in a display device according to the present embodiment 1. FIG. 3 is a diagram showing an example of a display image displaying an estimated value of a subject's bone density at a time point after a time point that serves as a base point of estimation according to the present embodiment 1. FIG. 4 is a diagram showing another example of a display image displaying an estimated value of a subject's bone density at a time point after a time point that serves as a base point of estimation according to the present embodiment 1. FIG. 5 is a diagram showing another example of a display image displaying an estimated value of a subject's bone density at a time point before a time point that serves as a base point of estimation according to the present embodiment 1. FIG. 6 is a diagram showing an example of a display image displaying information related to the subject's bone density and a transition of the bone density of a reference person according to the present embodiment 1. FIG. 7 is a diagram showing an example of a display image displaying an estimated value in accordance with a change in the timing of starting treatment for a subject according to the present embodiment 1. FIG. 8 is a diagram showing an example of a display image displaying an error range of the estimated value of the bone density of a subject according to the present embodiment 1. FIG. 9 is a diagram showing an example of a display image displaying an estimation probability of the estimated value of the bone density of a subject according to the present embodiment 1. FIG. 1 is a diagram showing an example of a display image displaying medical images used to estimate estimated values ​​and measurement results corresponding to measured values ​​according to this embodiment 1. FIG. 2 is a diagram showing an example of a display image displaying changes in bone density of each of a plurality of parts according to this embodiment 1. FIG. 3 is a diagram showing an example of a display showing locations where a fracture is likely to occur, included in a display image displayed on a display unit according to this embodiment 1. FIG. 4 is a block diagram showing the configuration of the main parts of an example of an information processing system according to this embodiment 2. FIG. 5 is a block diagram showing the configuration of the main parts of an example of an information processing system according to this embodiment 3. FIG. 6 is a block diagram showing the configuration of the main parts of an example of an information processing system according to this embodiment 4. FIG. 7 is a diagram showing an example of a display image displaying a fracture risk of a subject according to this embodiment 4. FIG. 8 is a block diagram showing the configuration of the main parts of an example of an information processing system according to this embodiment 5. FIG. 9 is a block diagram showing the configuration of the main parts of an example of an information processing system according to this embodiment 6.

[0009] First Embodiment Hereinafter, one embodiment of the present disclosure will be described in detail with reference to FIGS. 1 to 15. FIG.

[0010] (Overview) First, an overview of this embodiment will be described using Fig. 1. Fig. 1 is a block diagram showing the configuration of a main part of an example of an information processing system 1 according to this embodiment. As shown in Fig. 1, the information processing system 1 includes a measuring device 2, an imaging device 3, and a display device (image generating device) 4.

[0011] In this embodiment, the measurement device 2 measures the subject's bones and transmits first information related to the subject's bones to the display device 4. That is, the display device 4 acquires first information indicating information related to the subject's bones obtained based on the bone measurement. Here, the first information may be information regarding at least one of the subject's bone density, bone mass, and bone quality. Furthermore, the information related to the first method may be, for example, an actual measurement value, an estimated value, or a predicted value. Furthermore, the first information may be information indicating the measurement value of the subject's bone density. In this embodiment, an example is described in which the first information is measurement information 431 indicating the measurement value of the subject's bone density. However, the term "bone density" in the "first information" described in this disclosure may be appropriately interpreted as at least one of "bone density," "bone mass," and "bone quality."

[0012] In this specification, the expression "measurement" may mean "actual measurement." Furthermore, the expression "measured value" may mean "actually measured value."

[0013] In the following description, the case where the subject to which the surgical treatment is being considered is a human (i.e., a "subject") will be described as an example, but the subject is not limited to humans. That is, the "subject" according to the present disclosure may also be a mammal other than a human, such as an equine, feline, canine, bovine, or porcine. The present disclosure also includes, among the following embodiments, embodiments in which "subject," "patient," and "person" are replaced with "animal" if the embodiments are applicable to these animals.

[0014] The first information may include information indicating the name of a disease from which the subject is suffering. The information indicating the name of the disease may be information indicating at least one of osteoporosis, rheumatism, osteonecrosis (e.g., osteonecrosis of the femoral head), systemic sclerosis, kidney disease, osteopetrosis, etc. The first information may also include bone evaluation information. The bone evaluation information may include information evaluated using a Fracture Risk Assessment Tool (FRAX (registered trademark)).

[0015] The bone density may be a value related to the density of bone. For example, the bone density may be expressed as bone mineral density per unit area (g / cm ２ ), bone mineral density per unit volume (g / cm ３ It may be expressed by at least one of the following: YAM (%), YAM (%), AGE (%), T-score, and Z-score. YAM is an abbreviation for "Young Adult Mean" and may also be referred to as young adult mean percent, %YAM, YAM ratio, etc. AGE may be a value compared to the average bone mineral density of the same age or age group. T-score is a value obtained by comparing the measured bone mineral density with the average young adult value and dividing it by the standard deviation for young adults. Furthermore, Z-score is a value obtained by comparing the measured bone mineral density with the average value for the same age as the subject and dividing it by the standard deviation for the same age. Bone mineral density is not limited to these examples and may be, for example, a unique numerical value.

[0016] The above-mentioned bone mass is an index related to bone density, and the bone mass is the amount of bone tissue in the skeleton.

[0017] The bone quality may be based on at least one of bone statistical properties, bone geometric properties, bone mechanical properties, and bone chemical properties. Bone quality may also include information on the subject's attributes, as described below. Bone quality may be based on at least one of bone metabolic markers, gender, race, menopausal status, age, cortical bone condition, cancellous bone condition, cancellous bone trabecular status, disease information, bone evaluation information, medication information, fracture presence / absence, number of fractures, fracture location, and fracture history. More specifically, bone quality may be based on at least one of bone formation markers, bone resorption markers, bone quality markers (e.g., vitamin K level), cortical bone thickness, trabecular density, trabecular orientation, and cancellous bone structure index (trabecular bone score). Bone quality may also be bone age. Bone age may be, for example, a measure of bone biological maturity expressed in terms of age. Bone age may be determined, for example, by the degree of ossification from medical images (eg, X-rays) of the hand.

[0018] The imaging device 3 transmits medical image data 432, which is data showing a medical image of the subject, to the display device 4.

[0019] The medical image may be at least one of an X-ray image, an MRI (magnetic resonance imaging) image, a CT (computed tomography) image, a PET (positron emission tomography) image, and an ultrasound image. The medical image may be, for example, an image showing at least a portion of the neck, chest, lower back, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, or temporomandibular joint. The X-ray image may also show parts other than bones. For example, a plain chest X-ray may include an image of the lungs and an image of the thoracic vertebrae. The X-ray image may be a frontal image showing the target area from the front, or a lateral image showing the target area from the side. The medical image may use multiple images. For example, the medical image may use multiple X-ray images, or an X-ray image and a CT image. When a plurality of X-ray images are used, for example, a plurality of frontal images may be used, a frontal image and a side image may be used, or a plurality of images showing different regions may be used.

[0020] The display device 4 estimates second information about the subject's bones using the medical image. Here, the second information may be information about at least one of the subject's bone density, bone mass, and bone quality. Furthermore, the second information may be information indicating an estimated value of at least one of the subject's bone density, bone mass, and bone quality. That is, the display device 4 acquires, for the subject, second information indicating information about the subject's bones obtained based on a second method different from the first method. Here, the first method may be a measurement of the subject's bones. Furthermore, the second method may be an estimation of the subject's bones.

[0021] The first and second methods may differ in at least one of the following, but are not limited to: measurement site, calculation method, calculation conditions, measurement method, measurement device, and measurement conditions. For example, the first method may be a measurement performed at a hospital equipped with a measurement device for measuring bones. The second method may be an estimation from a medical image. The medical image may be captured at a hospital without a measurement device, or may be captured at a hospital with a measurement device. The first and second methods may be performed at different hospitals. The first and second methods may use, for example, measured information, estimated information, and inferred information, but are not limited to these. The measured information may be, for example, the name of a disease confirmed by a doctor. The estimated or predicted information may be the name of a disease estimated or predicted from various information.

[0022] In this embodiment, an example is described in which the second information is estimated information 433 indicating an estimated value of the subject's bone density, but the term "bone density" in relation to the "second information" described in this disclosure may be appropriately interpreted as at least one of "bone density," "bone mass," and "bone quality."

[0023] Other examples of the first information indicating information about the subject's bones obtained based on bone measurements and the second information obtained based on bone estimation are given below.

[0024] In the case where the bones mainly include bones of the vertebrae or intervertebral discs, the first information and the second information may include, for example, information indicating at least one of vertebral body height, intervertebral body distance, anterior intervertebral disc space distance, posterior intervertebral disc space distance, poor posture classification, ossification occupation rate (ratio of ossified ligament thickness to the anterior-posterior diameter of the spinal canal), residual effective anterior-posterior diameter (thickness between the anterior-posterior diameter of the spinal canal and the ossified ligament), vertebral body posterior slip distance (distance between the posterior-inferior corner of the superior vertebra and the tip of the vertebral arch of the inferior vertebra), and Meyerding classification.

[0025] Diseases that can be diagnosed using the above information include, for example, spondylosis deformans, vertebral fractures (including traumatic and osteoporotic), spinal stenosis, herniation, degenerative spondylolisthesis, scoliosis, thoracic and lumbar degenerative diseases, bilateral congenital dislocation of the hip, and ossification of the spinal ligaments.

[0026] The following describes a case where the bones mainly include the hip bone. In this case, the first and second information may include, for example, information indicating at least one of the following: joint space area, joint space distance (width), minimum joint space distance (width), CE angle (Center Edge angle: the angle between a perpendicular line passing through the center of the femoral head and a line connecting the center of the femoral head and the lateral rim of the acetabulum), Sharp angle (Acetabular angle: the angle between a line connecting the lateral rim of the acetabulum to the tip of the teardrop and a line connecting both teardrops), ARO (Acetabular Roof Obliquity: the angle between a horizontal line passing through the acetabular floor and a line connecting the lateral rim of the acetabulum and the acetabular floor), AHI (Acetabular Head Index: the distance from the medial end of the femoral head to the lateral end of the acetabulum divided by the transverse diameter of the femoral head), and Kellgren-Lawrence (KL) classification. The first and second information may be at least one of an actual measurement, an estimated value, and a predicted value.

[0027] Diseases that can be diagnosed using the above information include, for example, osteoarthritis of the hip (OA), femoroacetabular impingement (one of the pathological conditions of OA), hip dysplasia, hip dislocation (including congenital), slipped femoral osteonecrosis, and femoral head necrosis.

[0028] In the following example, the bones mainly include knee bones. In this case, the first and second information may include, for example, information indicating at least one of the following: joint space area, joint space distance (width), minimum joint space distance (width), osteophyte area, femoro-tibial angle (lateral femoro-tibial angle), Osteoarthritis Research Society International (OARSI) classification, and Kellgren-Lawrence (KL) classification. The above information may enable, for example, a diagnosis of knee osteoarthritis (OA).

[0029] The case where the bones mainly include bones of the foot will be described. In this case, the first information and the second information may be information indicating at least one of the talus inclination angle and the talus anterior drawer distance. Diseases that can be diagnosed using the above information include, for example, ankle sprain, osteonecrosis of the talus, rheumatoid arthritis, and ankle impingement syndrome.

[0030] The display device 4 generates a display image that displays the first information and the second information. In this embodiment, an example will be described in which the display device 4 generates a display image that displays the measurement value of the subject's bone density indicated by the measurement information 431 and the estimated value of the subject's bone density indicated by the estimation information 433.

[0031] According to the above configuration, the information processing system 1 can generate a display image displaying information about a subject's bones obtained based on a plurality of different methods, for example, a measurement method and an estimation method. That is, in a medical setting, even without periodically measuring the subject's bones using a predetermined measuring device, it is possible to grasp the progress of the subject's bone condition by using a method other than measurement, such as estimation. For example, the progress of the subject's bone condition may be the progress of at least one of the subject's bone mineral density, bone mass, and bone quality.

[0032] Furthermore, the first information and the second information may be information on the same item indicating the bone condition, or may be information on different items. For example, the first information and the second information may be information on items that correspond to each other. Specifically, the first information and the second information may be information on items that correspond to each other by unit conversion, a conversion formula, or the like.

[0033] The measured values ​​of the subject displayed on the image may be at least one of the measured values ​​of bone density, bone mass, and bone quality of the subject measured at a first time point. Furthermore, the estimated values ​​of the subject displayed on the image may be estimated by an estimation model 4221 or the like provided in the display device 4 from a medical image showing the subject's bones at a second time point different from the first time point. The estimated values ​​may be at least one of the estimated values ​​of bone density, bone mass, and bone quality. The display device 4 may generate a display image in which the measured values ​​and the estimated values ​​of the subject are arranged in chronological order of the first and second time points.

[0034] According to the above configuration, the information processing system 1 generates an image in which measured information about the subject's bones at a first time point and estimated information about the subject's bones at a second time point are arranged in chronological order based on the first and second time points. That is, the information processing system 1 can generate an image in which measured values ​​and estimated values ​​of at least one of the subject's bone density, bone mass, and bone quality are arranged in chronological order. This allows the information processing system 1 to improve user convenience when estimating or inferring physical information, etc., of the subject.

[0035] (Information Processing System 1) Next, the information processing system 1 according to this embodiment will be described in detail. As described above, the information processing system 1 includes the measuring device 2, the imaging device 3, and the display device 4.

[0036] <<Measuring Device 2>> The measuring device 2 is a device that measures the bone density of a subject. The measuring device 2 may be any device that can measure the bone density of a subject, and is not particularly limited, but examples thereof include the following.

[0037] For example, the measuring device 2 may be a device that measures bone density using the DXA (Dual Energy X-ray Absorptiometry) method, the ultrasound method, or the MD (Micro Densitometry) method.

[0038] The measuring device 2 transmits measurement information 431 indicating the measurement value of the subject's bone density to the display device 4. In other words, the measurement information 431 can be said to be information measured using a predetermined measuring device that performs bone measurements. The measurement information 431 may also include information indicating the date and time the measurement was performed. Communication between the measuring device 2 and the display device 4 may be wired or wireless. Furthermore, the communication may be via the Internet.

[0039] <Imaging Device 3> The imaging device 3 may be, for example, a device that captures images to obtain information about the inside of a subject's body. The imaging device 3 is a device that captures medical images of the subject. The medical images captured by the imaging device 3 may be images that include information about the subject's bones, and are not particularly limited to the following examples: For example, the medical images may be X-ray images, MRI images, CT images, PET images, or ultrasound images.

[0040] The imaging device 3 transmits medical image data 432, which is data indicating the captured medical image, to the display device 4. The medical image data 432 may include information indicating the date and time when the medical image was captured. Communication between the imaging device 3 and the display device 4 may be wired or wireless. Furthermore, the communication may be via the Internet. If the imaging device 3 cannot communicate directly with the display device 4, the medical image data 432 may be transferred using a portable memory or the like.

[0041] The display device 4 generates a display image showing the measured value of the subject's bone mineral density and the estimated value of the subject's bone mineral density, and displays the image. The display device 4 includes a communication unit 41, a control unit 42, a storage unit 43, an operation input unit 44, and a display unit 45.

[0042] <Communication Unit 41> The communication unit 41 communicates with the measurement device 2 and the imaging device 3. The communication unit 41 receives measurement information 431 from the measurement device 2. The communication unit 41 also receives medical image data 432 from the imaging device 3.

[0043] <Control Unit 42> The control unit 42 comprehensively controls each unit of the display device 4. The control unit 42 includes an acquisition unit 421 (first acquisition unit), a first generation unit 422, an operation acceptance unit 423, an image generation unit (generation unit) 424, an information acquisition unit (first acquisition unit, second acquisition unit, third acquisition unit) 425, a display control unit 426, and a second generation unit 427.

[0044] <<Acquisition Unit 421>> The acquisition unit 421 acquires measurement information 431 from the measurement device 2 via the communication unit 41. The acquisition unit 421 also acquires medical image data 432 from the imaging device 3 via the communication unit 41.

[0045] The acquisition unit 421 stores the acquired measurement information 431 and medical image data 432 in the storage unit 43. The acquisition unit 421 also transmits the acquired medical image data 432 to the first generation unit 422.

[0046] <<First generation unit 422>> The first generation unit 422 generates estimation information 433 indicating an estimated value of the bone mineral density of the subject estimated from a medical image (first image) indicated by medical image data 432 of the subject received from the acquisition unit 421. The first generation unit 422 stores the generated estimation information 433 in the storage unit 43. The first generation unit 422 includes an estimation model 4221.

[0047] <Estimation Model 4221> The estimation model 4221 is a model that estimates the bone density of a subject from a medical image represented by the subject's medical image data 432. The estimation model 4221 may be set based on training data including a medical image (second image) showing the bones of the first person and teacher data including at least one value of the first person's bone density, bone mass, and bone quality. In other words, the estimation model 4221 is a trained model.

[0048] For example, the bone density of a person whose bones appear in the training plain X-ray image shown by each of the multiple training image data included in the training data may be associated as training data. The bone density of the training data may be bone density obtained using DXA, ultrasound, or MD. Furthermore, the above-mentioned "first person" does not have to be limited to a human. For example, it may be an animal of the same species as the "subject."

[0049] <<Operation Receiving Unit 423>> The operation receiving unit 423 receives an operation input from a user via the operation input unit 44. The operation receiving unit 423 receives an operation input instructing display of a display image showing information about the bones of the subject. Upon receiving the operation input, the operation receiving unit 423 instructs the image generating unit 424 to generate an image.

[0050] The operation accepting unit 423 accepts an operation input instructing a guess regarding the subject's bones. Upon accepting the operation input, the operation accepting unit 423 instructs the second generating unit 427 to generate guess information 434 indicating a guess regarding the subject's bones. Furthermore, the operation accepting unit 423 instructs the image generating unit 424 to generate a display image displaying the guess information 434.

[0051] The operation accepting unit 423 accepts an operation input instructing to end the display. Upon accepting the operation input, the operation accepting unit 423 instructs the display control unit 426 to end the display.

[0052] <<Image Generation Unit 424>> When the image generation unit 424 receives an instruction to generate a display image that displays information about the subject's bones, the image generation unit 424 acquires the measurement information 431 and the estimated information 433 from the memory unit 43 via the information acquisition unit 425. The image generation unit 424 uses the measurement information 431 and the estimated information 433 to generate a display image that displays information about the subject's bones.

[0053] When the image generation unit 424 receives an instruction to generate a display image that displays the estimated information 434, the image generation unit 424 acquires the measurement information 431, the estimated information 433, and the estimated information 434 from the storage unit 43 via the information acquisition unit 425. The image generation unit 424 uses the measurement information 431, the estimated information 433, and the estimated information 434 to generate a display image that displays information about the subject's bones.

[0054] The image generating unit 424 transmits data of the generated display image to the display control unit 426. Examples of the display image generated by the image generating unit 424 will be described in detail later.

[0055] <<Information Acquisition Unit 425 >> The information acquisition unit 425 acquires the measurement information 431 and the estimation information 433 stored in the storage unit 43 in response to an instruction from the image generation unit 424 , and transmits them to the image generation unit 424 .

[0056] In response to an instruction from the image generating section 424 , the information acquiring section 425 acquires the measurement information 431 , the estimation information 433 and the inferred information 434 stored in the storage section 43 and transmits them to the image generating section 424 .

[0057] <<Display Control Unit 426 >> The display control unit 426 causes the display unit 45 to display a display image indicated by the image data received from the image generation unit 424 .

[0058] The display control unit 426 ends the image display on the display unit 45 in response to the instruction to end the display received from the operation accepting unit 423 .

[0059] <<Second generation unit 427>> The second generation unit 427 generates estimated information 434 indicating an estimated value of the subject's bone density estimated from the medical image indicated by the medical image data 432 stored in the memory unit 43.

[0060] For example, the second generator 427 may generate estimated information 434 relating to at least one of bone density, bone mass, and bone quality estimated from a medical image depicting the subject's bones at a first time point when at least one of the subject's bone density, bone mass, and bone quality was measured. In this embodiment, an example will be described in which the estimated information 434 indicates an estimated value of the subject's bone density, but the term "estimated information 434" in the present disclosure may be appropriately interpreted as information indicating at least one of "bone density," "bone mass," and "bone quality."

[0061] Here, the measurement of bone density at the first time point and the capture of a medical image depicting the subject's bones at the first time point do not necessarily have to be performed at exactly the same time. For example, the measurement and the capture may be performed within a predetermined period of time. The predetermined period is not particularly limited as long as it is a period during which a significant change in the subject's bone density is unlikely to occur, and examples include half a day, one day, one week, one month, three months, and six months. Furthermore, if a medical image depicting the subject's bones has not been captured at the first time point, the display device 4 may not receive medical image data 432 representing a medical image depicting the subject's bones at the first time point. In other words, if a medical image depicting the subject's bones has been captured at the first time point, the display device 4 may accept medical image data 432 representing a medical image depicting the subject's bones at the first time point. In this specification, the time point at which the medical image used to generate the estimated information 434 was captured is referred to as the base point of estimation. The estimated information 434 may also be information indicating an estimated value of the subject's bone density at a third time point. The third time point may be a time point different from the first time point.

[0062] The second generating unit 427 may generate estimated information 434 regarding the bone density of the subject, estimated from the medical image showing the bones of the subject at the second time point used by the first generating unit 422 to estimate the estimated value of the bone density of the subject. The estimated information 434 may be information indicating the estimated value of the bone density of the subject at a third time point. The third time point may be a time point different from the second time point.

[0063] When the image generating section 424 generates an image using the estimated information 434, the image generating section 424 may generate a display image that displays the estimated information 434 associated with the third time point.

[0064] The second generating unit 427 may generate the inferred information 434 in response to an instruction to generate inferred information 434 indicating an inference regarding the bones of the subject, received from the operation receiving unit 423. The second generating unit 427 stores the generated inferred information 434 in the storage unit 43. The second generating unit 427 also includes an inferred model 4271.

[0065] <Estimation Model 4271> The estimation model 4271 is a model that estimates the bone density of a subject at a time point different from the time point at which the medical image indicated by the subject's medical image data 432 was captured, based on a medical image of the subject. The estimation model 4271 may be set based on training data including a medical image (third image) showing the bones of a second person and teacher data including the bone density of the second person. The above-mentioned "second person" does not particularly need to be limited to a human. For example, it may be an animal of the same species as the "subject."

[0066] For example, the bone density of a person whose bones appear in the training plain X-ray images shown by the training image data is associated as training data with multiple training image data. Each of the multiple training image data may be associated with a training data, or multiple sets of training image data may be associated with one training data. The training data may be bone density obtained using the DXA method, the ultrasound method, or the MD method. Alternatively, the training data may be a value estimated or predicted from another trained model.

[0067] The training data can be measured at any time. The training data can be measured at the same time as the training image data was captured, or on a different time axis. For example, a predetermined criterion for the time when the training image data was captured can be set for the training data measurement. The predetermined criterion can be, for example, within three months, six months, or one year from the time when the training image data was captured. The predetermined criterion for the training data can also be at least one of the following: whether or not the subject in the training data has received treatment, age, menopause, and gender. The treatment can be at least one of medication, nutritional guidance (including regular use of supplements), and exercise guidance.

[0068] The training image data of the training data may be a series of image data of the same person taken on different time axes. That is, the training image data may include the following first training data and second training data. The first training data may be training data including X-ray images of the bones of a certain person. The second training data may be training data including X-ray images of the same person as the certain person but taken after the first training data.

[0069] The training image data of the training data may be a group of data obtained by photographing the same body part of different people but with different ages, etc. Also, the training image data of the training data may be a series of image data obtained by photographing the same person on different time axes. The training image data of the training data may be a series of data obtained by photographing the same body part on different time axes.

[0070] The estimated information 434 generated by the second generator 427 may be information about the bone density of the subject when the subject has undergone treatment for bone density. In this case, the second person described above is a person who has undergone treatment for bone density, and the estimated model 4271 may be set based on training data including medical images depicting the bones of the second person who has undergone treatment and teacher data including the bone density value of the second person. For example, the training data may include medical images before and after the treatment. Furthermore, the teacher data may include measurement values ​​of bone density before and after the treatment.

[0071] The second generation unit 427 may include the following two models as the estimation model 4271: (1) A model set based on training data including medical images showing the bones of a second person who has received treatment and teacher data including measured values ​​of the bone density of the second person who has received treatment. (2) A model set based on training data including medical images showing the bones of a second person who has not received treatment and teacher data including measured values ​​of the bone density of the second person who has not received treatment. The treatment may include medication (also referred to as drug treatment, for example), nutritional guidance, exercise guidance, etc.

[0072] The second generating unit 427 may generate inferred information 434 according to the timing at which treatment of the subject will be started.

[0073] Furthermore, when the medical image represented by the medical image data 432 used to estimate the subject's bone density is an image at a first time point, the third time point may be a time point earlier than the first time point or a time point later than the first time point. Furthermore, when the medical image represented by the medical image data 432 used to estimate the subject's bone density is an image at a second time point, the third time point may be a time point earlier than the second time point or a time point later than the second time point.

[0074] The medical image (second image) showing the first human bone, which is the training data for the estimated model 4221, and the medical image (third image) showing the second human bone, which is the training data for the predicted model 4271, may be the same image.

[0075] The second image and the third image may be medical images of different body parts. The person captured in the second image and the person captured in the third image may be the same person. Furthermore, the person captured in the second image and the person captured in the third image may be different people.

[0076] The training data including the bone density measurement values ​​of the second person may be data measured at a time different from the time the third image was captured. For example, the training data may be at least one of data six months, one year, several years, and several years after the third image was captured. Furthermore, the training data may be bone density measurement values ​​measured during a predetermined period including the time the third image was captured.

[0077] <Storage unit 43> The storage unit 43 stores the measurement information 431, medical image data 432, estimation information 433, and inference information 434 described above. The storage unit 43 may also store subject data indicating information about the subject. The subject data may include data indicating whether or not the subject received treatment, the duration of the treatment given to the subject, the details of the treatment given to the subject, the date and time when the subject was injured, etc.

[0078] <Operation Input Unit 44> The operation input unit 44 accepts operation input from the user. For example, the operation input unit 44 may be integrated with the display unit 45. The display unit 45 may display instructions for the processing performed by the control unit 42, such as the processing for estimating the subject's bone density, the processing for generating an image, and the processing for displaying an image. A configuration may be adopted in which the processing corresponding to one of these display areas is executed when the user selects one of these display areas.

[0079] <Display Unit 45> The display unit 45 displays to the user the image generated by the image generation unit 424. Examples of the display unit 45 include a display and a monitor.

[0080] (Flow of display processing) Next, the flow of display processing in the display device 4 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the flow of display processing in the display device 4. As shown in Fig. 2, when the operation receiving unit 423 receives an operation input instructing the display of a display image that displays information about the bones of the subject (YES in S1), the information acquisition unit 425 performs the following processing.

[0081] The information acquisition unit 425 acquires measurement information 431 (S2: first acquisition step). The information acquisition unit 425 also acquires estimated information 433 (S3: second acquisition step). For example, the first information may be information indicating the subject's bone density measured at a first time point, and the second information may be information indicating bone density estimated by an estimation model from a first image depicting the subject's bones at a second time point different from the first time point. Next, the image generation unit 424 generates a display image displaying the measurement values ​​indicated by the measurement information 431 and the estimated values ​​indicated by the estimation information 433 (S4: generation step). Next, the display control unit 426 causes the display unit 45 to display the display image generated by the image generation unit 424 (S5).

[0082] When the operation receiving unit 423 receives an operation input instructing an estimation regarding the subject's bones (YES in S6), the information acquiring unit 425 acquires the estimation information 434 (S7). The operation input may include an input indicating a base point of the estimation and an input indicating a time from the base point to which the estimation is made. For example, without limitation, examples include an estimation six months from the base point, an estimation one year from the base point, an estimation six months prior to the base point, and an estimation one year prior to the base point. The operation input may also include an input regarding whether or not the subject has received treatment. For example, examples of the estimation include an estimation when the subject has received treatment, an estimation when no treatment has been performed, or an estimation of both cases where treatment has been performed and where no treatment has been performed. The operation input may also include an input regarding the timing of the start of treatment for the subject. For example, the estimation may be based on the timing of the start of treatment for the subject. Alternatively, if treatment has been performed, multiple estimations may be made depending on the content of the treatment. For example, the prediction may include at least one of prediction of drug therapy, prediction of nutritional guidance therapy, and prediction of exercise guidance therapy.

[0083] Next, the image generating unit 424 generates a display image that displays the estimated value indicated by the estimation information 434 (S8), and the process returns to S5.

[0084] If the operation receiving unit 423 does not receive an operation input instructing a guess regarding the subject's bones (NO in S6) but receives an operation input instructing to end the display (YES in S9), the processing ends.

[0085] If the operation receiving unit 423 has not received an operation input instructing the display of a display image that displays information about the bones of the subject (NO in S1), the process of S1 is repeated.

[0086] If the operation receiving unit 423 does not receive an operation input instructing a guess regarding the subject's bones (NO in S6) or an operation input instructing the end of display (NO in S9), the process returns to S6.

[0087] (Examples of Display Images Generated by Image Generator 424) Next, examples of display images generated by the image generator 424 will be described with reference to Fig. 3 to Fig. 9. Furthermore, the term "bone density" shown in Fig. 3 to Fig. 9 may be appropriately interpreted as at least one of "bone density," "bone mass," and "bone quality."

[0088] <<Example of a display image displaying an estimated value at a time point after the time point serving as the base point of estimation>> First, an example of a display image displaying an estimated value at a time point after the time point serving as the base point of estimation will be described. Figures 3 to 5 are diagrams showing an example of a display image displaying an estimated value of the subject's bone mineral density at a time point after the time point serving as the base point of estimation.

[0089] 3 shows the subject's bone density measurement values ​​m1-m3 at the time of measurement, the subject's bone density estimate values ​​n1-n2 at the time the medical image was taken, and a bone density transition T1 based on the measurement values ​​m1-m3 and the estimate values ​​n1-n2. Image D1 may also display the name of the facility where the medical image was taken.

[0090] The display mode of the subject's measured bone mineral density may be different from the display mode of the subject's estimated bone mineral density, for example, the measured value may be displayed as an open circle, or the estimated value may be displayed as a black circle.

[0091] The image D1 displays the estimated values ​​o1 and o2. The measured values, estimated values, and estimated values ​​may be displayed in different display modes. In the example shown in FIG. 3, the estimated values ​​o1 and o2 are shown as open triangles. The estimated values ​​are connected by dotted lines, and the estimated values ​​are shown at the end points of the dotted lines. The start points of the dotted lines indicate the time points that are the base points of the estimation.

[0092] In image D1, the estimated value o1 is an estimated value based on December 21, 2021. The estimated value o1 is an estimated value estimated from a medical image captured on December 21, 2021. The estimated value o1 is also an estimated value of the subject's bone density as of February 21, 2022, two months after the base point.

[0093] The estimated value o2 is an estimated value based on the present time and is an estimated value estimated from medical images taken at the present time. The estimated value o2 is an estimated value of the subject's bone density six months from the present time.

[0094] When a bone density measurement of a subject is performed at a certain time, the measurement value at that time displayed in image D1 is the measurement value obtained by the measurement and indicated by the corresponding measurement information 431. On the other hand, when the same time as that time is used as the base point of estimation, the estimated value using that time as the base point of estimation displayed in image D1 is the estimated value estimated from the medical image captured at that time.

[0095] Each measured value, estimated value, and estimated value displays the type of value. Specifically, each measured value, estimated value, and estimated value may be labeled as "measured value," "estimated value," or "estimated value." The "measured value," "estimated value," and "estimated value" may be displayed in different display modes. By using different display modes in this way, users can easily recognize that the values ​​are based on different techniques. The different display modes may be represented, for example, by different character sizes, fonts, or colors. Furthermore, the "measured value," "estimated value," and "estimated value" may be displayed in different display modes, such as icons, abbreviations, and markers. Furthermore, the image D1 may display information regarding the technique used to obtain the displayed value. Examples of such techniques include "measurement," "estimation," and "estimation." That is, the image generation unit 424 may generate a display image including an indication of the first technique and the second technique. By displaying the techniques in this way, user convenience can be improved. Image D1 also displays the YAM (Young Adult Mean) ratio, which is a value corresponding to bone mineral density. In image D1, the YAM ratio is a value indicating a diagnostic standard, and the region where the YAM ratio is between 100% and 80% is labeled "normal," the region where the YAM ratio is between 80% and 70% is labeled "bone mass reduction," and the region where the YAM ratio is lower than 70% is labeled "osteoporosis."

[0096] Image D1 shows bone mineral density per unit area (g / cm ２), bone mineral density per unit volume (g / cm ３ At least one of AGE, T-score, and Z-score may be displayed. For example, AGE may be displayed as a value compared with the average bone density of people of the same age or age group. Image D1 may indicate categories based on these values ​​according to predetermined criteria. Categories may be "normal," "bone loss," and "osteoporosis" according to these values. For example, when T-scores are displayed in image D1, a T-score of -1.0 or higher may be displayed as normal bone density, a T-score of -1.0 to -2.5 may be displayed as low bone density or osteopenia, and a T-score of -2.5 or lower may be displayed as osteoporosis. Furthermore, when Z-scores are displayed in image D1, a Z-score lower than -2.0 may be displayed as being lower than appropriate for age, and a Z-score higher than -2.0 may be displayed as being higher than appropriate for age.

[0097] The second generating unit 427 may predict when the subject's bone mineral density will reach "normal," "low bone mineral density or osteopenia," or "osteoporosis" in terms of the YAM ratio, T-score, or Z-score. That is, the second generating unit 427 may generate predicted information 434 indicating the time. As an example, the predicted information 434 indicates that the subject's bone mineral density will reach a YAM ratio of 70% or less (osteoporosis) in one year, etc.

[0098] That is, the image generating unit 424 generates a display image showing information indicating criteria for bone density. The criteria may be information indicating criteria according to the age of the subject. The criteria may also be information indicating original criteria or criteria based on already known guidelines.

[0099] <Example 1 of Displaying Estimated Values ​​If a Subject Undergoes Treatment> Next, an example of a display image displaying an estimated value of a subject's bone density if the subject receives treatment will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of a display image displaying an estimated value of a subject's bone density if the subject receives treatment. Image D2 shown in FIG. 4 shows bone density measurement values ​​m1-m3 at the time of measurement, estimated bone density values ​​n1-n2 of the subject at the time the medical image was taken, and a bone density transition T1 based on these measurement values ​​and estimated values. In image D2, the same displays as those described for image D1 above are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0100] Image D2 displays estimated values ​​o1, o2, and o3. The estimated values ​​o1 and o2 displayed in image D2 are estimated values ​​when the subject does not receive treatment.

[0101] The estimated value o3 is an estimated value when the subject receives treatment. Image D2 shows the display of "drug therapy" at the current time. This display indicates that "drug therapy" will start from now. In other words, the estimated value o3 is an estimated value six months from now when the treatment "drug therapy" is started for the subject from now.

[0102] In the example shown in Figure 4, the estimated values ​​o1 and o2, which are estimated values ​​when the subject is not receiving treatment, are indicated by open triangles, and the estimated value o3, which is the estimated value when the subject is receiving treatment, is indicated by a black triangle.

[0103] According to the above configuration, the effectiveness of the treatment can be visually recognized by the user.

[0104] In this example, an example of a display image displaying an estimated value six months from now when a "drug therapy" treatment is initiated on a subject from the present has been described. Alternatively, the image may display an estimated value of the subject's bone mineral density when the subject is treated at a time point earlier than the present, i.e., a time point in the past. Furthermore, in the drug therapy, the display image may display estimated values ​​for multiple drugs. More specifically, for example, the display image may display estimated values ​​for each of multiple drugs (e.g., three types) used in the drug therapy. In this case, the display image may display estimated values ​​for each of the top few (e.g., three types) of drugs with the highest therapeutic effects from a large number of drugs (e.g., ten types).

[0105] <Example 2 of Displaying Estimated Values ​​If Subject Receives Treatment: Displaying Treatment Period> Next, another example of a display image displaying an estimated value of a subject's bone density if the subject receives treatment will be described with reference to FIG. 5 . FIG. 5 is a diagram showing another example of a display image displaying an estimated value of a subject's bone density if the subject receives treatment. Image D3 shown in FIG. 5 displays the bone density measurement value m1 at the time of measurement, measurement values ​​m11-m12, the subject's bone density estimates n1-n2 at the time the medical image was captured, and a bone density transition T2 based on these measurement values ​​and estimates. In image D3, the same displays as those described above for images D1-D2 are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0106] Image D3 displays a display p1 indicating the period of treatment actually performed on the subject. Display p1 indicates the period of treatment from February 21, 2022 to the present. That is, the image generation unit 424 may generate a display image displaying information indicating the period of bone density treatment for the subject. Image D3 may also display information indicating the content of the treatment. Image D3 may also display a display p2 indicating the date and time the subject was injured. The image generation unit 424 may generate a display image displaying display p1, display p2, and other information about the subject, using subject data stored in the storage unit 43. Regarding the display of display p1 indicating the period of treatment and display p2 indicating the date and time the subject was injured, the display device 4 may accept an operation input selecting whether or not the display is necessary.

[0107] Image D3 displays estimated values ​​o11, o12, and o13. Estimated value o11 is an estimated value if the subject received treatment, with the start of treatment on February 21, 2021 as the base point of estimation. Estimated value o11 is an estimated value as of May 13, 2021, a point after the base point.

[0108] The estimated value o12 is an estimated value six months from now, assuming that no further treatment is given to the subject.

[0109] The estimated value o13 is an estimated value six months from now, with the present as the starting point for estimation, if treatment for the subject continues.

[0110] <<Example of a display image displaying an estimated value at a time point before the time point that serves as the base point of estimation>> Next, an example of a display image displaying an estimated value at a time point before the time point that serves as the base point of estimation will be described. Figures 6 and 7 are diagrams showing an example of a display image displaying an estimated value of the subject's bone mineral density at a time point before the time point that serves as the base point of estimation.

[0111] <Example of a display image displaying estimated values ​​of bone density in the past, estimated from the present as the base point of estimation> Image D4 shown in Fig. 6 shows the subject's bone density measurement value m1 at the time of measurement, the subject's bone density estimates n1, n2, and n21 at the time the medical image was taken, and a bone density transition T3 based on these measurement and estimate values. Image D4 also displays estimated values ​​o21, o22, and o23. In image D4 shown in Fig. 6, the same elements as those described in images D1 to D3 above are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0112] The estimated value o21 is an estimated value as of May 13, 2022, which is a point after February 21, 2022, which is the base point of the estimation.

[0113] The estimated value o22 is an estimated value for a point in time six months from the current point in time.

[0114] The estimated value o23 is an estimated value at May 13, 2021, which is the point in time before the present, with the present being the base point of the estimation. In other words, the estimated value o23 is an estimated value at a point in time earlier than the present, which is the base point of the estimation.

[0115] <Example of a display image displaying an estimated bone density value prior to the base point, where the base point of the estimation is a past time point> Image D5 shown in Fig. 7 shows the subject's bone density measurement value m1 and measurement value m31 at the time of measurement, the subject's bone density estimate value n31 at the time the medical image was taken, and a bone density transition T4 based on these measurement and estimate values. Image D5 also displays estimated values ​​o31, o32, and o33. In image D5 shown in Fig. 7, the same elements as those described in images D1 to D4 above are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0116] The estimated value o31 is an estimated value as of December 21, 2021, which is a point in time before the base point of estimation, with February 21, 2022 as the base point of estimation. In other words, the estimated value o31 is an estimated value as of a point in time even further in the past than the past point in time that serves as the base point of estimation.

[0117] The estimated value o32 is an estimated value as of May 13, 2022, a point in time prior to the present, with the present as the base point of the estimation.

[0118] The estimated value o33 is an estimated value for a point in time six months from now, with the present as the base point of estimation.

[0119] 6 and 7, it is possible to compensate for the bone density value of a subject at a time point in the past when no medical images were taken or no bone density measurement was performed. Furthermore, it is possible to estimate the bone density value of a subject at a time point in the past at the time of the subject's first consultation at a medical institution.

[0120] <<Example of a display image displaying transition information showing a transition of bone density of a referent different from the subject>> Next, an example of a display image displaying information related to the bone density of the subject and transition information showing a transition of bone density of a referent different from the subject will be described. Fig. 8 is a diagram showing an example of a display image displaying information related to the bone density of the subject and transition information showing a transition of bone density of the referent.

[0121] Image D6 shown in FIG. 8 displays the subject's bone density measurements m41-m43 at the time of measurement, the subject's bone density estimates n41-n42 at the time the medical image was taken, and the subject's bone density over time T5 based on these measurements and estimates. Image D6 also displays an estimated value o41. The estimated value o41 is an estimate of the subject's bone density six months from the current point in time, with the current point being the base point of the estimation. In image D6 shown in FIG. 8, the same displays as those described in images D1-D5 above are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0122] Image D6 displays display p3 indicating the change in bone density of a referee different from the subject. That is, the image generation unit 424 generates image D6 displaying change information indicating the change in bone density of a referee different from the subject. For example, the storage unit 43 may store referee data indicating the change in bone density of the referee. The image generation unit 424 may use the referee data stored in the storage unit 43 to generate a display image displaying display p3, which is information indicating the change in bone density of the referee.

[0123] For example, the reference person may be a person with the same attributes as the subject. Here, the same attributes may mean that at least one of age, weight, sex, disease, blood relatives, type of osteoporosis, blood marker value, and assessment by the Fracture Risk Assessment Tool (FRAX (registered trademark)) is equivalent or similar. The transition in bone density of the reference person may be used as a comparison target for the transition in bone density of the subject. For example, the reference person may be a person who is already receiving treatment for bone density, or may be a person who is not receiving treatment for bone density.

[0124] <Example of a display image displaying an estimated value according to a change in the timing of starting treatment for a subject> Next, an example of a display image displaying an estimated value according to a change in the timing of starting treatment for a subject will be described.

[0125] 9 is a diagram showing an example of a display image that transitions in response to a change in the timing of treatment start for a subject. In this example, a transition of images is described when the timing of treatment start for a subject is changed from six months from now to the present by a user's operation input. As shown in FIG. 9 , when this change is made, the displayed image transitions from image D7 to image D8. The timing of treatment start may also be set to any time by a user's operation input.

[0126] Image D7 is an example of a display image displaying an estimated value of the subject's bone density when treatment is initiated for the subject six months from now. Image D7 displays the subject's bone density measurement values ​​m51-m52 at the time of measurement, the subject's bone density estimates n51-n52 at the time the medical image was taken, and the subject's bone density transition T6 based on these measurement values ​​and estimates. Image D7 also displays estimated values ​​o51, o52, and o53. In image D7, the same displays as those described above for images D1-D6 are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0127] The estimated value o51 is an estimated value of the subject's bone density six months from now, with the present as the base point of estimation.

[0128] The estimated value o52 is the estimated value of the subject's bone mineral density one year from now, assuming that treatment for the subject is initiated six months from now. The estimated value o52, the estimated value o53 described below, the estimated value o62 of image D8, and the estimated value o64 of image D8 are indicated at the end points of the connecting dotted lines, and the starting points of the dotted lines are connected to the estimated value of the subject's bone mineral density six months from now. The starting points of the dotted lines connecting these estimated values ​​do not indicate the starting points of the estimation.

[0129] The estimated value o53 is an estimated value of the subject's bone density one year from now, assuming that treatment for the subject is not initiated, with the present as the starting point of the estimation.

[0130] Image D8 is an example of a display image displaying an estimated value of the subject's bone mineral density when treatment is currently initiated for the subject. In image D8, the same elements as those described above for images D1 to D7 are denoted by the same reference numerals, and their descriptions will not be repeated here. Image D8 displays estimated values ​​o51, o52, o63, and o64.

[0131] The estimated value o61 is an estimated value of the subject's bone density six months from now, assuming that treatment is not initiated for the subject, with the present as the starting point of the estimation.

[0132] The estimated value o62 is an estimated value of the subject's bone density one year from now, assuming that treatment is not initiated for the subject, with the present as the starting point of the estimation.

[0133] The estimated value o63 is an estimated value of the subject's bone density six months from now, assuming that treatment for the subject has started at the present time.

[0134] The estimated value o54 is the estimated bone density of the subject one year from now, assuming that treatment for the subject has started at the present time.

[0135] In this example, the example has been described in which the displayed image transitions from image D7 to image D8 in response to a change in the timing of starting treatment, but image D7 and image D8 may be displayed simultaneously.

[0136] A single image, for example, a single graph, may display each of the estimated values ​​corresponding to different timings for starting treatment.

[0137] (Another Example) Next, another example of the above-mentioned images D1 to D8 will be described.

[0138] (Example of Displaying Names of Bones Appearing in Medical Images Used to Estimate Estimated Values) Images D1 to D8 may display the names of bones appearing in the medical images used to estimate each estimated value. That is, the image generation unit 424 may generate a display image displaying information indicating the names of bones appearing in the medical images used to estimate the estimated information 433. Furthermore, the first generation unit 422 may generate the estimated information 433 from images of bones at multiple locations appearing in the medical images. The image generation unit 424 may generate a display image displaying information indicating the names of multiple bones appearing in the medical images used to estimate the estimated information 433. The bone names may be, for example, at least one of the skull, clavicle, sternum, scapula, ribs, humerus, radius, ulna, hand bones, spine, sacrum, coccyx, femur, patella, tibia, fibula, and foot bones.

[0139] Images D1 to D8 may display the names of bones appearing in the medical images used to estimate each estimated value. That is, the image generation unit 424 may generate a display image displaying information indicating the names of bones appearing in the medical images used to estimate the estimated information 434. Furthermore, the second generation unit 427 may generate the estimated information 434 from images of bones at multiple locations appearing in the medical images. The image generation unit 424 may generate a display image displaying information indicating the names of multiple bones appearing in the medical images used to estimate the estimated information 434.

[0140] (Example of displaying estimated values ​​and medical images used for estimating the estimated values) Images D1 to D8 may display the estimated values ​​and medical images used for estimating the estimated values. The medical images may be, for example, medical images of the head, neck, chest, lumbar region (lumbar vertebrae (lumbar): L1-L4, etc.), hip joints, knee joints, ankle joints, feet, toes, shoulder joints, elbow joints, wrist joints, hands, fingers, or temporomandibular joints. The types of captured regions of the medical images are not limited to these. Furthermore, the medical images may be frontal images in which the target region irradiated with plain X-rays is viewed from the front, or lateral images in which the target region is viewed from the side. The X-ray images may be images of at least one of cortical bone and cancellous bone.

[0141] (Example of Display When Bone Density is Estimated or Predicted from Each of Multiple Regions) When bone density is estimated or presumed from each of multiple regions captured in a medical image, images D1 to D8 may display respective graphs showing the progression of bone density at each region. For example, the captured images of each region used to estimate each estimated value and to estimate each estimated value may be superimposed on the corresponding graph. For example, the multiple regions may be set as regions for each vertebra. For example, the multiple regions may be set as regions that combine multiple vertebrae. For example, the multiple regions may be set as regions with different areas. Furthermore, the multiple regions may be set as regions obtained by dividing the area of ​​a specific portion of a certain region. The multiple regions may be set as regions arbitrarily based on, for example, the subject's attribute information. For example, if the subject has a history of fractures, the specific portion may be set as a smaller area than a subject without a history of fractures.

[0142] (Example of display of assessment criteria) In a display image showing information indicating assessment criteria such as normal, bone mass loss, and osteoporosis, the assessment criteria may be superimposed on the graph. The assessment criteria may be based on the subject's age at the time of measurement, the subject's age at the time of estimation, or the subject's age at the time of prediction. The time of prediction here may be, for example, the third time point described above.

[0143] (Example of Displaying Error Range or Estimation Probability of Estimated Value) The second generator 427 may generate estimation information 434 including information indicating an error range for the estimated value. The image generator 424 may generate a display image displaying information indicating the error range for each estimated value.

[0144] Specifically, the second generator 427 may generate the estimation information 434 including information indicating the estimation probability of the estimated value or the error range of the estimated value. The estimation probability may be, for example, a predicted probability. At least one of the estimation probability and the error range may be displayed in the above-described images D1 to D8.

[0145] (Example of Displaying Error Range of Estimated Value) Here, an example of a display image displaying the error range of an estimated value will be described. FIG. 10 is a diagram showing an example of a display image displaying the error range of the estimated value of the subject's bone density. Image D9 shown in FIG. 10 displays the measured bone density values ​​m71-m72 of the subject at the time of measurement, the estimated bone density values ​​n71-n72 of the subject at the time the medical image was taken, and a transition T7 of the subject's bone density based on these measured and estimated values. Image D9 also displays the estimated values ​​o71 and o72.

[0146] The estimated value o71 is an estimated value of the subject's bone mineral density for a predetermined period including a point six months from the current point of time. The estimated value o72 is an estimated value of the subject's bone mineral density for a predetermined period including a point one year from the current point of time. In other words, the estimated values ​​o71 and o72 may be displayed as estimated values ​​of the subject's bone mineral density for a predetermined period including a point a predetermined period after the current point of time.

[0147] In the example shown in Fig. 10, the estimated values ​​o71 and o72 are represented by circles that include the estimated values ​​and their error ranges. The error ranges of the estimated values ​​o71 and o72 may also be displayed using error bars or box-and-whisker plots. The upper limits of the error ranges of the estimated values ​​o71 and o72 may also be connected by a dotted line or the like. The lower limits of the error ranges of the estimated values ​​o71 and o72 may also be connected by a dotted line or the like.

[0148] (Example in which the estimated value is a range) The first generating unit 422 may generate the estimation information 433 including information indicating a range for each estimated value. The image generating unit 424 may generate a display image that displays the range for each estimated value. A specific example of the range for the estimated value is an estimated bone density range of 0.79-0.81 (median 0.80) g / cm. ２ The following can be cited as an example.

[0149] (Example of Displaying Estimated Probability of Estimated Value) Next, an example of a display image displaying the estimated probability of an estimated value will be described. FIG. 11 is a diagram showing an example of a display image displaying the estimated probability of an estimated value of a subject's bone density. Image D10 shown in FIG. 11 displays measured values ​​m81-m83 of the subject's bone density at the time of measurement, estimated values ​​n81-n82 of the subject's bone density at the time the medical image was taken, and a transition T8 of the subject's bone density based on these measured values ​​and estimated values. Image D10 also displays an estimated value o82.

[0150] The estimated value o82 is an estimated value of the subject's bone mineral density six months from the current point of estimation. Furthermore, in image D10, above the estimated value o82, a display p4 indicating the estimation probability of the estimated value o82 is displayed as "69%." That is, the second generator 427 may generate estimation information 434 indicating the estimation probability of the estimated value. Furthermore, the image generator 424 may generate a display image displaying the estimation probability for each estimated value.

[0151] (Example of Displaying Measurement Results When Measuring Measurement Values) Images D1 to D10 may further display medical images captured when each measurement value was measured. The medical images may be, for example, medical images of the head, neck, chest, lumbar region (lumbar vertebrae (Lumbar): L1-L4, etc.), hip joints, knee joints, ankle joints, feet, toes, shoulder joints, elbow joints, wrist joints, hands, fingers, or temporomandibular joints. The types of captured regions of the medical images are not limited to these. Furthermore, the medical images may be frontal images in which the target region irradiated with plain X-rays is viewed from the front, or lateral images in which the target region is viewed from the side. The X-ray images may be images of at least one of cortical bone and cancellous bone. Furthermore, images D1 to D10 may display measurement results corresponding to the displayed measurement values. For example, the displayed measurement results may be displays showing measurement results obtained by DXA.

[0152] (Another example of displaying estimated values ​​and medical images used to estimate the estimated values) In images D1 to D10, if the estimated values ​​and medical images used to estimate the estimated values ​​are further displayed, the medical images may be displayed so that the areas used for the estimation can be seen.

[0153] For example, the first generating unit 422 may generate a heat map for the region used to estimate the estimated value. In this case, for example, the outer edge of the heat map may indicate the segmentation region of the region used for estimation. The heat map may represent the magnitude of bone density using any color intensity. For example, the first generating unit 422 may generate a heat map indicating the degree of attention. Alternatively, the first generating unit 422 may generate a heat map indicating the numerical value of bone density. Alternatively, the first generating unit 422 may generate a heat map indicating the possibility (probability) of a fracture. The display image may display an image in which the heat map is superimposed on a medical image. The image used for the heat map may be a still image or a video. Displaying the heat map as a video makes it easier to visually recognize the relationship between the heat maps, for example, by fading various heat maps in sequence. Furthermore, if the analysis result is a bone density heat map that includes regions other than the segmentation region, some of the segmentation regions may be surrounded by a frame.

[0154] (Method of Displaying Medical Images and Measurement Results) In the case where the estimation of each estimated value and the medical images used to estimate each estimated value are further displayed in images D1 to D10, when a mouse cursor or the like is placed over the display area for the estimated value or the display area for the estimated value, a medical image corresponding to the estimated value or the estimated value may be displayed. Here, the corresponding medical image may be the medical image used to estimate the estimated value or the estimated value. For example, the medical image may be displayed so that the area in the medical image used to estimate the estimated value or the estimated value can be seen. Furthermore, when a mouse cursor or the like is placed over the display area for the measured value, the measurement result corresponding to the measured value may be displayed. For example, the displayed measurement result may be a display showing the measurement result measured by the DXA method. The display of these medical images or the measurement results may be displayed in a balloon from the display area for the measured value or the estimated value where the mouse cursor or the like is placed. Furthermore, when a click operation is performed on the display area for the estimated value, the estimated value, or the measured value, a display image including the above-mentioned medical image or the measurement result may be displayed.

[0155] An example of a display image displaying medical images corresponding to estimated values ​​and measured values ​​will be described. FIG. 12 is a diagram showing an example of a display image displaying medical images used to estimate estimated values ​​and measurement results corresponding to measured values. In FIG. 12, the same displays as those described in FIG. 3 above are denoted by the same reference numerals, and their description will not be repeated here. Image D11 shown in FIG. 12 includes display P6 showing a graph of bone density progression T1, display P7 displaying a medical image corresponding to estimated value n2, and display P8 displaying a measurement result corresponding to measured value m3. As shown in FIG. 12, when a mouse cursor P5a is placed over the display area of ​​estimated value n2, image D11 may display display P7 displaying a medical image corresponding to estimated value n2. For example, when the operation accepting unit 423 accepts an operation of placing the mouse cursor P5a over the display area of ​​estimated value n2, the image generating unit 424 generates a display image including display P7, and the display control unit 426 displays the display image on the display unit 45. Furthermore, as shown in FIG. 12, when a mouse cursor P5b is placed over the display area of ​​a measurement value m3, a display P8 of the measurement result corresponding to the measurement value m3 may be displayed.

[0156] 13 is a diagram showing another example of the display of a medical image corresponding to the estimated value n2. In this example, when the mouse cursor P5a is placed over the display area of ​​the estimated value n2 shown in FIG. 12, a display P7a of the medical image corresponding to the estimated value n2 is displayed. A display P710 is displayed in the display P7a as a frame indicating the area in the medical image used to estimate the estimated value n2.

[0157] (Example of Displaying Changes in Bone Density at Each of Multiple Locations) The display image displayed on the display unit 45 may display changes in bone density at each of multiple locations captured in a medical image. An example of a display image displaying changes in bone density at each of multiple locations will be described. FIG. 14 is a diagram showing an example of a display image displaying changes in bone density at each of multiple locations. In FIG. 14, the same displays as those described in FIG. 3 above are denoted with the same reference numerals, and their descriptions will not be repeated here. Image D12 shown in FIG. 14 includes display P11 showing a graph indicating changes in bone density T1 and display P9, which is a display of a medical image. Display P9 of the medical image displays displays P91 to P94, which are borders indicating the areas in which each of the lumbar vertebrae L1-L4 is imaged. For example, as shown in FIG. 14, when a mouse cursor P10 is positioned within display P91, which is the display area for lumbar vertebra L1, an image showing changes in bone density at lumbar vertebra L1 may be displayed. Furthermore, a display image simultaneously displaying changes in bone density at each of the lumbar vertebrae L1-L4 may also be displayed. In such a display image, for example, when a mouse cursor P10 is positioned within a display P91 that is a display area for the lumbar vertebra L1, the transition of bone density in the lumbar vertebra L1 may be displayed in an emphasized manner.

[0158] (Example of displaying a predicted location of a fracture as a prediction result) The operation accepting unit 423 may accept an operation input instructing a prediction regarding the subject's fracture. Upon accepting the operation input, the operation accepting unit 423 instructs the second generating unit 427 to generate prediction information 434 indicating a prediction regarding the subject's fracture. Furthermore, the operation accepting unit 423 instructs the image generating unit 424 to generate a display image displaying a prediction regarding the subject's fracture indicated by the prediction information 434. The operation accepting unit 423 accepts an operation input instructing to end the display. Upon accepting the operation input, the operation accepting unit 423 instructs the display control unit 426 to end the display.

[0159] <<Another Example of the Second Generator 427>> The second generator 427 included in the control unit 42 may perform the following process. The second generator 427 generates inferred information 434 including information on the possibility of a future fracture in the subject inferred from medical images indicated by medical image data 432 stored in the storage unit 43. For example, the second generator 427 may generate inferred information 434 including information on the possibility of a future fracture inferred from medical images depicting the subject's bones at a first time point. The inferred information 434 may include, for example, at least any of information indicating the probability of a future fracture in the subject, the time when the fracture is estimated to occur, and the location of the fracture.

[0160] The second generator 427 may generate inferred information 434 including information about the subject's fracture inferred from a medical image showing the subject's bones at a second time point. The inferred information 434 may include information about the subject's fracture at a fourth time point. The fourth time point may be a time point different from the second time point and the third time point.

[0161] When the image generating section 424 generates an image using the estimated information 434, the image generating section 424 may generate a display image that displays the estimated information 434 associated with the fourth time point.

[0162] <Another Example of the Estimation Model 4271> The estimation model 4271 included in the second generation unit 427 according to this example may perform the following processing. The estimation model 4271 may estimate, from a medical image indicated by the medical image data 432 of the subject, information regarding the possibility of the subject suffering a future fracture at a time point different from the time point at which the medical image was captured. The estimation model 4271 may be set based on training data including a medical image (fourth image) showing the bones of a third person and training data including information regarding the fracture of the third person. The above-mentioned "third person" does not particularly need to be limited to a human. For example, it may be an animal of the same species as the "subject."

[0163] For example, for each of a plurality of training image data included in the training data, information regarding a fracture of a person having a bone that is shown in the training plain X-ray image shown by the training image data is associated as training data. The training data may be at least one of bone evaluation information and a medical image (fifth image) showing the fractured part of a third person. The fifth image may be an image showing the same part as the fourth image. The fifth image may be an image showing a different part from the fourth image.

[0164] The training data may be measured at a time different from the time when the training image data was captured, or may be measured at approximately the same time when the training image data was captured.

[0165] The estimation model 4271 may be configured by a plurality of models. For example, the estimation model 4271 may be configured by a model that estimates the bone density of a subject at a time point different from the time point at which the medical image was captured from a medical image, or a model that estimates the possibility of a future fracture in the subject at a time point different from the time point at which the medical image was captured from a medical image. The estimation results output from the plurality of models may be displayed together in the display image displayed on the display unit 45, or only one of them may be displayed.

[0166] The estimated information 434 may be a method for displaying different types of information from the first method and the second method. For example, the first method may be a method for displaying information related to bone density, and the second method may be a method for displaying information related to a fracture. For example, the measured value of the subject's bone density and the estimated value of the subject's bone density may be displayed in the same display format in the display image displayed on the display unit 45. Information related to the subject's fracture may be displayed in a display format different from the display format of the measured value of the subject's bone density and the estimated value of the subject's bone density in the display image displayed on the display unit 45.

[0167] The display image displayed on the display unit 45 may display the location where a future fracture is likely to occur, as indicated by the estimated information 434. Fig. 15 is a diagram showing an example of a display indicating a location where a fracture is likely to occur, included in the display image displayed on the display unit 45. In the example shown in the left diagram of Fig. 15, the display image displayed on the display unit 45 includes a display P13 which is a display of an X-ray image. The display P13 includes a display P131 which is a frame line indicating an area where a future fracture is likely to occur.

[0168] In the example shown on the right side of Fig. 15, the display image displayed on the display unit 45 includes display P14, which is a display of an X-ray image. Display P14 includes display P14, which is a frame line indicating an area used to estimate bone density, and displays P141 and P142, which are frames indicating areas where a future fracture is likely to occur. For example, a notation such as "prediction" may be displayed around display P141 and P142, which are frame lines indicating areas where a future fracture is likely to occur.

[0169] The display device 4 may include a user interface that accepts a user operation to switch between displaying and hiding estimated values ​​in the display image. The interface may accept a user operation to select a time point at which estimated values ​​are to be displayed. The interface may also accept a user operation to switch between displaying and hiding medication, nutritional guidance, and exercise guidance in the display image.

[0170] [Embodiment 2] Another embodiment of the present disclosure will be described below with reference to Fig. 16. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0171] Fig. 16 is a block diagram showing an example of the main configuration of an information processing system 1a according to this embodiment. As shown in Fig. 16, the information processing system 1a includes a measuring device 2, an imaging device 3, a display device (image generating device) 4a, a first generating device 4220a, and a second generating device 4270a. Each device communicates with the other devices via wired or wireless communication. Alternatively, the communication may be performed via a network 5a, as shown in Fig. 16.

[0172] The first generation device 4220a performs processing that is the same as or similar to the processing performed by the first generation unit 422 described in embodiment 1. The first generation device 4220a transmits the generated estimated information 433 to the display device 4 via the communication unit 4221a.

[0173] The second generating device 4270a performs processing that is the same as or similar to the processing performed by the second generating unit 427 described in embodiment 1. The second generating device 4270a receives an instruction to generate inferred information 434 via the communication unit 4271a. The second generating device 4270a also transmits the generated inferred information 434 to the display device 4 via the communication unit 4271a.

[0174] The display device 4a includes a communication unit 41a, a control unit 42a, a storage unit 43a, an operation input unit 44, and a display unit 45. The acquisition unit 421a of the control unit 42a receives measurement information 431, estimation information 433, and inferred information 434 via the communication unit 41a. The display device 4a performs processing that is the same as or similar to the image generation processing described in embodiment 1. Furthermore, when the operation reception unit 423a of the display device 4a receives an instruction to receive an operation input instructing an inference regarding the subject's bones, the display device 4a transmits an instruction to generate inferred information 434 to the second generation device 4270a via the communication unit 41a. The operation reception unit 423a accepts operations that are the same as or similar to those of the operation reception unit 423. Furthermore, an operation input device that performs the processing of the operation input unit 44 and the operation reception unit 423a may be included in the information processing system 1a as a device separate from the display device 4a. In this case, the operation input device may communicate with the display device 4a, the second generation device 4270a, etc. via the network 5a.

[0175] [Embodiment 3] Another embodiment of the present disclosure will be described below with reference to Fig. 17. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0176] FIG. 17 is a block diagram showing an example of the essential components of an information processing system 1b according to this embodiment. As shown in FIG. 17, the information processing system 1b includes a measuring device 2, an imaging device 3, a display device 4b, a first generating device 4220a, a second generating device 4270a, and a server device (image generating device) 6b. Each device communicates with the other devices via wired or wireless communication. Alternatively, the communication may be performed via a network 5a, as shown in FIG. 17. The server device 6b may be a cloud server.

[0177] The first generation device 4220a performs processing that is the same as or similar to the processing performed by the first generation unit 422 described in embodiment 1. The first generation device 4220a transmits the generated estimation information 433 to the server device 6b via the communication unit 4221a.

[0178] The second generating device 4270a performs processing that is the same as or similar to the processing performed by the second generating unit 427 described in embodiment 1. The second generating device 4270a receives an instruction to generate inferred information 434 via the communication unit 4271a. The second generating device 4270a also transmits the generated inferred information 434 to the server device 6b via the communication unit 4271a.

[0179] The server device 6b includes a communication unit 61b, a control unit 62b, and a storage unit 63b. An acquisition unit 621b of the control unit 62b stores the measurement information 431, the estimation information 433, and the inference information 434 received via the communication unit 61b in the storage unit 63b. An information acquisition unit 625b and an image generation unit 624b of the control unit 62b perform image generation processing in the same manner as or similar to the information acquisition unit 425 and the image generation unit 424 described in the first embodiment. The server device 6b transmits the generated image data 430b to the display device 4b via the communication unit 61b.

[0180] The display device 4b includes a communication unit 41b, a control unit 42b, a storage unit 43b, an operation input unit 44, and a display unit 45. When the operation reception unit 423b receives an instruction to receive an operation input instructing an estimation regarding the subject's bones, the display device 4b transmits an instruction to generate estimation information 434 to the second generation device 4270a via the communication unit 41b. The operation reception unit 423b receives the same or similar operations as those received by the operation reception unit 423. The acquisition unit 421b of the control unit 42b acquires image data 430b from the server device 6b via the communication unit 41b. The acquisition unit 421b stores the acquired image data 430b in the storage unit 43b and transmits a signal indicating that the image data 430b has been stored to the display control unit 426b. When the display control unit 426b receives the signal from the acquisition unit 421b, it acquires the image data 430b from the storage unit 43b and displays the image represented by the image data 430b on the display unit 45.

[0181] [Embodiment 4] Another embodiment of the present disclosure will be described below with reference to Fig. 18. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0182] 18 is a block diagram showing the main configuration of an example of an information processing system 1c according to this embodiment. As shown in FIG. 18, the information processing system 1c includes a measuring device 2, an imaging device 3, and a display device 4c. Below, only the functions of the display device 4c that are added to the display device 4 described in embodiment 1 will be described. The description of the same functions of the display device 4c as those of the display device 4 will not be repeated.

[0183] The display device 4 c includes a communication unit 41 , a control unit 42 c , a storage unit 43 c , an operation input unit 44 , and a display unit 45 .

[0184] <Control unit 42c> The control unit 42c comprehensively controls each unit of the display device 4. The control unit 42c includes an acquisition unit 421 (first acquisition unit), a first generation unit 422, an operation acceptance unit 423c, an image generation unit (generation unit) 424c, an information acquisition unit (first acquisition unit, second acquisition unit, third acquisition unit, fourth acquisition unit) 425c, a display control unit 426, a second generation unit 427, and a third generation unit 428c.

[0185] The operation reception unit 423c, the image generation unit 424c, and the information acquisition unit 425c each perform the following processes in addition to the processes performed by the operation reception unit 423, the image generation unit 424, and the information acquisition unit 425, respectively.

[0186] <<Operation Receiving Unit 423c>> The operation receiving unit 423c receives an operation input instructing the display of a display image displaying third information, which is information about the subject's bones that is different from the first information and / or second information. Here, the third information may be bone information calculated based on the first information and / or second information. Furthermore, the third information may be bone information different from the first information and / or second information, calculated from the subject's measurement data and / or medical images used to acquire the first information and / or second information. For example, if the first information and / or second information is the subject's bone mineral density, the third information may be the subject's fracture risk information. In this embodiment, a case is described in which the first information and second information are the subject's bone mineral density, and the third information is fracture risk information, but this is not limited thereto. Upon receiving the operation input, the operation receiving unit 423c instructs the third generating unit 428c to generate fracture risk information (third information) 435c indicating the subject's fracture risk. Furthermore, the operation receiving unit 423c instructs the image generating unit 424c to generate a display image that displays the fracture risk information 435c.

[0187] <<Third Generator 428c>> The third generator 428c generates fracture risk information 435c in response to an instruction to generate fracture risk information 435c indicating the subject's fracture risk received from the operation receiver 423c. The third generator 428c stores the generated fracture risk information 435c in the memory unit 43c. For example, the third generator 428c may generate the fracture risk information 435c as follows.

[0188] The third generator 428c generates fracture risk information 435c indicating the subject's fracture risk at the first time point from measurement information 431 indicating the measurement value of the subject's bone density measured at the first time point.

[0189] In addition, the third generation unit 428c generates fracture risk information 435c indicating the subject's fracture risk at the second time point from estimated information 433 indicating an estimated bone density value estimated from a medical image showing the subject's bones at the second time point.

[0190] In addition, the third generator 428c generates fracture risk information 435c indicating the subject's fracture risk at the third time point from the estimated information 434 indicating the estimated value of the subject's bone density at the third time point.

[0191] That is, the third generator 428c generates fracture risk information 435c regarding the subject's bones calculated from at least one of the measurement information 431, the estimation information 433, and the inferred information 434.

[0192] The third generator 428c may calculate the fracture risk from at least one of the measured, estimated, and predicted bone density values ​​using a fracture risk assessment tool, such as FRAX (registered trademark).

[0193] On the other hand, the third generator 438c may generate the fracture risk information 435c of the subject at the first time point from measurement data related to the subject's bones at the first time point measured by the measuring device 2. Furthermore, the third generator 428c may generate the fracture risk information 435c of the subject at the second time point from medical images depicting the subject's bones at the second time point. Furthermore, the third generator 428c may generate the fracture risk information 435c of the subject at the third time point from medical images depicting the subject's bones at the second time point.

[0194] That is, the third generation unit 428c may generate measurement data regarding the subject's bones at a first time point measured by the measuring device 2, or fracture risk information 435c regarding the subject's bones at a second time point calculated from at least one of the medical images showing the subject's bones at the second time point.

[0195] <<Image Generation Unit 424c>> When the image generation unit 424c receives an instruction to generate a display image displaying the fracture risk information 435c, it acquires the measurement information 431, the estimation information 433, and the fracture risk information 435c from the memory unit 43c via the information acquisition unit 425c. The image generation unit 424c generates a display image displaying the subject's fracture risk information 435c using the fracture risk information 435c. The image generation unit 424c transmits data of the generated display image to the display control unit 426. Details of examples of display images generated by the image generation unit 424c will be described later.

[0196] In this embodiment, an example will be described in which the third generator 428c calculates a fracture risk from a bone mineral density value. As another example, the third generator 428c may calculate a YAM (%), a T-score, or a Z-score from a bone mineral density value indicated by at least one of the measurement information 431, the estimation information 433, and the prediction information 434. That is, the third generator 428c may generate information indicating the YAM (%), the T-score, or the Z-score. The image generator 424c may generate a display image displaying the YAM (%), the T-score, or the Z-score calculated from at least one of the measurement information 431, the estimation information 433, and the prediction information 434.

[0197] <Example of Displaying a Subject's Fracture Risk> Next, an example of a display image displaying a subject's fracture risk will be described with reference to FIG. 19. FIG. 19 is a diagram showing an example of a display image displaying a subject's fracture risk. Image D13 shown in FIG. 19 shows bone density measurement values ​​m1-m3 at the time of measurement, estimated bone density values ​​n1-n2 of the subject at the time the medical image was taken, and a bone density transition T1 based on these measurement values ​​and estimated values. Image D13 also displays estimated values ​​o1 and o2. In image D13, the same displays as those described in image D1 above are denoted by the same reference numerals, and their descriptions will not be repeated here.

[0198] Image D13 displays fracture risk q1, fracture risk q2, fracture risk q3, and fracture risk q4, which indicate fracture risk. Fracture risk q1 is a value indicating fracture risk calculated from estimated value n1. Furthermore, fracture risk q2 is a value indicating fracture risk calculated from estimated value n2. Furthermore, fracture risk q3 is a value indicating fracture risk calculated from measured value m3. Furthermore, fracture risk q4 is a value indicating fracture risk calculated from predicted value o2.

[0199] The fracture risk calculated from the measured values, the fracture risk calculated from the estimated values, and the fracture risk calculated from the predicted values ​​may be displayed in different display modes. In the example shown in Figure 19, the fracture risks q1 and q2 calculated from the predicted values ​​are shown as filled circles, the fracture risk q3 calculated from the measured values ​​is shown as a hollow circle, and the fracture risk q4 calculated from the predicted values ​​is shown as a hollow triangle.

[0200] Fifth Embodiment Another embodiment of the present disclosure will be described below with reference to Fig. 20. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0201] FIG. 20 is a block diagram showing an example of the essential components of an information processing system 1d according to this embodiment. As shown in FIG. 20, the information processing system 1d includes a measuring device 2, an imaging device 3, a display device (image generating device) 4d, a first generating device 4220a, a second generating device 4270a, and a third generating device 4280d. Each device communicates with the other devices via wired or wireless communication. Alternatively, as shown in FIG. 20, the communication may be performed via a network 5a. The processes performed by each device included in the information processing system 1d described below may be performed by another device included in the information processing system 1d. In other words, the location of the functions performed by each device included in the information processing system 1d is not limited.

[0202] The first generation device 4220a performs processing that is the same as or similar to the processing performed by the first generation device 4220a described in the second embodiment.

[0203] The second generation device 4270a performs processing that is the same as or similar to the processing performed by the second generation device 4270a described in embodiment 2. The second generation device 4270a receives an instruction to generate inferred information 434 via the communication unit 4271a. The second generation device 4270a also transmits the generated inferred information 434 to the display device 4d via the communication unit 4271a.

[0204] The third generation device 4280d performs processing that is the same as or similar to the processing performed by the third generation unit 428c described in embodiment 4. A storage unit 4283d included in the third generation device 4280d stores measurement information 431, estimation information 433, and inference information 434. For example, the third generation device 4280d acquires these pieces of information via the network 5a.

[0205] The third generating device 4280d receives an instruction to generate the fracture risk information 435c via the communication unit 4281d. The third generating device 4280d also transmits the generated fracture risk information 435c to the display device 4d via the communication unit 4281d.

[0206] The display device 4d includes a communication unit 41d, a control unit 42d, a memory unit 43d, an operation input unit 44, and a display unit 45. The acquisition unit 421d of the control unit 42d receives the measurement information 431, the estimation information 433, the inferred information 434, and the fracture risk information 435c via the communication unit 41d. The display device 4d performs processing that is the same as or similar to the image generation processing described in embodiment 4. Furthermore, when the operation reception unit 423d receives an operation input instructing the display device 4d to display the subject's fracture risk, the display device 4d transmits an instruction to generate the fracture risk information 435c to the third generation device 4280d via the communication unit 41d. The operation reception unit 423d accepts the operation received by the operation reception unit 423c. Furthermore, the operation input device that performs the processing of the operation input unit 44 and the operation reception unit 423c may be included in the information processing system 1d as a device separate from the display device 4d. In this case, the operation input device may communicate with the display device 4d, the second generating device 4270a, the third generating device 4280d, etc. via the network 5a.

[0207] Sixth Embodiment Another embodiment of the present disclosure will be described below with reference to Fig. 21. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0208] FIG. 21 is a block diagram showing the essential components of an example of an information processing system 1e according to this embodiment. As shown in FIG. 21 , the information processing system 1e includes a measuring device 2, an imaging device 3, a display device 4e, a first generating device 4220a, a second generating device 4270a, a third generating device 4270d, and a server device (image generating device) 6e. Each device communicates with the other devices via wired or wireless communication. As shown in FIG. 21 , the communication may be performed via a network 5a. The server device 6e may also be a cloud server. The processes performed by each device included in the information processing system 1e described below may be performed by another device included in the information processing system 1e. In other words, the location of the functions performed by each device included in the information processing system 1e is not limited.

[0209] The first generation device 4220a performs processing that is the same as or similar to the processing performed by the first generation unit 422 described in embodiment 1. The first generation device 4220a transmits the generated estimation information 433 to the server device 6e via the communication unit 4221a.

[0210] The second generating device 4270a performs processing that is the same as or similar to the processing performed by the second generating unit 427 described in embodiment 1. The second generating device 4270a receives an instruction to generate inferred information 434 via the communication unit 4271a. The second generating device 4270a also transmits the generated inferred information 434 to the server device 6e via the communication unit 4271a.

[0211] The third generating device 4280d performs processing that is the same as or similar to the processing performed by the third generating unit 428c described in embodiment 4. The third generating device 4280d receives an instruction to generate fracture risk information 435c via the communication unit 4281d. The third generating device 4280d also transmits the generated fracture risk information 435c to the server device 6e via the communication unit 4281d.

[0212] The server device 6e includes a communication unit 61b, a control unit 62e, and a storage unit 63e. An acquisition unit 621e of the control unit 62e stores the measurement information 431, the estimation information 433, the inference information 434, and the fracture risk information 435c received via the communication unit 61b in the storage unit 63e. An information acquisition unit (fourth acquisition unit) 625e and an image generation unit 624e of the control unit 62e perform processing that is the same as or similar to the image generation processing performed by the information acquisition unit 425c and the image generation unit 424c described in embodiment 4. The server device 6e transmits the generated image data 430b to the display device 4e via the communication unit 61b.

[0213] The display device 4e performs the following processes in addition to the processes performed by the display device 4b. The display device 4e includes a communication unit 41b, a control unit 42e, a storage unit 43b, an operation input unit 44, and a display unit 45. When the operation reception unit 423e receives an operation input instructing the display device 4e to display the subject's fracture risk, the display device 4e transmits an instruction to generate fracture risk information 435c to the third generation device 4280d via the communication unit 41b. The display device 4e also instructs the server device 6e to generate display image data 430b that displays the subject's fracture risk. The operation reception unit 423e receives the operation received by the operation reception unit 423. The acquisition unit 421b of the control unit 42e acquires the image data 430b from the server device 6e via the communication unit 41b. The acquisition unit 421b stores the acquired image data 430b in the storage unit 43b and transmits a signal indicating that the image data 430b has been stored to the display control unit 426b. When the display control unit 426b receives the signal from the acquisition unit 421b, the display control unit 426b acquires the image data 430b from the storage unit 43b and displays the image indicated by the image data 430b on the display unit 45.

[0214] [Example of implementation by software] The functions of the display device (4, 4a, 4b, 4c, 4d, 4e), the first generating device (4220a), the second generating device (4270a), the third generating device (4280d), and the server device (6b, 6e) (hereinafter referred to as "devices") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit).

[0215] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in each of the above embodiments are realized by executing the program using the control device and storage device.

[0216] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0217] In addition, some or all of the functions of each of the control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits that function as each of the control blocks are formed are also included in the scope of the present disclosure. In addition, the functions of each of the control blocks can also be realized by, for example, a quantum computer.

[0218] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0219] The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to the above-described embodiments. In other words, the invention according to the present disclosure can be modified in various ways within the scope of the present disclosure, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the invention according to the present disclosure. In other words, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. It should also be noted that these modifications or corrections are included in the scope of the present disclosure.

[0220] [Summary] The information processing system according to aspect 1 of the present disclosure includes a first acquisition unit that acquires first information indicating information about the bones of a subject obtained based on a first method, a second acquisition unit that acquires second information indicating information about the bones of the subject obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0221] In the information processing system of aspect 2 of the present disclosure, in aspect 1, the first information and the second information may be information regarding at least one of the subject's bone density, bone mass, and bone quality.

[0222] An information processing system according to aspect 3 of the present disclosure may be configured such that, in aspect 2, the first acquisition unit acquires the first information measured at a first time point, the second acquisition unit acquires the second information indicating at least one of bone density, bone mass, and bone quality estimated by an estimation model from a first image showing the subject's bones at a second time point different from the first time point, and the generation unit generates the display image in which the first information and the second information are arranged in chronological order of the first time point and the second time point.

[0223] In an information processing system according to aspect 4 of the present disclosure, in aspect 3, the estimation model may be set based on training data including a second image showing the bones of a first person and teacher data including at least one of bone density, bone mass, and bone quality of the first person.

[0224] The information processing system of aspect 5 of the present disclosure, in aspect 3 or 4, further includes a third acquisition unit that acquires estimated information regarding at least one of bone density, bone mass, and bone quality estimated by an estimation model from an image showing the subject's bones at the first time point or the second time point, and if the image showing the subject's bones is an image showing the subject's bones at the first time point, the estimated information is information at a third time point different from the first time point, and if the image showing the subject's bones is an image showing the subject's bones at the second time point, the estimated information is information at a third time point different from the second time point, and the generation unit may generate the display image displaying the estimated information associated with the third time point.

[0225] In an information processing system according to aspect 6 of the present disclosure, in aspect 5, the prediction model may be set based on training data including a third image showing the bones of a second person and teacher data including at least one of the bone density, bone mass, and bone quality of the second person.

[0226] An information processing system according to aspect 7 of the present disclosure is, in aspect 6, wherein the estimated information is information regarding the bone density of the subject when the subject is treated for at least one of bone density, bone mass, and bone quality, the second person is a person who has received treatment for at least one of bone density, bone mass, and bone quality, and the estimated model may be set based on learning data including the third image showing the bones of the second person who has received the treatment, and teacher data including at least one of bone density, bone mass, and bone quality of the second person.

[0227] In the information processing system of aspect 8 of the present disclosure, in any of aspects 5 to 7, if the image showing the subject's bones is an image showing the subject's bones at the first time point, the third time point may be a time point earlier than the first time point or a time point later than the first time point, and if the image showing the subject's bones is an image showing the subject's bones at the second time point, the third time point may be a time point earlier than the second time point or a time point later than the second time point.

[0228] The information processing system of aspect 9 of the present disclosure, in any of aspects 5 to 8, may further include a fourth acquisition unit that acquires third information regarding the subject's bones calculated from at least one of the first information, the second information, and the estimated information, and the generation unit may generate the display image that displays the third information.

[0229] In the information processing system of aspect 10 of the present disclosure, in any of aspects 3 to 9, the generation unit may generate the display image displaying information indicating the names of bones appearing in the first image used to estimate the second information.

[0230] In an information processing system according to aspect 11 of the present disclosure, in any one of aspects 1 to 10, the generation unit may generate the display image showing the second information in a display format different from that of the first information.

[0231] In the information processing system according to aspect 12 of the present disclosure, in any one of aspects 1 to 11, the generation unit may generate the display image displaying information indicating the duration of treatment for at least one of bone density, bone mass, and bone quality for the subject.

[0232] In the information processing system according to aspect 13 of the present disclosure, in any one of aspects 1 to 12, the generation unit may generate the display image displaying trend information showing the trend of at least one of bone density, bone mass, and bone quality of a reference person different from the subject.

[0233] In the information processing system according to aspect 14 of the present disclosure, in any of aspects 1 to 13, the generator may generate the display image displaying information indicating criteria for at least one of bone density, bone mass, and bone quality.

[0234] In an information processing system according to aspect 15 of the present disclosure, in any one of aspects 1 to 14, the generation unit may generate the display image including an indication of the first technique and the second technique.

[0235] An information processing system according to aspect 16 of the present disclosure may include a measuring device that performs measurements on bones in any of aspects 1 to 15, and the first information may include information measured using the measuring device that performs measurements on bones.

[0236] The information processing system of aspect 17 of the present disclosure may be, in any of aspects 3 to 10, provided with an imaging device that captures an image showing a bone, and the first image may be an image captured using the imaging device.

[0237] An information processing method according to aspect 18 of the present disclosure includes a first acquisition step of acquiring first information indicating information about the subject's bones obtained based on a first method, a second acquisition step of acquiring second information indicating information about the subject's bones obtained based on a second method different from the first method, and a generation step of generating a display image displaying the first information and the second information.

[0238] An image generating device according to aspect 19 of the present disclosure includes a first acquisition unit that acquires first information indicating information about the bones of a subject obtained based on a first method, a second acquisition unit that acquires second information indicating information about the bones of the subject obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0239] The image generation program of aspect 20 of the present disclosure is an image generation program for causing a computer to function as the information processing system described in aspect 19, and is an image generation program for causing a computer to function as the first acquisition unit, the second acquisition unit, and the generation unit.

[0240] 1, 1a, 1b, 1c, 1d, 1e Information processing system 4, 4a, 4c, 4d Display device (image generating device) 6b, 6e Server device (image generating device) 421, 421a, 421c, 621b Acquisition unit (first acquisition unit) 424, 424c, 624b, 624e Image generation unit (generation unit) 425, 425c, 625b, 625e Information acquisition unit (first acquisition unit, second acquisition unit, third acquisition unit, fourth acquisition unit) 431 Measurement information (first information) 433 Estimated information (second information) 434 Estimated information 435c Fracture risk information (third information) 4221 Estimation model 4271 Estimation model S2 First acquisition step S3 Second acquisition step S4 Generation step

Claims

1. An information processing system comprising: a first acquisition unit that acquires first information indicating information relating to a subject's bones obtained based on a first method; and a generation unit that generates a display image that displays the first information together with information relating to the first method.

2. An information processing system as described in claim 1, wherein the information relating to the first method relates to estimated information regarding at least one of bone density, bone mass and bone quality estimated by an estimation model from an image showing the subject's bones at a certain point in time, or estimated information regarding at least one of bone density, bone mass and bone quality estimated by an estimation model from an image showing the subject's bones at a certain point in time.

3. The information processing system of claim 1, further comprising a second acquisition unit that acquires second information indicating information about the subject's bones obtained for the subject based on a second method different from the first method, and the generation unit displays the first information and the second information.

4. An information processing system according to claim 3, wherein the first information and the second information are information relating to at least one of the subject's bone density, bone mass and bone quality.

5. The information processing system of claim 4, wherein the first acquisition unit acquires the first information measured at a first time point, the second acquisition unit acquires the second information estimated by an estimation model from a first image showing the bones of the subject at a second time point different from the first time point, and the generation unit generates the display image in which the first information and the second information are arranged in chronological order of the first time point and the second time point.

6. The information processing system of claim 5, wherein the estimation model is set based on learning data including a second image showing the bones of a first person, and teacher data including at least one of the bone density, bone mass, and bone quality of the first person.

7. An information processing system as described in claim 5 or 6, further comprising a third acquisition unit that acquires estimated information regarding at least one of bone density, bone mass and bone quality estimated by a prediction model from an image showing the subject's bones at the first time point or the second time point, wherein if the image showing the subject's bones is an image showing the subject's bones at the first time point, the estimated information is information at a third time point different from the first time point, and if the image showing the subject's bones is an image showing the subject's bones at the second time point, the estimated information is information at a third time point different from the second time point, and the generation unit generates the display image displaying the estimated information associated with the third time point.

8. The information processing system of claim 7, wherein the prediction model is set based on learning data including a third image showing the bones of a second person, and teacher data including at least one of the bone density, bone mass, and bone quality of the second person.

9. The information processing system of claim 8, wherein the estimated information is information regarding the bone density of the subject when the subject undergoes treatment for at least one of bone density, bone mass and bone quality, the second person is a person who has received treatment for at least one of bone density, bone mass and bone quality, and the estimated model is set based on learning data including the third image showing the bones of the second person who has received the treatment and teacher data including at least one of bone density, bone mass and bone quality of the second person.

10. An information processing system as claimed in any one of claims 7 to 9, wherein, if the image showing the subject's bones is an image showing the subject's bones at the first time point, the third time point is a time point earlier than the first time point or a time point later than the first time point, and, if the image showing the subject's bones is an image showing the subject's bones at the second time point, the third time point is a time point earlier than the second time point or a time point later than the second time point.

11. An information processing system as described in any one of claims 7 to 10, further comprising a fourth acquisition unit that acquires third information regarding the subject's bones calculated from at least one of the first information, the second information, and the estimated information, and the generation unit generates the display image that displays the third information.

12. An information processing system as described in any one of claims 5 to 11, wherein the generation unit generates the display image displaying information indicating the names of bones appearing in the first image used to estimate the second information.

13. The information processing system according to any one of claims 3 to 12, wherein the generation unit generates the display image showing the second information in a display mode different from that of the first information.

14. An information processing system according to any one of claims 1 to 13, wherein the generation unit generates the display image displaying information indicating the duration of treatment for at least one of bone density, bone mass, and bone quality for the subject.

15. An information processing system according to any one of claims 1 to 14, wherein the generation unit generates the display image displaying trend information showing the trend of at least one of bone density, bone mass and bone quality of a referent individual different from the subject individual.

16. An information processing system according to any one of claims 1 to 15, wherein the generation unit generates the display image displaying information indicating a judgment criterion for at least one of bone density, bone mass, and bone quality.

17. An information processing system according to any one of claims 3 to 13, wherein the generation unit generates the display image including an indication of the first technique and the second technique.

18. An information processing system according to any one of claims 3 to 13, further comprising a measuring device for performing measurements on bones, and the second information includes information measured using the measuring device.

19. An information processing system according to any one of claims 5 to 12, further comprising an imaging device that captures an image showing a bone, and the first image is an image captured using the imaging device.

20. An information processing method including: a first acquisition step of acquiring first information indicating information regarding the subject's bones obtained based on a first method; a second acquisition step of acquiring second information indicating information regarding the subject's bones obtained based on a second method different from the first method; and a generation step of generating a display image displaying the first information and the second information.

21. An image generating device comprising: a first acquisition unit that acquires first information indicating information regarding the subject's bones obtained based on a first method; a second acquisition unit that acquires second information indicating information regarding the subject's bones obtained based on a second method different from the first method; and a generation unit that generates a display image displaying the first information and the second information.

22. An image generating program for causing a computer to function as the image generating device according to claim 21, the image generating program causing a computer to function as the first acquisition unit, the second acquisition unit, and the generation unit.

Images