Information processing system, information processing method, image generation device and image generation program

Abstract

To grasp transition of a subject's bone without measuring the bone regularly.SOLUTION: An information processing system includes: an information acquisition part acquiring information about a subject's bone obtained on the basis of a first method and information about the subject's bone obtained on the basis of a second method different from the first method; and an image generation part generating a display image displaying the first information and second information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing system and an information processing method that assist in understanding a bone condition. [Background technology]

[0002] In recent years, it has been proposed to estimate or predict the physical information of a subject using computer calculations. For example, Patent Document 1 proposes a method for predicting the future bone density of a postmenopausal woman using a machine learning model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-200788 A Summary of the Invention [Problem to be solved by the invention]

[0004] When using such computer calculations to estimate or predict the subject's physical information, etc., one of the challenges is to improve user convenience. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing system according to one embodiment of the present disclosure includes a first acquisition unit that acquires first information indicating information regarding the subject's bones obtained based on a first method, a second acquisition unit that acquires second information indicating information regarding the subject's bones obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0006] Furthermore, an information processing method according to one embodiment of the present disclosure includes a first acquisition step of acquiring first information indicating information regarding the subject's bones obtained based on a first method, a second acquisition step of acquiring second information indicating information regarding the subject's bones obtained based on a second method different from the first method, and a generation step of generating a display image displaying the first information and the second information.

[0007] In addition, a server device according to one embodiment of the present disclosure includes a first acquisition unit that acquires first information indicating information regarding the subject's bones obtained based on a first method, a second acquisition unit that acquires second information indicating information regarding the subject's bones obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0008] The display device, first generating device and second generating device according to each aspect of the present disclosure may be realized by a computer. In this case, the control programs of the display device, first generating device and second generating device that cause the computer to operate as each unit (software element) of the display device, first generating device and second generating device, and the computer-readable recording medium on which they are recorded, also fall within the scope of the present disclosure. Effect of the Invention

[0009] According to one aspect of the present disclosure, it is possible to improve user convenience. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a main part of an example of an information processing system according to a first embodiment of the present invention. [Diagram 2] 5 is a flowchart showing an example of the flow of a display process in the display device according to the first embodiment. [Diagram 3]FIG. 13 is a diagram showing an example of a display image displaying an estimated value of a subject's bone mineral density at a time point after the time point serving as a base point of estimation according to the first embodiment. [Figure 4] FIG. 13 is a diagram showing another example of a display image displaying an estimated value of a bone mineral density of a subject at a time point subsequent to the time point serving as a base point of estimation according to the first embodiment. [Diagram 5] FIG. 13 is a diagram showing another example of a display image displaying an estimated value of a bone mineral density of a subject at a time point subsequent to the time point serving as a base point of estimation according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of a display image displaying an estimated value of a subject's bone mineral density at a time point prior to a time point that serves as a base point of estimation according to the first embodiment. [Figure 7] FIG. 11 is a diagram showing another example of a display image displaying an estimated value of a bone mineral density of a subject at a time point prior to the time point serving as a base point of estimation according to the first embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a display image displaying information relating to the bone density of a subject and a transition of the bone density of a referrer according to the first embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a display image displaying an estimated value in response to a change in the timing of starting treatment for a subject according to the first embodiment. [Figure 10] FIG. 2 is a diagram showing an example of a display image displaying an error range of an estimated value of a subject's bone mineral density according to the first embodiment. [Figure 11] FIG. 2 is a diagram showing an example of a display image displaying the estimation probability of the estimated value of the bone mineral density of a subject according to the first embodiment. [Figure 12] FIG. 2 is a diagram showing an example of a display image that displays a medical image used to estimate an estimated value and a measurement result corresponding to a measurement value according to the first embodiment. [Figure 13] FIG. 4 is a diagram showing an example of a display of a medical image corresponding to an estimated value according to the first embodiment. [Figure 14] FIG. 2 is a diagram showing an example of a display image displaying the transition of bone density at each of a plurality of parts according to the first embodiment. [Figure 15] FIG. 2 is a diagram showing an example of a display showing a location where a fracture is likely to occur, included in a display image displayed on a display unit according to the first embodiment. [Figure 16] FIG. 11 is a block diagram showing a configuration of a main part of an example of an information processing system according to a second embodiment of the present invention. [Figure 17] FIG. 11 is a block diagram showing a configuration of a main part of an example of an information processing system according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Embodiment 1] Hereinafter, one embodiment of the present disclosure will be described in detail with reference to FIGS.

[0012] (overview) First, an overview of this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing a configuration of a main part of an example of an information processing system 1 according to this embodiment. As shown in Fig. 1, the information processing system 1 includes a measuring device 2, an imaging device 3, and a display device (image generating device) 4.

[0013] In this embodiment, the measurement device 2 measures the bones of the subject and transmits first information regarding the bones of the subject to the display device 4. That is, the display device 4 acquires first information indicating information regarding the bones of the subject obtained based on the bone measurement. Here, the first information may be information regarding at least one of the bone density, bone mass, and bone quality of the subject. Furthermore, the first information may be information indicating the measurement value of the bone density of the subject. In this embodiment, an example is described in which the first information is measurement information 431 indicating the measurement value of the bone density of the subject, but the description of "bone density" regarding the "first information" described in this disclosure may be appropriately interpreted as at least one of "bone density," "bone mass," and "bone quality."

[0014] In this specification, the expression "measurement" may mean "actual measurement." Furthermore, the expression "measurement value" may mean "actual measurement value."

[0015] In the following description, the case where the subject to which the application of a surgical treatment is considered is a human (i.e., a "subject") is taken as an example, but the subject is not limited to humans. That is, the "subject" according to the present disclosure may be a mammal other than a human, such as an equine, feline, canine, bovine, or porcine animal. The present disclosure also includes, among the following embodiments, embodiments in which "subject," "patient," and "person" are replaced with "animal" if the embodiments are applicable to these animals.

[0016] The first information may include information indicating the name of a disease from which the subject suffers. The information indicating the name of the disease may be information indicating at least one of osteoporosis, rheumatism, bone necrosis (e.g., femoral head necrosis), systemic sclerosis, kidney disease, osteopetrosis, and the like. The first information may also include bone evaluation information. The bone evaluation information may include information evaluated by a fracture risk assessment tool (FRAX (registered trademark); Fracture Risk Assessment Tool).

[0017] The bone density may be any value related to the density of bone. The bone density may be, for example, bone mineral density per unit area (g / cm 2 ), bone mineral density per unit volume (g / cm 3 ), YAM(%), AGE(%), T-score, and / or Z-score. YAM is an abbreviation for "Young Adult Mean" and may also be referred to as young adult mean percent, %YAM, YAM ratio, etc. AGE may be a value compared to the average bone density of the same age or age group. T-score is a value obtained by comparing the measured bone density with the average young adult value and dividing it by the standard deviation of the young adult. Z-score is a value obtained by comparing the measured bone density with the average value of the same age as the subject and dividing it by the standard deviation of the same age. Bone density is not limited to these examples, and may be, for example, a unique numerical value.

[0018] The above-mentioned bone mass is an index related to bone density, and the bone mass is the amount of bone tissue in the skeleton.

[0019] The bone quality may be based on at least one of the following: statistical properties of bone, geometric properties of bone, mechanical properties of bone, and chemical properties of bone. The bone quality may include information on the attributes of the subject, which will be described later. The bone quality may be based on at least one of the following: bone metabolism marker, sex, race, menopause, age, cortical bone condition, cancellous bone condition, cancellous bone trabecular condition, disease information, bone evaluation information, drug information, presence or absence of fracture, number of fractures, fracture location, and fracture history. More specifically, the bone quality may be based on at least one of the following: bone formation marker, bone resorption marker, bone quality marker (e.g., vitamin K value), cortical bone thickness, trabecular density, trabecular direction, and cancellous bone structure index (trabecular bone score).

[0020] The imaging device 3 transmits medical image data 432, which is data showing a medical image of the subject, to the display device 4.

[0021] The medical image may be, for example, an image showing at least a part of the neck, chest, lower back, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, or jaw joint. The X-ray image may show a part other than bones. For example, a plain chest X-ray image may include an image of the lungs and an image of the thoracic vertebrae. The X-ray image may be a frontal image showing the target part from the front, or a lateral image showing the target part from the side.

[0022] The display device 4 estimates second information about the subject's bones using the medical image. Here, the second information may be information about at least one of the subject's bone density, bone mass, and bone quality. Furthermore, the second information may be information indicating an estimated value of at least one of the subject's bone density, bone mass, and bone quality. That is, the display device 4 acquires, for the subject, second information indicating information about the subject's bones obtained based on a second method different from the first method. Here, the first method may be a measurement of the subject's bones. Furthermore, the second method may be an estimation of the subject's bones.

[0023] For example, the first method may be a measurement performed in a hospital equipped with a measurement device for measuring bones. The second method may be an estimation from a medical image captured in a hospital that does not have the measurement device. The first method and the second method may be performed in different hospitals.

[0024] In this embodiment, an example is described in which the second information is estimated information 433 indicating an estimated value of the subject's bone density, but the description of "bone density" in relation to the "second information" described in this disclosure may be appropriately interpreted as at least one of "bone density," "bone mass," and "bone quality."

[0025] Other examples of the first information indicating information about a subject's bones obtained based on bone measurements and the second information obtained based on bone estimation are given below.

[0026] A case will be described in which the bones mainly include bones of the spine or intervertebral disc. In this case, the first information and the second information may include, for example, information indicating at least one of vertebral body height, intervertebral body distance, anterior intervertebral disc space distance, posterior intervertebral disc space distance, poor posture classification, ossification occupation rate (ratio of thickness of ossified ligament to anteroposterior diameter of spinal canal), residual effective anteroposterior diameter (thickness between anteroposterior diameter of spinal canal and ossified ligament), vertebral body posterior slip distance (distance between posterior inferior corner of superior vertebral body and tip of inferior vertebral arch) and Meyerding classification.

[0027] Diseases that can be diagnosed using the above information include, for example, degenerative spondylosis, vertebral fractures (including traumatic and osteoporotic), spinal stenosis, hernia, degenerative spondylolisthesis, scoliosis, thoracic and lumbar degenerative diseases, bilateral congenital dislocation of the hip, and ossification of the spinal ligaments.

[0028] A case where the bone mainly includes a hip bone will be described. In this case, the first information and the second information may include at least one of the following: joint space area, joint space distance (width), minimum joint space distance (width), CE angle (Center Edge angle: angle between a perpendicular line passing through the center of the femoral head and a line connecting the center of the femoral head and the lateral edge of the acetabulum), Sharp angle (Acetabular Angle: angle between a line connecting the lateral edge of the acetabulum and the tip of the teardrop and a line connecting both sides of the teardrop), ARO (Acetabular Roof Obliquity: angle between a horizontal line passing through the acetabular floor and a line connecting the lateral edge of the acetabulum and the acetabular floor), AHI (Acetabular Head Index: distance from the medial end of the femoral head to the lateral end of the acetabulum divided by the transverse diameter of the femoral head), and Kellgren-Lawrence (KL) classification.

[0029] Diseases that can be diagnosed using the above information include, for example, osteoarthritis of the hip (OA), femoroacetabular impingement (one of the pathological conditions of OA), hip dysplasia, hip dislocation (including congenital), femoral slippage, and femoral head necrosis.

[0030] A case will be described in which the bones mainly include knee bones. In this case, the first information and the second information may include, for example, information indicating at least one of the joint space area, joint space distance (width), minimum joint space distance (width), osteophyte area, femoro-tibial angle (femoro-tibial lateral angle), Osteoarthritis Research Society International (OARSI) classification, and Kellgren-Lawrence (KL) classification. The above information makes it possible to diagnose, for example, knee osteoarthritis (OA).

[0031] A case will be described in which the bones mainly include bones of the foot. In this case, the first information and the second information may be information indicating at least one of the talus inclination angle and the talus anterior drawer distance. Diseases that can be diagnosed using the above information include, for example, ankle sprain, talus necrosis, rheumatoid arthritis, ankle impingement syndrome, etc.

[0032] The display device 4 generates a display image displaying the first information and the second information. In this embodiment, an example will be described in which the display device 4 generates a display image displaying the measurement value of the subject's bone density indicated by the measurement information 431 and the estimated value of the subject's bone density indicated by the estimated information 433.

[0033] According to the above configuration, the information processing system 1 can generate a display image that displays information about the subject's bones obtained based on a plurality of different methods, for example, a method of measurement and estimation. That is, in a medical setting, even if the subject's bones are not periodically measured using a predetermined measuring device, the transition of the subject's bone condition can be grasped by using a method other than measurement, such as estimation. For example, the transition of the subject's bone condition may be the transition of at least one of the subject's bone density, bone mass, and bone quality.

[0034] The measured values ​​of the subject displayed in the image may be at least one of the measured values ​​of the subject's bone density, bone mass, and bone quality measured at a first time point. The estimated values ​​of the subject displayed in the image may be estimated by an estimation model 4221 or the like provided in the display device 4 from a medical image showing the subject's bones at a second time point different from the first time point. The estimated value may be at least one of the estimated values ​​of the bone density, bone mass, and bone quality. The display device 4 may generate a display image in which the measured values ​​and the estimated values ​​of the subject are arranged in chronological order of the first time point and the second time point.

[0035] According to the above configuration, the information processing system 1 generates an image in which measured information about the subject's bones at a first time point and estimated information about the subject's bones at a second time point are arranged in chronological order based on the first and second time points. That is, an image can be generated in which measured values ​​and estimated values ​​of at least one of the subject's bone density, bone mass, and bone quality are arranged in chronological order.

[0036] (Information Processing System 1) Next, details of the information processing system 1 according to this embodiment will be described. As described above, the information processing system 1 includes the measuring device 2, the imaging device 3, and the display device 4.

[0037] <Measuring device 2> The measuring device 2 is a device that measures the bone density of a subject. The measuring device 2 may be any device that can measure the bone density of a subject, and is not particularly limited, but examples thereof include the following.

[0038] For example, the measuring device 2 may be a device that measures bone density using a dual energy X-ray absorptiometry (DXA) method, an ultrasonic method, or a micro densitometry (MD) method.

[0039] The measuring device 2 transmits measurement information 431 indicating the measurement value of the subject's bone density to the display device 4. In other words, the measurement information 431 can be said to be information measured using a specific measuring device that performs measurements on bones. The measurement information 431 may include information indicating the date and time when the measurement was performed. The communication between the measuring device 2 and the display device 4 may be wired communication or wireless communication. Furthermore, the communication may be communication via the Internet.

[0040] <Imaging device 3> The imaging device 3 may be, for example, a device that captures images to obtain information about the inside of the subject's body. The imaging device 3 is a device that captures medical images of the subject. The medical images captured by the imaging device 3 may be images that include information about the subject's bones, and are not particularly limited, but examples thereof include the following.

[0041] For example, the medical image may be an X-ray image, a magnetic resonance imaging (MRI) image, a computed tomography (CT) image, a positron emission tomography (PET) image, or an ultrasound image.

[0042] The imaging device 3 transmits medical image data 432, which is data showing a captured medical image, to the display device 4. The medical image data 432 may include information showing the date and time when the medical image was captured. The communication between the imaging device 3 and the display device 4 may be wired communication or wireless communication. Furthermore, the communication may be communication via the Internet.

[0043] ≪Display device 4≫ The display device 4 generates a display image showing the measured value of the subject's bone density and the estimated value of the subject's bone density, and displays the image. The display device 4 includes a communication unit 41, a control unit 42, a storage unit 43, an operation input unit 44, and a display unit 45.

[0044] <Communications Division 41> The communication unit 41 communicates with the measurement device 2 and the imaging device 3. The communication unit 41 receives measurement information 431 from the measurement device 2. In addition, the communication unit 41 receives medical image data 432 from the imaging device 3.

[0045] <Control unit 42> The control unit 42 comprehensively controls each unit of the display device 4. The control unit 42 includes an acquisition unit 421 (first acquisition unit), a first generation unit 422, an operation acceptance unit 423, an image generation unit (generation unit) 424, an information acquisition unit (first acquisition unit, second acquisition unit, third acquisition unit) 425, a display control unit 426, and a second generation unit 427.

[0046] 《Acquisition section 421》 The acquiring unit 421 acquires measurement information 431 from the measurement device 2 via the communication unit 41. The acquiring unit 421 also acquires medical image data 432 from the imaging device 3 via the communication unit 41.

[0047] The acquisition unit 421 stores the acquired measurement information 431 and medical image data 432 in the storage unit 43. In addition, the acquisition unit 421 transmits the acquired medical image data 432 to the first generation unit 422.

[0048] 《First generation section 422》 The first generating unit 422 generates estimated information 433 indicating an estimated value of the bone density of the subject estimated from a medical image (first image) indicated by medical image data 432 of the subject received from the acquiring unit 421. The first generating unit 422 stores the generated estimated information 433 in the storage unit 43. The first generating unit 422 includes an estimation model 4221.

[0049] <Estimated Model 4221> The estimation model 4221 is a model that estimates the bone density of a subject from a medical image indicated by the medical image data 432 of the subject. The estimation model 4221 may be set based on learning data including a medical image (second image) showing the bones of the first person, and teacher data including at least one value of the bone density, bone mass, and bone quality of the first person. In other words, the estimation model 4221 is a trained model.

[0050] For example, the bone density of a person whose bones are shown in the training plain X-ray image represented by each of the multiple training image data included in the training data may be associated as training data. The bone density of the training data may be bone density obtained using a dual energy X-ray absorptiometry (DXA) method, an ultrasound method, or a micro densitometry (MD) method. Furthermore, the above-mentioned "first person" does not need to be limited to a human. For example, it may be an animal of the same species as the "subject."

[0051] <<Operation Reception Unit 423>> The operation reception unit 423 receives an operation input by a user via the operation input unit 44. The operation reception unit 423 receives an operation input instructing display of a display image displaying information about the bones of a subject. When the operation reception unit 423 receives the operation input, it instructs the image generation unit 424 to generate an image.

[0052] The operation accepting unit 423 accepts an operation input instructing a guess regarding the bones of the subject. When the operation accepting unit 423 accepts the operation input, the operation accepting unit 423 instructs the second generating unit 427 to generate guess information 434 indicating a guess regarding the bones of the subject. Furthermore, the operation accepting unit 423 instructs the image generating unit 424 to generate a display image that displays the guess information 434.

[0053] The operation accepting unit 423 accepts an operation input instructing to end the display. When the operation accepting unit 423 accepts the operation input, it instructs the display control unit 426 to end the display.

[0054] Image Generation Unit 424 When the image generation unit 424 receives an instruction to generate a display image displaying information about the subject's bones, the image generation unit 424 acquires the measurement information 431 and the estimated information 433 from the memory unit 43 via the information acquisition unit 425. The image generation unit 424 uses the measurement information 431 and the estimated information 433 to generate a display image displaying information about the subject's bones.

[0055] When the image generation unit 424 receives an instruction to generate a display image displaying the estimated information 434, it acquires the measurement information 431, the estimated information 433, and the estimated information 434 from the memory unit 43 via the information acquisition unit 425. The image generation unit 424 uses the measurement information 431, the estimated information 433, and the estimated information 434 to generate a display image displaying information related to the subject's bones.

[0056] The image generating unit 424 transmits data of the generated display image to the display control unit 426. An example of the display image generated by the image generating unit 424 will be described later in detail.

[0057] 《Information acquisition section 425》 In response to an instruction from the image generating section 424 , the information acquiring section 425 acquires the measurement information 431 and the estimated information 433 stored in the storage section 43 , and transmits them to the image generating section 424 .

[0058] In response to an instruction from the image generating section 424 , the information acquiring section 425 acquires the measurement information 431 , the estimated information 433 and the inferred information 434 stored in the memory section 43 , and transmits them to the image generating section 424 .

[0059] <<Display control unit 426>> The display control unit 426 causes the display unit 45 to display the display image indicated by the image data received from the image generating unit 424 .

[0060] In response to an instruction to end the display received from the operation receiving unit 423, the display control unit 426 ends the image display on the display unit 45.

[0061] 《Second generation section 427》 The second generating unit 427 generates estimated information 434 indicating an estimated value of the bone density of the subject estimated from the medical image indicated by the medical image data 432 stored in the storage unit 43.

[0062] For example, the second generator 427 may generate inferred information 434 regarding at least one of bone density, bone mass, and bone quality inferred from a medical image depicting the subject's bones at a first time point when at least one of the subject's bone density, bone mass, and bone quality was measured. In this embodiment, an example will be described in which the inferred information 434 indicates an estimated value of the subject's bone density, but the term "inferred information 434" in the present disclosure may be appropriately interpreted as information indicating at least one of "bone density," "bone mass," and "bone quality."

[0063] Here, the measurement of bone density at the first time point and the capture of the medical image showing the bones of the subject at the first time point do not have to be performed strictly at the same time. For example, the measurement and the capture may be performed within a predetermined period. The predetermined period is not particularly limited as long as it is a period during which no significant change occurs in the bone density of the subject, and may be, for example, half a day, one day, one week, one month, three months, six months, and the like. Furthermore, if a medical image showing the bones of the subject is not captured at the first time point, the display device 4 may not receive the medical image data 432 showing the medical image showing the bones of the subject at the first time point. That is, if a medical image showing the bones of the subject is captured at the first time point, the display device 4 may accept the medical image data 432 showing the medical image showing the bones of the subject at the first time point. In this specification, the time point at which the medical image used to generate the estimated information 434 is captured is referred to as the base point of estimation. The estimated information 434 may be information showing an estimated value of the bone density of the subject at a third time point. The third time point may be a different time point than the first time point.

[0064] The second generating unit 427 may generate estimated information 434 on the bone density of the subject, estimated from the medical image showing the bones of the subject at the second time point used by the first generating unit 422 to estimate the estimated value of the bone density of the subject. The estimated information 434 may be information indicating an estimated value of the bone density of the subject at a third time point. The third time point may be a time point different from the second time point.

[0065] When the image generating section 424 generates an image using the estimated information 434, the image generating section 424 may generate a display image that displays the estimated information 434 associated with the third time point.

[0066] The second generating unit 427 may generate the inferred information 434 in response to an instruction to generate inferred information 434 indicating an inference regarding the bones of the subject received from the operation receiving unit 423. The second generating unit 427 stores the generated inferred information 434 in the storage unit 43. The second generating unit 427 further includes an inferred model 4271.

[0067] <Speculative Model 4271> The estimation model 4271 is a model that estimates the bone density of a subject at a time point different from the time point at which the medical image indicated by the medical image data 432 of the subject was captured, from the medical image. The estimation model 4271 may be set based on learning data including a medical image (third image) showing the bones of the second person, and teacher data including the bone density of the second person. The above-mentioned "second person" does not need to be limited to a human being. For example, it may be an animal of the same species as the "subject."

[0068] For example, the bone density of a person whose bone appears in the training plain X-ray image shown by a plurality of training image data included in the training data is associated as training data. Training data may be associated with each of the plurality of training image data, or a plurality of sets of training image data may be associated with one training data. The training data may be bone density obtained using a dual energy X-ray absorptiometry (DXA) method, an ultrasound method, or a micro densitometry (MD) method. Alternatively, a value estimated or predicted from another trained model may be used as the training data.

[0069] The teacher data may be data measured at any time. The teacher data may be data measured at the same time when the learning image data was captured, or may be data measured on a different time axis. For example, a predetermined criterion may be set for the measurement of the teacher data with respect to the time when the learning image data was captured. As the predetermined criterion, for example, within 3 months, 6 months, or 1 year including the past with respect to the time when the learning image data was captured may be used. In addition, the predetermined criterion for the teacher data may be at least one of the presence or absence of drug treatment, age, menopause, and sex of the subject in the teacher data.

[0070] The training image data of the training data may be a series of image data of the same person taken on different time axes. That is, the training image data may include the following first training data and second training data. The first training data may be training data including an X-ray image of a person's bones. The second training data may be training data including an X-ray image of the same person as the certain person taken after the first training data.

[0071] The learning image data of the learning data may be a data group of the same body part of other people with different ages, etc. Also, the learning image data of the learning data may be a series of image data of the same person on different time axes. The learning image data of the learning data may be a series of data of the same body part on different time axes.

[0072] The estimated information 434 generated by the second generating unit 427 may be information on the bone density of the subject when the subject is treated for bone density. In this case, the above-mentioned second person is a person who has received treatment for bone density, and the estimated model 4271 may be set based on learning data including medical images showing the bones of the second person who has received treatment, and teacher data including the value of the bone density of the second person. For example, the above-mentioned learning data may be learning data including medical images before and after the treatment. Also, the above-mentioned teacher data may be teacher data including measurement values ​​of bone density before and after the treatment.

[0073] The second generation unit 427 may include the following two models as the estimation model 4271. (1) A model set based on learning data including medical images showing the bones of a second person who has received treatment and teacher data including the measurement value of the bone density of the second person who has received treatment. (2) A model set based on learning data including medical images showing the bones of a second person who has not received treatment and teacher data including the measurement value of the bone density of the second person who has not received treatment.

[0074] The second generating section 427 may generate inferred information 434 according to the timing at which treatment of the subject is to be started.

[0075] Furthermore, when the medical image indicated by the medical image data 432 used to estimate the subject's bone density is an image at a first time point, the third time point may be a time point earlier than the first time point or a time point later than the first time point. Furthermore, when the medical image indicated by the medical image data 432 used to estimate the subject's bone density is an image at a second time point, the third time point may be a time point earlier than the second time point or a time point later than the second time point.

[0076] The medical image (second image) showing the first human bone, which is the training data for the estimation model 4221, and the medical image (third image) showing the second human bone, which is the training data for the prediction model 4271, may be the same image.

[0077] The second image and the third image may be medical images of different parts of the body. The person captured in the second image and the person captured in the third image may be the same person. Also, the person captured in the second image and the person captured in the third image may be different people.

[0078] The training data including the measurement value of the bone density of the second person may be data measured at a time different from the time when the third image was taken. For example, the training data may be at least one of data six months, one year, several years, and several years after the third image was taken. The learning data may be a measurement value of the bone density measured during a predetermined period including the time when the third image was taken.

[0079] <Storage section 43> The storage unit 43 stores the measurement information 431, the medical image data 432, the estimation information 433, and the inference information 434 described above. The storage unit 43 may also store subject data indicating information about the subject. The subject data may include data indicating whether or not the subject received treatment, the duration of the treatment given to the subject, the contents of the treatment given to the subject, the date and time when the subject was injured, and the like.

[0080] <Operation input unit 44> The operation input unit 44 accepts operation input from the user. For example, the operation input unit 44 may be integrated with the display unit 45. The display unit 45 may display instructions for the process of estimating the bone density of the subject, the image generation process, the image display process, and the like, which are processes performed by the control unit 42. A configuration may be adopted in which the process corresponding to the display area is executed by the user selecting one of these display areas.

[0081] <Display section 45> The display unit 45 displays to the user the image generated by the image generation unit 424. Examples of the display unit 45 include a display and a monitor.

[0082] (Display process flow) Next, the flow of the display process in the display device 4 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of the flow of the display process in the display device 4. As shown in Fig. 2, when the operation receiving unit 423 receives an operation input instructing the display of a display image showing information related to the bones of a subject (YES in S1), the information acquiring unit 425 performs the following process.

[0083] The information acquiring unit 425 acquires the measurement information 431 (S2: first acquiring step). The information acquiring unit 425 also acquires the estimated information 433 (S3: second acquiring step). For example, the first information may be information indicating the bone density of the subject measured at a first time point, and the second information may be information indicating the bone density estimated by an estimation model from a first image showing the bones of the subject at a second time point different from the first time point. Next, the image generating unit 424 generates a display image displaying the measurement value indicated by the measurement information 431 and the estimated value indicated by the estimated information 433 (S4: generating step). Next, the display control unit 426 causes the display unit 45 to display the display image generated by the image generating unit 424 (S5).

[0084] When the operation receiving unit 423 receives an operation input instructing an inference regarding the bones of the subject (YES in S6), the information acquiring unit 425 acquires the inference information 434 (S7). The operation input may include an input indicating a base point of the inference and an input indicating when the inference is from the base point. For example, although not particularly limited, examples include an inference six months after the base point, an inference one year after the base point, an inference six months before the base point, and an inference one year before the base point. The operation input may also include an input regarding the presence or absence of treatment for the subject. For example, examples of the inference include an inference when treatment is performed on the subject, an inference when treatment is not performed, and an inference when both treatment is performed and treatment is not performed. The operation input may also include an input regarding the timing of the start of treatment for the subject. For example, the inference may be an inference according to the timing of the start of treatment for the subject.

[0085] Next, the image generating unit 424 generates a display image that displays the estimated value indicated by the estimation information 434 (S8). Then, the process returns to S5.

[0086] If the operation receiving unit 423 does not receive an operation input instructing to make a guess regarding the bones of the subject (NO in S6), but receives an operation input instructing to end the display (YES in S9), the process ends.

[0087] When the operation receiving unit 423 has not received an operation input instructing the display of a display image showing information about the bones of the subject (NO in S1), the process of S1 is repeated.

[0088] If the operation receiving unit 423 does not receive an operation input instructing a guess regarding the subject's bones (NO in S6) or an operation input instructing to end the display (NO in S9), the process returns to S6.

[0089] (Example of a display image generated by the image generating unit 424) Next, examples of display images generated by image generating unit 424 will be described with reference to Figures 3 to 9. Furthermore, the term "bone density" shown in Figures 3 to 9 may be appropriately interpreted as at least one of "bone density," "bone mass," and "bone quality."

[0090] <Example of a display image showing an estimated value at a point after the point in time that serves as the base point of estimation> First, an example of a display image displaying an estimated value at a time point after the time point that is the base point of estimation will be described. Figures 3 to 5 are diagrams showing an example of a display image displaying an estimated value of the subject's bone density at a time point after the time point that is the base point of estimation.

[0091] 3 shows measured values ​​m1-m3 of the subject's bone density at the time of measurement, estimated values ​​n1-n2 of the subject's bone density at the time the medical image was taken, and a transition T1 of the bone density based on the measured values ​​m1-m3 and the estimated values ​​n1-n2. The image D1 may also show the name of the facility where the medical image was taken.

[0092] The display manner of the subject's measured bone mineral density may be different from the display manner of the subject's estimated bone mineral density, for example, the measured value may be displayed as a white circle, and the estimated value may be displayed as a black circle.

[0093] The image D1 displays the estimated values ​​o1 and o2. The measured values, estimated values, and estimated values ​​may be displayed in different display modes. In the example shown in Fig. 3, the estimated values ​​o1 and o2 are shown as open triangles. The estimated values ​​are shown connected with dotted lines, and the estimated values ​​are shown at the end points of the dotted lines. The start point of the dotted lines indicates the time point that is the base point of the estimation.

[0094] In image D1, the estimated value o1 is an estimated value based on December 21, 2021 as the base point of estimation. The estimated value o1 is an estimated value estimated from a medical image taken on December 21, 2021. In addition, the estimated value o1 is an estimated value of the subject's bone density as of February 21, 2022, which is two months after the base point.

[0095] The estimated value o2 is an estimated value based on the present, and is an estimated value estimated from medical images taken at the present. The estimated value o2 is an estimated value of the subject's bone density at a point six months from the present.

[0096] When the bone density of a subject is measured at a certain time, the measurement value at the certain time displayed in image D1 is the measurement value obtained by the measurement and indicated by the corresponding measurement information 431. On the other hand, when the same time as the certain time is used as the base point of estimation, the estimated value using the certain time as the base point of estimation displayed in image D1 is the estimated value estimated from the medical image captured at the certain time.

[0097] Each measured value, each estimated value, and each estimated value may be displayed with the notation "measured value", "estimated value", and "estimated value". In addition, the method by which the displayed value was obtained may be displayed in the image D1. Examples of the method include "measurement", "estimate", and "estimate". That is, the image generating unit 424 may generate a display image including an indication of the first method and the second method. By displaying the methods in this way, it is possible to improve user convenience. In addition, the image D1 displays the YAM (Young Adult Mean) ratio, which is a value corresponding to bone density. In the image D1, the YAM ratio is a value indicating a diagnostic criterion, and the notation "normal" is displayed in the area of ​​the YAM ratio from 100% to 80%, "bone mass reduction" is displayed in the area of ​​the YAM ratio from 80% to 70%, and "osteoporosis" is displayed in the area of ​​the YAM ratio lower than 70%.

[0098] Image D1 shows the bone mineral density per unit area (g / cm 2 ), bone mineral density per unit volume (g / cm 3), AGE, T-score, and / or Z-score may be displayed. For example, AGE may be displayed as a value compared with the average value of bone density for the same age or age group. The image D1 may show categories according to a predetermined standard for these values. The categories may be "normal", "bone loss", and "osteoporosis" according to these values. For example, when a T-score is displayed in the image D1, a T-score of -1.0 or more may be displayed as normal bone density, a T-score of -1.0 to -2.5 may be displayed as low bone density or osteopenia, and a T-score of -2.5 or less may be displayed as osteoporosis. In addition, when a Z-score is displayed in the image D1, a Z-score lower than -2.0 may be displayed as lower than age-appropriate, and a Z-score exceeding -2.0 may be displayed as higher than age-appropriate.

[0099] The second generating unit 427 may predict the time when the subject's bone density will reach "normal", "low bone density or osteopenia", or "osteoporosis" in terms of the YAM ratio, T score, or Z score. That is, the second generating unit 427 may generate predicted information 434 indicating the time. As an example, the predicted information 434 indicates that the subject's bone density will reach a YAM ratio of 70% or less (osteoporosis) in one year, etc.

[0100] That is, the image generating unit 424 generates a display image displaying information indicating a judgment criterion for bone density. The information indicating the judgment criterion may be information indicating a judgment criterion according to the age of the subject. The information indicating the judgment criterion may be information indicating a judgment criterion based on an original criterion or an already known guideline.

[0101] <Example 1 showing the estimated value if the subject receives treatment> Next, an example of a display image displaying an estimated value of the subject's bone density when the subject receives treatment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a display image displaying an estimated value of the subject's bone density when the subject receives treatment. Image D2 shown in Fig. 4 shows measured values ​​m1-m3 of bone density at the time of measurement, estimated values ​​n1-n2 of the subject's bone density at the time of capturing the medical image, and a transition T1 of bone density based on these measured values ​​and estimated values. In image D2, the same displays as those described in image D1 above are denoted by the same reference numerals, and the description thereof will not be repeated here.

[0102] In image D2, estimated values ​​o1, o2, and o3 are displayed. The estimated values ​​o1 and o2 displayed in image D2 are estimated values ​​in the case where the subject is not receiving treatment.

[0103] The estimated value o3 is an estimated value when the subject receives treatment. In image D2, the display of "drug therapy" is shown at the current time. The display indicates that "drug therapy" will start from now. In other words, the estimated value o3 is an estimated value six months from now when the treatment "drug therapy" is started for the subject from now.

[0104] In the example shown in Figure 4, the estimated values ​​o1 and o2, which are the estimated values ​​when the subject is not receiving treatment, are shown with open triangles, and the estimated value o3, which is the estimated value when the subject is receiving treatment, is shown with a black triangle.

[0105] According to the above-described configuration, the effectiveness of the treatment can be visually recognized by the user.

[0106] In this example, an example of a display image showing an estimated value six months from now when the subject is started on the treatment "drug therapy" has been described. On the other hand, the image may show an estimated value of the subject's bone density when the subject is treated at a time point earlier than the present, i.e., at a time point in the past.

[0107] <Example 2: Displaying the estimated value if the subject receives treatment> Next, another example of a display image displaying an estimated value of the subject's bone density when the subject receives treatment will be described with reference to FIG. 5. FIG. 5 is a diagram showing another example of a display image displaying an estimated value of the subject's bone density when the subject receives treatment. Image D3 shown in FIG. 5 shows the measurement value m1 of the bone density at the time of measurement, the measurement values ​​m11-m12, the estimated values ​​n1-n2 of the subject's bone density at the time of capturing the medical image, and the transition T2 of the bone density based on these measurement values ​​and estimated values. In image D3, the same displays as those described in the above-mentioned images D1-D2 are denoted by the same reference numerals, and the description thereof will not be repeated here.

[0108] The image D3 displays a display p1 indicating the period of treatment actually performed on the subject. The display p1 indicates the period of treatment from February 21, 2022 to the present. That is, the image generating unit 424 may generate a display image that displays information indicating the period of treatment for bone density for the subject. In addition, information indicating the contents of the treatment may be displayed in the image D3. In addition, the image D3 displays a display p2 indicating the date and time when the subject was injured. The image generating unit 424 may generate a display image that displays the display p1 and the display p2, which are information about the subject, using the subject data stored in the storage unit 43. Regarding the display p1 indicating the period of treatment and the display p2 indicating the date and time when the subject was injured, the display device 4 may accept an operation input of selecting whether the display is necessary or unnecessary.

[0109] Image D3 displays estimated values ​​o11, o12, and o13. Estimated value o11 is an estimated value when the subject receives treatment, with the start of treatment on February 21, 2021 as the base point of estimation. Estimated value o11 is an estimated value as of May 13, 2021, which is a point after the base point.

[0110] The estimated value o12 is the estimated value six months from now, based on the present, if no further treatment is provided to the subject.

[0111] The estimated value o13 is an estimated value six months from now, assuming that treatment for the subject continues.

[0112] <Example of a display image showing an estimated value at a point prior to the point in time that serves as the base point of estimation> Next, an example of a display image displaying an estimated value at a time point before the time point that is the base point of estimation will be described. Fig. 6 and Fig. 7 are diagrams showing an example of a display image displaying an estimated value of the subject's bone density at a time point before the time point that is the base point of estimation.

[0113] <Example of a display image showing the estimated bone density value from the past, with the current point as the starting point of the estimation> Image D4 shown in Fig. 6 shows the measured value m1 of the subject's bone density at the time of measurement, the estimated values ​​n1, n2, and n21 of the subject's bone density at the time of taking the medical image, and the transition T3 of the bone density based on these measured values ​​and estimated values. Image D4 also shows estimated values ​​o21, o22, and o23. In image D4 shown in Fig. 6, the same symbols are used for the displays that are the same as those described in images D1 to D3 above, and the description thereof will not be repeated here.

[0114] The estimated value o21 is an estimate as of May 13, 2022, a point after February 21, 2022, which is the base point of the estimate.

[0115] The estimated value o22 is an estimated value for a point in time six months after the current point in time.

[0116] The estimated value o23 is an estimated value as of May 13, 2021, which is a point in time before the present, with the present as the base point of the estimation. In other words, the estimated value o23 is an estimated value at a point in time earlier than the present, which is the base point of the estimation.

[0117] <Example of a display image showing the estimated bone density value from the past, with the base point of estimation set as a past time> Image D5 shown in Fig. 7 shows the subject's bone density measurement value m1 at the time of measurement, measurement value m31, the subject's bone density estimate value n31 at the time of medical image capture, and the bone density transition T4 based on these measurement values ​​and estimate values. Image D5 also shows estimated values ​​o31, o32, and o33. In image D5 shown in Fig. 7, the same symbols are used for the displays that are the same as those described in images D1 to D4 above, and the description thereof will not be repeated here.

[0118] The estimated value o31 is an estimated value as of December 21, 2021, which is a point in time prior to the base point of the estimation, with February 21, 2022 as the base point of the estimation. In other words, the estimated value o31 is an estimated value as of a point in time further back in time than the past point in time that serves as the base point of the estimation.

[0119] The estimated value o32 is an estimate as of May 13, 2022, a point prior to the present, with the present as the base point of the estimate.

[0120] The estimated value o33 is an estimated value for a point in time six months from now, with the present as the base point of the estimation.

[0121] 6 and 7, it is possible to compensate for the bone density value of the subject at a past time when no medical images were taken or no bone density measurement was performed. Also, it is possible to estimate the bone density value of the subject at a past time when the subject is first examined at a medical institution.

[0122] <Example of a display image showing transition information showing the transition of bone density in a referent who is different from the subject> Next, an example of a display image displaying information on the bone density of a subject and transition information showing the transition of the bone density of a referee different from the subject will be described. Fig. 8 is a diagram showing an example of a display image displaying information on the bone density of a subject and transition information showing the transition of the bone density of the referee.

[0123] Image D6 shown in FIG. 8 displays the subject's bone density measurement values ​​m41-m43 at the time of measurement, the subject's bone density estimate values ​​n41-n42 at the time of medical image capture, and the subject's bone density transition T5 based on these measurement values ​​and estimate values. Image D6 also displays an estimated value o41. The estimated value o41 is an estimated value of the subject's bone density six months after the current point of estimation. In image D6 shown in FIG. 8, the same displays as those described in images D1-D5 above are denoted by the same reference numerals, and the description thereof will not be repeated here.

[0124] Image D6 displays display p3 indicating a transition in bone density of a referrer different from the subject. That is, image generating unit 424 generates image D6 displaying transition information indicating a transition in bone density of a referrer different from the subject. For example, referrer data indicating a transition in bone density of the referrer may be stored in storage unit 43. Image generating unit 424 may generate a display image displaying display p3, which is information indicating a transition in bone density of the referrer, by using the referrer data stored in storage unit 43.

[0125] For example, the reference person may be a person with the same attributes as the subject. Here, the same attributes may be that at least one of age, weight, sex, disease, blood relatives, osteoporosis type, blood marker value, and Fracture Risk Assessment Tool (FRAX (registered trademark)) evaluation is equivalent or similar. The transition of the bone density of the reference person may be a comparison target for the transition of the bone density of the subject. For example, the reference person may be a person who has already received treatment for bone density, or a person who has not received treatment for bone density.

[0126] <Example of a display image showing an estimated value according to changes in the timing of starting treatment for a subject> Next, an example of a display image that displays an estimated value according to a change in the timing of starting treatment for a subject will be described.

[0127] FIG. 9 is a diagram showing an example of a display image that transitions in response to a change in the timing of starting treatment for a subject. In this example, a description is given of the transition of images when the timing of starting treatment for a subject is changed from six months later to the present by a user's operational input. As shown in FIG. 9, when the change is made, the displayed image transitions from image D7 to image D8. The timing of starting treatment may also be set to any point in time by a user's operational input.

[0128] Image D7 is an example of a display image that displays the estimated value of the subject's bone density when treatment is started for the subject in six months. Image D7 displays the measured values ​​m51-m52 of the subject's bone density at the time of measurement, the estimated values ​​n51-n52 of the subject's bone density at the time of taking the medical image, and the transition T6 of the subject's bone density based on these measured values ​​and estimated values. Image D7 also displays the estimated values ​​o51, o52, and o53. In image D7, the same displays as those described in images D1-D6 above are denoted by the same reference numerals, and the description thereof will not be repeated here.

[0129] The estimated value o51 is the estimated bone density of the subject six months from now, with the present as the base point of estimation.

[0130] The estimated value o52 is the estimated value of the subject's bone density one year from now, assuming that treatment for the subject is started six months from now. The estimated value o52, the estimated value o53 described below, the estimated value o62 in image D8, and the estimated value o64 in image D8 are shown at the end points of the connecting dotted lines, and the starting points of the dotted lines are connected to the estimated value of the subject's bone density six months from now. The starting points of the dotted lines connected to these estimated values ​​do not indicate the time points that serve as the base points of the estimation.

[0131] The estimated value o53 is the estimated bone mineral density of the subject one year from now, assuming that no treatment is initiated for the subject.

[0132] Image D8 is an example of a display image that displays the estimated value of the subject's bone density when treatment is currently started for the subject. In image D8, the same symbols are used for the displays that are the same as those described in images D1 to D7 above, and the description thereof will not be repeated here. Image D8 displays estimated values ​​o51, o52, o63, and o64.

[0133] The estimated value o61 is an estimate of the subject's bone density six months from now, assuming that treatment is not initiated for the subject.

[0134] The estimated value o62 is the estimated bone mineral density of the subject one year from now, assuming that no treatment is initiated for the subject.

[0135] The estimated value o63 is an estimate of the subject's bone density six months from now if treatment for the subject were to begin at the present time.

[0136] The estimated value o54 is the estimated bone density of the subject one year from now if treatment for the subject were to begin at the present time.

[0137] In this example, the example has been described in which the image displayed transitions from image D7 to image D8 in response to a change in the timing of starting treatment, but image D7 and image D8 may be displayed simultaneously.

[0138] A single image, for example a single graph, may display the estimated values ​​corresponding to different timings for starting treatment.

[0139] (Other examples) Next, another example of the above-mentioned images D1 to D8 will be described.

[0140] (Example of displaying the names of bones in the medical images used to estimate the estimate) The image D1 to the image D8 may display the names of bones appearing in the medical image used to estimate each estimated value. That is, the image generating unit 424 may generate a display image displaying information indicating the names of bones appearing in the medical image used to estimate the estimated information 433. The first generating unit 422 may generate the estimated information 433 from images of bones at multiple sites appearing in the medical image. The image generating unit 424 may generate a display image displaying information indicating the names of multiple bones appearing in the medical image used to estimate the estimated information 433. The names of bones may be, for example, the skull, clavicle, sternum, scapula, ribs, humerus, radius, ulna, hand bones, spine, sacrum, coccyx, femur, patella, tibia, fibula, and foot bones.

[0141] The images D1 to D8 may display the names of bones appearing in the medical images used to estimate each estimated value. That is, the image generating unit 424 may generate a display image displaying information indicating the names of bones appearing in the medical images used to estimate the estimated information 434. The second generating unit 427 may generate the estimated information 434 from images of bones at multiple sites appearing in the medical images. The image generating unit 424 may generate a display image displaying information indicating the names of multiple bones appearing in the medical images used to estimate the estimated information 434.

[0142] (Example of displaying the estimated value and the medical image used to estimate the estimated value) The images D1 to D8 may display medical images used to estimate each estimated value and to estimate each estimated value. The medical images may be, for example, medical images of the head, neck, chest, lumbar region (lumbar vertebrae (Lumbar): L1-L4, etc.), hip joints, knee joints, ankle joints, feet, toes, shoulder joints, elbow joints, wrist joints, hands, fingers, or jaw joints. The types of the captured sites of the medical images are not limited to the above. In addition, the medical images may be frontal images in which the target site irradiated with plain X-rays is captured from the front, or side images in which the target site is captured from the side. The X-ray images may be images in which at least one of cortical bone and cancellous bone is captured.

[0143] (Example of display when bone density is estimated or predicted from multiple sites) In the case where bone density is estimated or predicted from each of a plurality of parts captured in a medical image, each graph showing the transition of bone density at each part may be displayed in the image D1 to the image D8. For example, the captured image of each part used for estimating each estimated value and estimating each estimated value may be displayed superimposed on the corresponding graph. For example, the plurality of parts may be set to a region for each vertebral body. For example, the plurality of parts may be set to a region that includes a plurality of vertebral bodies. For example, the plurality of parts may be set to a region with a different area. In addition, the plurality of parts may be set to a region obtained by dividing the area of ​​a specific part of a certain part. For example, the region may be set arbitrarily according to the attribute information of the subject. For example, if the subject has a history of fracture, the area of ​​the specific part may be set to a smaller region than that of a subject without a history of fracture.

[0144] (Example of display of evaluation criteria) In a display image showing information indicating criteria for judgment such as normal, bone loss, and osteoporosis, the criteria may be displayed superimposed on the graph. The criteria may be based on the subject's age at the time of measurement, the subject's age at the time of estimation, or the subject's age at the time of prediction. The time of prediction may be, for example, the third time point mentioned above.

[0145] (Example of displaying the margin of error for a guess or the guess probability) The second generating section 427 may generate the estimated information 434 including information indicating an error range for the estimated value. The image generating section 424 may generate a display image that displays information indicating an error range for each estimated value.

[0146] In detail, the second generator 427 may generate the guess information 434 including information indicating a guess probability for the guess value or an error range for the guess value. The guess probability may be, for example, a predicted probability. At least one of the guess probability and the error range may be displayed in the above-mentioned images D1 to D8.

[0147] (Example of displaying the margin of error for an estimated value) Here, an example of a display image displaying an error range of an estimated value will be described. Fig. 10 is a diagram showing an example of a display image displaying an error range of an estimated value of a subject's bone density. In image D9 shown in Fig. 10, measured values ​​m71-m72 of the subject's bone density at the time of measurement, estimated values ​​n71-n72 of the subject's bone density at the time of taking a medical image, and a transition T7 of the subject's bone density based on these measured values ​​and estimated values ​​are displayed. In addition, estimated values ​​o71 and o72 are displayed in image D9.

[0148] The estimated value o71 is an estimated value of the subject's bone density in a predetermined period including a point six months after the base point, with the present as the base point of estimation. The estimated value o72 is an estimated value of the subject's bone density in a predetermined period including a point one year after the base point, with the present as the base point of estimation. In other words, the estimated value o71 and the estimated value o72 may be displayed as estimated values ​​of the subject's bone density in a predetermined period including a point a predetermined period has passed since the base point, with the present as the base point of estimation.

[0149] In the example shown in Fig. 10, the estimated values ​​o71 and o72 are shown by circles including the estimated values ​​and the error ranges of the estimated values. The error ranges of the estimated values ​​o71 and o72 may be displayed by error bars or box plots. The upper limits of the error ranges of the estimated values ​​o71 and o72 may be connected by a dotted line or the like. The lower limits of the error ranges of the estimated values ​​o71 and o72 may be connected by a dotted line or the like.

[0150] (Example of an estimated value range) The first generator 422 may generate the estimated information 433 including information indicating a range for each estimated value. The image generator 424 may generate a display image displaying a range for each estimated value. As a specific example of the range for the estimated value, an estimated bone mineral density range of 0.79-0.81 (median 0.80) g / cm 2 The following can be cited as an example.

[0151] (Example of displaying the guess probability of a guess value) Next, an example of a display image displaying the estimation probability of the estimated value will be described. Fig. 11 is a diagram showing an example of a display image displaying the estimation probability of the estimated value of the subject's bone density. In the image D10 shown in Fig. 11, the measured values ​​m81-m83 of the subject's bone density at the time of measurement, the estimated values ​​n81-n82 of the subject's bone density at the time of taking the medical image, and the transition T8 of the subject's bone density based on these measured values ​​and estimated values ​​are displayed. In addition, the estimated value o82 is displayed in the image D10.

[0152] The estimated value o82 is an estimated value of the subject's bone density at a point six months after the base point of estimation, with the present being the base point of estimation. Moreover, in the image D10, a display p4 indicating the estimation probability of the estimated value o82 is displayed as "69%" above the estimated value o82. That is, the second generating unit 427 may generate estimation information 434 indicating the estimation probability regarding the estimated value. Moreover, the image generating unit 424 may generate a display image displaying the estimation probability for each estimated value.

[0153] (Example of displaying the measurement results when measuring a measurement value) In the images D1 to D10, medical images captured during the measurement of each measurement value may be further displayed. The medical images may be, for example, medical images of the head, neck, chest, lumbar region (lumbar vertebrae (Lumbar): L1-L4, etc.), hip joints, knee joints, ankle joints, feet, toes, shoulder joints, elbow joints, wrist joints, hands, fingers, or jaw joints. The types of the captured sites of the medical images are not limited to the above. In addition, the medical images may be frontal images in which the target site irradiated with simple X-rays is captured from the front, or side images in which the target site is captured from the side. The X-ray images may be images in which at least one of the cortical bone and the cancellous bone is captured. In addition, the images D1 to D10 may display measurement results corresponding to the displayed measurement values. For example, the displayed measurement results may be displays showing the measurement results measured by the DXA method.

[0154] (Another example of displaying an estimate and a medical image used to estimate the estimate) In the case where images D1 to D10 further display the estimation of each estimated value and the medical images used for estimating each estimated value, the medical images may be displayed so that the areas used for the estimation can be seen.

[0155] For example, the first generating unit 422 may generate a heat map for the region used to estimate the estimated value. In this case, for example, the outer edge of the heat map may indicate the segmentation region of the region used for the estimation. The heat map may represent the magnitude of bone density with any color intensity. For example, the first generating unit 422 may generate a heat map indicating the degree of attention. Also, the first generating unit 422 may generate a heat map indicating the numerical value of bone density. Also, the first generating unit 422 may generate a heat map indicating the possibility (probability) of fracture. The display image may display an image in which the heat map is superimposed on a medical image. The image used for the heat map may be a still image or a video. By displaying it as a video, for example, by fading various heat maps in order, it becomes easy to visually recognize the relationship between each heat map. Also, when the analysis result is a heat map of bone density including areas other than the segmentation area, a part of the segmentation area may be surrounded by a frame line.

[0156] (Method of displaying medical images and measurement results) In the case where the estimation of each estimated value and the medical image used for estimating each estimated value are further displayed in the image D1 to image D10, when the mouse cursor or the like overlaps the display area of ​​the estimated value or the display area of ​​the estimated value, the medical image corresponding to the estimated value or the estimated value may be displayed. Here, the corresponding medical image may be the medical image used for estimating the estimated value or the estimated value. For example, the medical image may be displayed so that the area in the medical image used for estimating the estimated value or the estimated value can be understood. In addition, when the mouse cursor or the like overlaps the display area of ​​the measured value, the measurement result corresponding to the measured value may be displayed. For example, the displayed measurement result may be a display showing the measurement result measured by the DXA method. The display of these medical images or the display of the measurement result may be displayed in a balloon from the display area of ​​the measured value or the display area of ​​the estimated value overlapped by the mouse cursor or the like. In addition, when a click operation is performed on the display area of ​​the estimated value, the estimated value, or the measured value, a display image including the above-mentioned medical image or the measurement result may be displayed.

[0157] An example of a display image displaying a medical image corresponding to the estimated value and the measured value will be described. FIG. 12 is a diagram showing an example of a display image displaying a medical image used to estimate the estimated value and a measurement result corresponding to the measured value. In FIG. 12, the same display as that described in FIG. 3 above is denoted by the same reference numerals, and the description thereof will not be repeated here. The image D11 shown in FIG. 12 includes a display P6 showing a graph showing the transition T1 of bone density, a display P7 which is a display of a medical image corresponding to the estimated value n2, and a display P8 which is a display of a measurement result corresponding to the measured value m3. As shown in FIG. 12, when the mouse cursor P5a is placed on the display area of ​​the estimated value n2, the image D11 may display a display P7 which is a display of a medical image corresponding to the estimated value n2. For example, when the operation receiving unit 423 receives an operation of placing the mouse cursor P5a on the display area of ​​the estimated value n2, the image generating unit 424 generates a display image including the display P7, and the display control unit 426 causes the display unit 45 to display the display image. Furthermore, as shown in FIG. 12, when the mouse cursor P5b is placed over the display area of ​​the measurement value m3, a display P8 of the measurement result corresponding to the measurement value m3 may be displayed.

[0158] Fig. 13 is a diagram showing another example of the display of a medical image corresponding to the estimated value n2. In this example, when the mouse cursor P5a is placed over the display area of ​​the estimated value n2 shown in Fig. 12, a display P7a is displayed, which is a display of a medical image corresponding to the estimated value n2. Display P7a displays a display P710, which is a frame showing the area in the medical image used to estimate the estimated value n2.

[0159] (Example of displaying the progression of bone density in multiple areas) The display image displayed on the display unit 45 may display the transition of bone density of each of the multiple sites captured in the medical image. An example of a display image displaying the transition of bone density of each of the multiple sites will be described. FIG. 14 is a diagram showing an example of a display image displaying the transition of bone density of each of the multiple sites. In FIG. 14, the same displays as those described in FIG. 3 above are denoted with the same reference numerals, and the description thereof will not be repeated here. The image D12 shown in FIG. 14 includes a display P11 showing a graph showing the transition of bone density T1 and a display P9 which is a display of a medical image. The display P9 of the medical image displays displays P91 to P94 which are frame lines showing the region in which the lumbar vertebrae L1-L4 are imaged. For example, as shown in FIG. 14, when the mouse cursor P10 is positioned within the display P91 which is the display region of the lumbar vertebrae L1, an image showing the transition of bone density in the lumbar vertebrae L1 may be displayed. In addition, a display image displaying the transition of bone density of each of the lumbar vertebrae L1-L4 at the same time may be displayed. In such a display image, for example, when the mouse cursor P10 is positioned within a display P91 that is a display region of the lumbar vertebra L1, the transition of the bone density in the lumbar vertebra L1 may be displayed in an emphasized manner.

[0160] (Example of displaying the predicted location of a fracture as a prediction result) The operation accepting unit 423 may accept an operation input instructing a guess regarding the subject's fracture. When the operation accepting unit 423 accepts the operation input, it instructs the second generating unit 427 to generate guess information 434 indicating a guess regarding the subject's fracture. Furthermore, the operation accepting unit 423 instructs the image generating unit 424 to generate a display image displaying the guess regarding the subject's fracture indicated by the guess information 434. The operation accepting unit 423 accepts an operation input instructing to end the display. When the operation accepting unit 423 accepts the operation input, it instructs the display control unit 426 to end the display.

[0161] Other Examples of the Second Generation Unit 427 The second generating unit 427 included in the control unit 42 may perform the following process. The second generating unit 427 generates inferred information 434 including information on the possibility of a future fracture of the subject inferred from a medical image indicated by the medical image data 432 stored in the storage unit 43. For example, the second generating unit 427 may generate inferred information 434 including information on the possibility of a future fracture inferred from a medical image showing the bones of the subject at a first time point. The inferred information 434 may include at least any of information indicating, for example, the probability of a future fracture of the subject, the time when the fracture is estimated to occur, and the location where the fracture will occur.

[0162] The second generating unit 427 may generate inferred information 434 including information on the subject's fracture inferred from a medical image showing the subject's bones at the second time point. The inferred information 434 may include information on the subject's fracture at a fourth time point. The fourth time point may be a time point different from the second time point and the third time point.

[0163] When the image generating section 424 generates an image using the estimated information 434, the image generating section 424 may generate a display image that displays the estimated information 434 associated with the fourth time point.

[0164] <Another example of the speculative model 4271> The estimation model 4271 included in the second generating unit 427 according to this example may perform the following processing. The estimation model 4271 may estimate, from a medical image indicated by the medical image data 432 of the subject, information on the possibility of the subject having a future fracture at a time point different from the time point at which the medical image was captured. The estimation model 4271 may be set based on learning data including a medical image (fourth image) showing the bones of the third person, and teacher data including information on the fracture of the third person. The above-mentioned "third person" may not be particularly limited to a human. For example, it may be an animal of the same species as the "subject."

[0165] For example, information on fractures of a person having a bone shown in a training plain X-ray image shown by each of a plurality of training image data included in the training data is associated as training data. The training data may be at least one of bone evaluation information and a medical image (fifth image) showing the fractured part of a third person. The fifth image may be an image showing the same part as the fourth image. The fifth image may be an image showing a different part from the fourth image.

[0166] The teacher data may be measured at a time different from the time when the learning image data was captured. The teacher data may be measured at approximately the same time when the learning image data was captured.

[0167] The estimation model 4271 may be composed of a plurality of models. For example, the estimation model 4271 may be composed of a model for estimating the bone density of a subject at a time point different from the time point when the medical image was captured from a medical image, a model for estimating the possibility of a future fracture of a subject at a time point different from the time point when the medical image was captured from a medical image, etc. The estimation results output from the plurality of models may be displayed together in the display image displayed on the display unit 45, or only one of them may be displayed.

[0168] The estimated information 434 may be a method of showing different types of information from the first method and the second method. For example, the first method may be a method of showing information related to bone density, and the second method may be a method of showing information related to fracture. For example, the measurement value of the subject's bone density and the estimated value of the subject's bone density may be shown in the same display form in the display image displayed on the display unit 45. Information related to the subject's fracture may be shown in a display form different from the display form of the measurement value of the subject's bone density and the estimated value of the subject's bone density in the display image displayed on the display unit 45.

[0169] The display image displayed on the display unit 45 may display a location where a fracture is likely to occur in the future, which is indicated by the inference information 434. Fig. 15 is a diagram showing an example of a display indicating a location where a fracture is likely to occur, which is included in a display image displayed on the display unit 45. In the example shown in the left diagram of Fig. 15, the display image displayed on the display unit 45 includes a display P13 which is a display of an X-ray image. The display P13 includes a display P131 which is a frame line indicating an area where a fracture is likely to occur in the future.

[0170] In the example shown in the right diagram of Fig. 15, the display image displayed on the display unit 45 includes display P14, which is a display of an X-ray image. Display P14 includes display P14, which is a frame line indicating an area used to estimate bone density, and display P141 and display P142, which are frames indicating an area where a fracture is likely to occur in the future. For example, a notation such as "prediction" may be displayed around display P141 and display P142, which are frame lines indicating an area where a fracture is likely to occur in the future.

[0171] [Embodiment 2] Another embodiment of the present disclosure will be described below with reference to Fig. 16. For ease of explanation, the same reference numerals will be given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0172] Fig. 16 is a block diagram showing a configuration of a main part of an example of an information processing system 1a according to this embodiment. As shown in Fig. 16, the information processing system 1a includes a measuring device 2, an imaging device 3, a display device (image generating device) 4a, a first generating device 4220a, and a second generating device 42270a. Each device communicates with each other by wire or wirelessly. In addition, the communication may be performed via a network 5a as shown in Fig. 16.

[0173] The first generation device 4220a performs the same processing as the processing performed by the first generation unit 422 described in the embodiment 1. The first generation device 4220a transmits the generated estimated information 433 to the display device 4 via the communication unit 4221a.

[0174] The second generating device 4270a performs the same process as the process performed by the second generating unit 427 described in the first embodiment. The second generating device 4270a receives an instruction to generate the inferred information 434 via the communication unit 4271a. In addition, the second generating device 4270a transmits the generated inferred information 434 to the display device 4 via the communication unit 4271a.

[0175] The display device 4a includes a communication unit 41a, a control unit 42a, a storage unit 43a, an operation input unit 44, and a display unit 45. The acquisition unit 421a of the control unit 42a receives the measurement information 431, the estimated information 433, and the inferred information 434 via the communication unit 41a. The display device 4a performs the image generation process described in the first embodiment. When the operation reception unit 423a receives an instruction to receive an operation input instructing an inference regarding the bones of the subject, the display device 4a transmits an instruction to generate the inferred information 434 to the second generation device 4270a via the communication unit 41a. The operation reception unit 423a receives the same operation as the operation reception unit 423. An operation input device that performs the processing of the operation input unit 44 and the operation reception unit 423a may be included in the information processing system 1a as a device different from the display device 4a. In this case, the operation input device may communicate with the display device 4a, the second generation device 4270a, and the like via the network 5a.

[0176] [Embodiment 3] Another embodiment of the present disclosure will be described below with reference to Fig. 17. For ease of explanation, the same reference numerals will be given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0177] Fig. 17 is a block diagram showing a main configuration of an example of an information processing system 1b according to this embodiment. As shown in Fig. 17, the information processing system 1b includes a measuring device 2, an imaging device 3, a display device 4b, a first generating device 4220a, a second generating device 42270a, and a server device (image generating device) 6b. Each device communicates with each other by wire or wirelessly. In addition, the communication may be performed via a network 5a as shown in Fig. 17. In addition, the server device 6b may be a cloud server.

[0178] The first generation device 4220a performs the same process as the process performed by the first generation unit 422 described in the embodiment 1. The first generation device 4220a transmits the generated estimated information 433 to the server device 6b via the communication unit 4221a.

[0179] The second generating device 4270a performs the same process as the process performed by the second generating unit 427 described in the first embodiment. The second generating device 4270a receives an instruction to generate the inferred information 434 via the communication unit 4271a. In addition, the second generating device 4270a transmits the generated inferred information 434 to the server device 6b via the communication unit 4271a.

[0180] The server device 6b includes a communication unit 61b, a control unit 62b, and a storage unit 63b. The acquisition unit 621b of the control unit 62b stores the measurement information 431, the estimation information 433, and the inference information 434 received via the communication unit 61b in the storage unit 63b. The information acquisition unit 625b and the image generation unit 624b of the control unit 62b perform image generation processing similarly to the information acquisition unit 425 and the image generation unit 424 described in the first embodiment. The server device 6b transmits the generated image data 430b to the display device 4b via the communication unit 61b.

[0181] The display device 4b includes a communication unit 41b, a control unit 42b, a storage unit 43b, an operation input unit 44, and a display unit 45. When the operation reception unit 423b receives an instruction to receive an operation input instructing to make a guess regarding the bones of the subject, the display device 4b transmits an instruction to generate the guess information 434 to the second generation device 4270a via the communication unit 41b. The operation reception unit 423b receives the same operation as the operation reception unit 423. The acquisition unit 421b of the control unit 42b acquires the image data 430b from the server device 6b via the communication unit 41b. The acquisition unit 421b stores the acquired image data 430b in the storage unit 43b, and transmits a signal indicating that the image data 430b has been stored to the display control unit 426b. When the display control unit 426b receives the signal from the acquisition unit 421b, it acquires the image data 430b from the storage unit 43b and displays the image indicated by the image data 430b on the display unit 45.

[0182] [Software implementation example] The functions of the display device (4, 4a, 4b), the first generating device (4220a), the second generating device (4270a), and the server device (6b) (hereinafter referred to as the "devices") can be realized by a program for causing a computer to function as the device, and a program for causing a computer to function as each control block of the device (particularly each part included in the control unit).

[0183] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program to realize each function described in each of the above embodiments.

[0184] The program may be non-transitory and may be recorded in one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

[0185] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present disclosure. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0186] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may be executed by the control device or another device (for example, an edge computer or a cloud server).

[0187] The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to the above-mentioned embodiments. In other words, the invention according to the present disclosure can be modified in various ways within the scope of the present disclosure, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments are also included in the technical scope of the invention according to the present disclosure. In other words, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. It should also be noted that these modifications or corrections are included in the scope of the present disclosure.

[0188] 〔summary〕 An information processing system according to aspect 1 of the present disclosure includes a first acquisition unit that acquires first information indicating information about the subject's bones obtained based on a first method, a second acquisition unit that acquires second information indicating information about the subject's bones obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0189] In the information processing system according to aspect 2 of the present disclosure, in aspect 1, the first information and the second information may be information regarding at least one of bone density, bone mass, and bone quality of the subject.

[0190] An information processing system according to aspect 3 of the present disclosure, in aspect 2, may be such that the first acquisition unit acquires the first information measured at a first time point, the second acquisition unit acquires the second information indicating at least one of bone density, bone mass and bone quality estimated by an estimation model from a first image showing the subject's bones at a second time point different from the first time point, and the generation unit generates the display image in which the first information and the second information are arranged in chronological order of the first time point and the second time point.

[0191] In an information processing system according to aspect 4 of the present disclosure, in aspect 3, the estimation model may be set based on learning data including a second image showing a bone of a first person, and teacher data including at least one of bone density, bone mass, and bone quality of the first person.

[0192] The information processing system of aspect 5 of the present disclosure, in aspect 3 or 4, further includes a third acquisition unit that acquires estimated information regarding at least one of bone density, bone mass, and bone quality estimated by a prediction model from an image showing the subject's bones at the first time point or the second time point, and when the image showing the subject's bones is an image showing the subject's bones at the first time point, the estimated information is information at a third time point different from the first time point, and when the image showing the subject's bones is an image showing the subject's bones at the second time point, the estimated information is information at a third time point different from the second time point, and the generation unit may generate the display image displaying the estimated information associated with the third time point.

[0193] In an information processing system of aspect 6 of the present disclosure, in aspect 5, the prediction model may be set based on learning data including a third image showing the bones of a second person and teacher data including at least one of bone density, bone mass, and bone quality of the second person.

[0194] An information processing system according to aspect 7 of the present disclosure, in aspect 6, is characterized in that the estimated information is information regarding the bone density of the subject when the subject is treated for at least one of bone density, bone mass and bone quality, the second person is a person who has received treatment for at least one of bone density, bone mass and bone quality, and the estimated model may be set based on learning data including the third image showing the bones of the second person who has received the treatment and teacher data including at least one of the bone density, bone mass and bone quality of the second person.

[0195] In an information processing system according to aspect 8 of the present disclosure, in any of aspects 5 to 7, when the image showing the subject's bones is an image showing the subject's bones at the first time point, the third time point may be a time point earlier than the first time point or a time point later than the first time point, and when the image showing the subject's bones is an image showing the subject's bones at the second time point, the third time point may be a time point earlier than the second time point or a time point later than the second time point.

[0196] In an information processing system of aspect 9 of the present disclosure, in any of aspects 3 to 8, the generation unit may generate the display image displaying information indicating the names of bones appearing in the first image used to estimate the second information.

[0197] In the information processing system according to aspect 10 of the present disclosure, in any one of aspects 1 to 9, the generation unit may generate the display image showing the second information in a display manner different from that of the first information.

[0198] In an information processing system according to aspect 11 of the present disclosure, in any one of aspects 1 to 10, the generation unit may generate the display image displaying information indicating the period of treatment for at least one of bone density, bone mass, and bone quality for the subject.

[0199] In an information processing system according to aspect 12 of the present disclosure, in any one of aspects 1 to 11, the generation unit may generate the display image displaying trend information showing a trend in at least one of bone density, bone mass, and bone quality of a reference person other than the subject.

[0200] In an information processing system according to aspect 13 of the present disclosure, in any one of aspects 1 to 12, the generation unit may generate the display image displaying information indicating judgment criteria for at least one of bone density, bone mass, and bone quality.

[0201] In the information processing system according to a fourteenth aspect of the present disclosure, in any one of the first to thirteenth aspects, the generation unit may generate the display image including an indication of the first technique and the second technique.

[0202] An information processing system according to aspect 15 of the present disclosure, in any of aspects 1 to 14, may include a measuring device that performs measurements on a bone, and the first information may include information measured using the measuring device that performs measurements on the bone.

[0203] An information processing system according to aspect 16 of the present disclosure, in any of aspects 3 to 9, may include an imaging device that captures an image showing a bone, and the first image may be an image captured using the imaging device.

[0204] An information processing method according to aspect 17 of the present disclosure includes a first acquisition step of acquiring first information indicating information regarding the subject's bones obtained based on a first method, a second acquisition step of acquiring second information indicating information regarding the subject's bones obtained based on a second method different from the first method, and a generation step of generating a display image displaying the first information and the second information.

[0205] An image generating device according to aspect 18 of the present disclosure includes a first acquisition unit that acquires first information indicating information regarding the subject's bones obtained based on a first method, a second acquisition unit that acquires second information indicating information regarding the subject's bones obtained based on a second method different from the first method, and a generation unit that generates a display image displaying the first information and the second information.

[0206] An image generation program of aspect 19 of the present disclosure is an image generation program for causing a computer to function as the information processing system described in aspect 18, the image generation program causing a computer to function as the first acquisition unit, the second acquisition unit, and the generation unit. [Explanation of symbols]

[0207] 1, 1a, 1b Information Processing System 4, 4a Display device (image generating device) 6b Server device (image generating device) 421, 421a, 621b Acquisition part (1st acquisition part) 424, 624b Image generation unit (generation unit) 425, 625b Information acquisition unit (1st acquisition unit, 2nd acquisition unit, 3rd acquisition unit) 431 Measurement information (first information) 433 Estimated Information (Second Information) 434 Speculation 4221 Estimation Model 4271 Inference Model S2 First acquisition step S3 Second acquisition step S4 Generation Step

Claims

1. a first acquisition unit that acquires first information indicating information about the subject's bones obtained based on a first method; a second acquisition unit that acquires second information indicating information about the subject's bones obtained based on a second method different from the first method for the subject; A generation unit that generates a display image that displays the first information and the second information.

2. The first information and the second information are information regarding at least one of bone density, bone mass, and bone quality of the subject. The information processing system according to claim 1 .

3. The first acquisition unit acquires the first information measured at a first time point, The second acquisition unit acquires the second information estimated by an estimation model from a first image in which a bone of the subject is captured at a second time point different from the first time point, and The generation unit generates the display image in which the first information and the second information are arranged in chronological order of the first time point and the second time point. The information processing system according to claim 2 .

4. The estimation model is set based on learning data including a second image showing a bone of a first person, and teacher data including at least one of bone density, bone mass, and bone quality of the first person. The information processing system according to claim 3 .

5. a third acquisition unit that acquires estimated information on at least one of bone density, bone mass, and bone quality estimated by an estimation model from an image showing the bones of the subject at the first time point or the second time point; When the image showing the bones of the subject is an image showing the bones of the subject at the first time point, the estimated information is information at a third time point different from the first time point, When the image showing the bones of the subject is an image showing the bones of the subject at the second time point, the estimated information is information at a third time point different from the second time point, The generation unit generates the display image that displays the estimated information associated with the third time point. The information processing system according to claim 3 .

6. The estimation model is set based on learning data including a third image showing a bone of a second person, and teacher data including at least one of bone density, bone mass, and bone quality of the second person.

6. The information processing system according to claim 5.

7. the estimated information is information about the bone density of the subject when the subject is treated with at least one of bone density, bone mass, and bone quality; The second person has received treatment for at least one of bone density, bone mass, and bone quality; The estimation model is set based on learning data including the third image showing the bones of the second person who has received the treatment, and teacher data including at least one of bone density, bone mass, and bone quality of the second person.

7. The information processing system according to claim 6.

8. When the image showing the bones of the subject is an image showing the bones of the subject at the first time point, the third time point is a time point before the first time point or a time point after the first time point, When the image showing the bones of the subject is an image showing the bones of the subject at the second time point, the third time point is a time point before the second time point or a time point after the second time point.

6. The information processing system according to claim 5.

9. The generation unit generates the display image displaying information indicating names of bones appearing in the first image used for estimating the second information. The information processing system according to claim 3 .

10. The generation unit generates the display image showing the second information in a display manner different from that of the first information. The information processing system according to claim 1 .

11. The generating unit generates the display image displaying information indicating a period of treatment for at least one of bone density, bone mass, and bone quality for the subject. The information processing system according to claim 1 .

12. The generating unit generates the display image displaying transition information showing a transition of at least one of bone density, bone mass, and bone quality of a referent different from the subject. The information processing system according to claim 1 .

13. The generating unit generates the display image displaying information indicating a judgment criterion for at least one of bone density, bone mass, and bone quality. The information processing system according to claim 1 .

14. The generating unit generates the display image including an indication of the first technique and the second technique. The information processing system according to claim 1 .

15. a measurement device for performing measurements on bone; The second information includes information measured using the measurement device. The information processing system according to claim 1 .

16. An imaging device is provided for capturing an image showing a bone, The first image is an image captured by the imaging device. The information processing system according to claim 3 .

17. a first acquisition step of acquiring first information indicating information about the subject's bones obtained based on a first technique; a second acquisition step of acquiring second information indicating information about the subject's bones obtained based on a second method different from the first method for the subject; a generating step of generating a display image displaying the first information and the second information; An information processing method comprising:

18. a first acquisition unit that acquires first information indicating information about the subject's bones obtained based on a first method; a second acquisition unit that acquires second information indicating information about the subject's bones obtained based on a second method different from the first method for the subject; a generation unit that generates a display image that displays the first information and the second information.

19. 20. An image generating program for causing a computer to function as the image generating apparatus according to claim 18, the image generating program causing a computer to function as the first acquisition unit, the second acquisition unit, and the generation unit.

Images