Information processing system and information processing method

Abstract

An information processing system is provided with: an information acquisition unit that acquires information relating to a bone of a subject based on a first technique and information relating to a bone of a subject based on a second technique different from the first technique; and an image generation unit that generates a display image on which the first information and the second information are displayed.

Description

Technical Field

[0001] This disclosure relates to an information processing system and method for assisting in the understanding of bone condition. Background Technology

[0002] In recent years, there have been proposals to use computer calculations to estimate or predict information about a subject's body. For example, Patent Document 1 proposes a method for predicting the future bone mineral density of postmenopausal women using a machine learning model.

[0003] Prior art literature

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-200788 Summary of the Invention

[0006] The information processing system disclosed herein includes: a first acquisition unit that acquires first information representing bone-related information of a subject obtained based on a first technique; and a generation unit that generates a display image that displays the first information together with information related to the first technique.

[0007] Furthermore, one aspect of the information processing method disclosed herein includes: a first acquisition step of acquiring first information representing bone-related information of a subject obtained based on a first technique; a second acquisition step of acquiring second information representing bone-related information of the subject obtained based on a second technique different from the first technique; and a generation step of generating a display image displaying the first information and the second information.

[0008] Furthermore, one aspect of the server apparatus disclosed herein includes: a first acquisition unit that acquires first information representing bone-related information of a subject obtained based on a first method; a second acquisition unit that acquires second information representing bone-related information of the subject obtained based on a second method different from the first method; and a generation unit that generates a display image displaying the first information and the second information.

[0009] The display device, first generation device, second generation device, and third generation device involved in the various embodiments of this disclosure can also be implemented by a computer. In this case, the display device, the control program of the display device, the second generation device, and the third generation device, and the computer-readable recording medium on which they are recorded, are also within the scope of this disclosure, by making the computer operate as each part (software element) of the display device, the first generation device, the second generation device, and the third generation device. Attached Figure Description

[0010] Figure 1 This is a block diagram showing the main structural components of an example of the information processing system according to Embodiment 1.

[0011] Figure 2 This is a flowchart illustrating an example of the display processing flow in the display device according to Embodiment 1.

[0012] Figure 3 This is an example of a display image showing the estimated value of the bone density of an object at a time point after the time point that serves as the base point for estimation, as described in Embodiment 1.

[0013] Figure 4 This is another example of a display image showing the estimated value of the bone density of an object at a time point after the time point that serves as the base point for estimation, as described in Embodiment 1.

[0014] Figure 5 This is another example of a display image showing the estimated value of the bone density of an object at a time point after the time point that serves as the base point for estimation, as described in Embodiment 1.

[0015] Figure 6 This is an example of a display image showing the estimated value of the bone density of an object at a time point prior to the time point that serves as the base point for estimation, as described in Embodiment 1.

[0016] Figure 7 This is another example of a display image showing the estimated value of the bone density of an object at a time point prior to the time point that serves as the base point for estimation, as described in Embodiment 1.

[0017] Figure 8 This is a diagram illustrating an example of a display image showing information related to the bone density of the subject and the shift in bone density of the reference, as described in Embodiment 1.

[0018] Figure 9 This is an example of a display image showing a predicted value based on the change in the timing of the start of treatment on the subject, as described in Embodiment 1.

[0019] Figure 10 This is an example of a display image showing the error range of the estimated value of the bone density of the display subject according to Embodiment 1.

[0020] Figure 11 This is an example of a display image showing the estimated probability of the estimated value of the bone density of the display subject according to Embodiment 1.

[0021] Figure 12This is a diagram illustrating an example of a medical image used in estimating the estimated value according to Embodiment 1, and a display image of the measurement result corresponding to the measured value.

[0022] Figure 13 This is a diagram illustrating an example of displaying a medical image corresponding to the estimated value according to Embodiment 1.

[0023] Figure 14 This is an example of a display image showing the shift in bone density at multiple sites, as described in Embodiment 1.

[0024] Figure 15 This is a diagram illustrating an example of a display image shown on the display unit that shows a region where a fracture is highly likely to occur, as described in Embodiment 1.

[0025] Figure 16 This is a block diagram showing the main structural components of an example of the information processing system according to Embodiment 2.

[0026] Figure 17 This is a block diagram showing the main structural components of an example of the information processing system according to Embodiment 3.

[0027] Figure 18 This is a block diagram showing the main structural components of an example of the information processing system involved in Embodiment 4.

[0028] Figure 19 This is an example of a display image showing the fracture risk of the subject according to Embodiment 4.

[0029] Figure 20 This is a block diagram showing the main structural components of an example of the information processing system according to Embodiment 5.

[0030] Figure 21 This is a block diagram showing the main structural components of an example of the information processing system according to Embodiment 6. Detailed Implementation

[0031] [Implementation Method 1]

[0032] The following is for reference. Figures 1 to 15 An embodiment of this disclosure will be described in detail below.

[0033] (summary)

[0034] First, use Figure 1 This section will provide an overview of the implementation method. Figure 1 This is a block diagram illustrating the main structural components of an example of the information processing system 1 according to this embodiment. For example... Figure 1As shown, the information processing system 1 includes a measuring device 2, a camera device 3, and a display device (image generating device) 4.

[0035] In this embodiment, the measuring device 2 measures the bone of the subject and sends first information related to the subject's bone to the display device 4. That is, the display device 4 acquires first information representing bone-related information of the subject obtained from the bone measurement. Here, the first information may be information related to at least one of the subject's bone mineral density, bone mass, and bone quality. Furthermore, information related to the first method may use measured values, estimated values, and inferred values, for example. Furthermore, the first information may be information representing the measured value of the subject's bone mineral density. In this embodiment, the first information is described as an example of measurement information 431 representing the measured value of the subject's bone mineral density, but the description of "bone mineral density" related to "first information" in this disclosure may also be appropriately interpreted as at least one of "bone mineral density," "bone mass," and "bone quality."

[0036] In this specification, "measurement" can also refer to "actual measurement". Furthermore, "measured value" can also refer to "actual value".

[0037] In the following description, the application of the surgical treatment method is illustrated using a human subject (i.e., the "subject"), but the subject is not limited to humans. That is, the "subject" involved in this disclosure can also be mammals other than humans, such as equines, felines, canines, bovines, or swine. Furthermore, if the following embodiments are applicable to these animals, this disclosure also includes embodiments in which the "subject," "patient," and "human" are replaced with "animal."

[0038] The first piece of information may include information indicating the name of the disease suffered by the subject. This information may also indicate at least one of osteoporosis, rheumatism, osteonecrosis (e.g., femoral head necrosis), systemic sclerosis, kidney disease, and marbled bone disease. Furthermore, the first piece of information may also include bone assessment information. Bone assessment information may also include information obtained through a fracture risk assessment tool (FRAX (registered trademark); Fracture Risk Assessment Tool).

[0039] The bone mineral density mentioned above can be any value related to bone density. Bone mineral density can be measured, for example, by the density of bone salts per unit area (g / cm³). 2 Bone salt density per unit volume (g / cm³) 3Bone mineral density (BMD) is expressed using at least one of the following methods: YAM (%), AGE (%), T-score, and Z-score. YAM is short for "Young Adult Mean," also known as the percentage of young adults, %YAM, YAM ratio, etc. AGE is a value compared to the average BMD of the same age or generation. The T-score is a value calculated by comparing the measured BMD to the average of young adults and dividing by the standard deviation of young adults. Furthermore, the Z-score is a value calculated by comparing the measured BMD to the average of the subject's age and dividing by the standard deviation of the age group. BMD is not limited to these examples; it can also be a single numerical value.

[0040] The aforementioned bone mass is an indicator related to bone mineral density, which is the amount of bone tissue within the skeletal structure.

[0041] As mentioned above, bone quality can be assessed using indicators based on at least one of the statistical properties of bone, the morphological properties of bone, the mechanical properties of bone, and the chemical properties of bone. Bone quality may also include information related to the attributes of the subject, as described later. For example, bone quality can be assessed using indicators based on at least one of bone metabolism markers, sex, race, presence or absence of menopause, age, state of cortical bone, state of spongy bone, state of trabecular bone in spongy bone, disease information, bone evaluation information, medication information, presence or absence of fractures, number of fractures, location of fractures, and fracture history. More specifically, bone quality can be assessed using at least one of bone formation markers, bone resorption markers, bone quality markers (e.g., vitamin K levels), cortical bone thickness, trabecular bone density, trabecular bone orientation, and trabecular bone score. Furthermore, bone quality can also be bone age. Bone age can be assessed using indicators that express the biological maturity of bone in terms of age. Bone age can be determined, for example, from medical images of the hand (e.g., X-ray images) based on the degree of ossification.

[0042] The camera device 3 sends the medical image data 432, representing the medical image of the subject, to the display device 4.

[0043] Medical images can utilize at least one image from X-ray, MRI (magnetic resonance imaging), CT (computed tomography), PET (positron emission tomography), and ultrasound. Medical images may include, for example, images of at least a portion of the neck, chest, lower back, proximal femur, knee, ankle, shoulder, elbow, wrist, finger, or temporomandibular joint. X-ray images may also show areas other than bones. For example, a simple chest X-ray may include images of the lungs and thoracic vertebrae. X-ray images can be frontal or lateral views of the affected area. Multiple images can be used in a medical setting. For example, multiple X-ray images or a combination of X-ray and CT images can be used. When using multiple X-ray images, multiple frontal images, frontal and lateral images, or images of different areas can be used.

[0044] Display device 4 uses the medical image to estimate second information related to the subject's bones. Here, the second information may be information related to at least one of the subject's bone mineral density, bone mass, and bone quality. Furthermore, the second information may be information representing at least one estimated value of the subject's bone mineral density, bone mass, and bone quality. That is, display device 4 acquires second information representing bone-related information of the subject obtained using a second method different from the first method. Here, the first method may also be a measurement of the subject's bones. Furthermore, the second method may also be an estimation of the subject's bones.

[0045] The first and second methods differ in that they may include, but are not limited to, methods differing in at least one of the measurement site, calculation method, calculation conditions, measurement method, measurement device, and measurement conditions. For example, the first method may be a measurement performed in a hospital equipped with a measuring device for measuring bone. The second method may be an inference based on medical images. The medical images may be taken in a hospital without a measuring device or in a hospital with a measuring device. The first and second methods may, for example, be performed in different hospitals. The first and second methods may, for example, use measured information, inferred information, and speculative information, but are not limited to these. The measured information may, for example, be a disease name confirmed by a physician. The inferred or predicted information may be a disease name inferred or predicted based on various information.

[0046] In this embodiment, the second information is described as an example of estimated information 433 representing the estimated value of the subject's bone density, but the description of "bone density" related to "second information" in this disclosure may be appropriately interpreted as at least one of "bone density", "bone mass" and "bone quality".

[0047] The following are other examples of information representing the first information, which is bone-related information about the subject obtained from bone-based measurements, and the second information, which is bone-based inference.

[0048] This indicates that the bone primarily comprises the vertebrae or intervertebral discs. In this case, as the first and second pieces of information, examples include information representing at least one of the following: vertebral body height, intervertebral distance, anterior intervertebral disc cavity distance, posterior intervertebral disc cavity distance, postural malformation classification, ossification occupancy rate (the ratio of the thickness of the ossified ligaments to the anteroposterior diameter of the spinal canal), residual effective anteroposterior diameter (the thickness between the anteroposterior diameter of the spinal canal and the ossified ligaments), posterior vertebral slippage distance (the distance between the posteroinferior angle of the superior vertebral body and the anterior end of the vertebral arch of the inferior vertebra), and Meyerding classification.

[0049] Examples of diseases that can be diagnosed using the above information include: degenerative spondylitis, vertebral fractures (including traumatic and osteoporotic), spinal stenosis, hernia, degenerative spondylolisthesis, scoliosis, degenerative diseases of the thoracic / lumbar spine, bilateral congenital hip dislocation, and ossification of the spinal ligaments.

[0050] This describes the period during which the bone primarily comprises the femur. In this case, as the first and second pieces of information, for example, information representing at least one of the following can be cited: joint cleft area, joint cleft distance (width), minimum joint cleft distance (width), CE angle (Center Edge angle: the angle formed by a vertical line passing through the center of the bone and a line connecting the center of the bone and the lateral edge of the acetabulum), Sharp angle (Acetabular Angle: the angle formed by a line connecting the lateral edge of the acetabulum and the anterior end of the teardrop and a line connecting the two teardrops), ARO (Acetabular Roof Obliquity: the angle formed by a horizontal line passing through the floor of the acetabulum and a line connecting the lateral edge of the acetabular cup and the floor of the acetabular cup), AHI (Acetabular Head Index: the value obtained by dividing the distance from the medial end of the femoral head to the lateral end of the acetabulum by the transverse diameter of the femoral head), and Kellgren-Lawrence (KL) classification. Information 1 and information 2 can use at least one of measured values, estimated values, and speculative values.

[0051] Diseases that can be diagnosed using the above information include, for example, osteoarthritis of the femur (OA), femoral impingement (one of the pathological forms of OA), hip dysplasia, hip dislocation (including congenital dislocation), femoral spondylolisthesis, and femoral head necrosis.

[0052] This describes a situation where the bone primarily comprises the bones of the knee. In this case, as first and second information, information such as the joint cleft area, joint cleft distance (width), minimum joint cleft distance (width), osteophyte area, tibiotibial angle (lateral tibiotibial angle), Osteoarthritis Research Society International (OARSI) classification, and Kellgren-Lawrence (KL) classification can be provided. Osteoarthritis of the knee (OA) can be diagnosed, for example, using the aforementioned information.

[0053] This describes the condition where the bone primarily comprises the bones of the foot. In this case, the first and second pieces of information may be information representing at least one of the talus's tilt angle and the distance the talus extends anteriorly. Diseases that can be diagnosed using the above information include, for example, foot joint contusions, talus necrosis, arthritis, and foot impingement syndrome.

[0054] Display device 4 generates a display image showing the first information and the second information. In this embodiment, an example will be described of display device 4 generating a display image showing the measured value of the subject's bone mineral density represented by measurement information 431 and the estimated value of the subject's bone mineral density represented by estimation information 433.

[0055] According to the aforementioned structure, the information processing system 1 can generate display images showing bone-related information of the subject obtained based on multiple different methods, such as measurement and estimation. That is, in a medical setting, even without regularly measuring the subject's bones using a given measuring device, the progression of the subject's bone condition can be understood using methods different from measurement, such as estimation. For example, the progression of the subject's bone condition can be the progression of at least one of the subject's bone density, bone mass, and bone quality.

[0056] Furthermore, the first and second pieces of information can be information about the same item representing the state of the bone, or they can be different items. For example, the first and second pieces of information can be information about items that are in a corresponding relationship. Specifically, the first and second pieces of information can also be information about items that are correspondingly established through unit conversions or transformations.

[0057] The measured values ​​of the subject displayed in the image can be the measured values ​​of at least one of bone mineral density, bone mass, and bone quality measured at the first time point. Furthermore, the estimated values ​​of the subject displayed in the image can be estimated values ​​derived from medical images of the subject's bones taken at a second time point, different from the first time point, using an estimation model 4221 or similar method provided in the display device 4. These estimated values ​​are the estimated values ​​of at least one of bone mineral density, bone mass, and bone quality. The display device 4 generates a display image that arranges the measured values ​​and estimated values ​​of the subject in a time-series order of the first and second time points.

[0058] According to the aforementioned structure, the information processing system 1 generates an image that arranges the measured bone-related information of the subject at a first time point and the estimated bone-related information of the subject at a second time point in a time-series order based on the first and second time points. That is, the information processing system 1 can generate an image that arranges the measured and estimated values ​​of at least one of the subject's bone mineral density, bone mass, and bone quality in a time-series order. Therefore, the information processing system 1 improves user convenience when estimating or inferring information about the subject's body.

[0059] (Information Processing System 1)

[0060] Next, the details of the information processing system 1 according to this embodiment will be described. As described above, the information processing system 1 includes a measuring device 2, a camera device 3, and a display device 4.

[0061] ≪Measuring Apparatus 2≫

[0062] The measuring device 2 is a device for measuring the bone mineral density of the subject. The measuring device 2 is not particularly limited to any device capable of measuring the bone mineral density of the subject; the following examples can be given.

[0063] For example, the measuring device 2 may be a device that uses DXA (Dual energy X-ray Absorptiometry), ultrasound, or MD (micro densitometry) to measure bone mineral density.

[0064] The measuring device 2 sends measurement information 431, representing the measured value of the subject's bone mineral density, to the display device 4. In other words, the measurement information 431 can be described as information measured using a given measuring device that performs the bone mineral density measurement. The measurement information 431 may include information indicating the date and time of the measurement. Communication between the measuring device 2 and the display device 4 can be based on wired communication or wireless communication. Furthermore, this communication can also be via the Internet.

[0065] ≪Camera Device 3≫

[0066] The imaging device 3 can be, for example, a device for capturing images to obtain information about the body of a subject. The imaging device 3 is a device for capturing medical images of the subject. The medical images captured by the imaging device 3 can be images containing information about the subject's bones; there are no particular limitations, and examples include: For instance, the medical images can be X-ray images, MRI images, CT images, PET images, or ultrasound images.

[0067] The camera device 3 transmits data representing the captured medical image, namely medical image data 432, to the display device 4. The medical image data 432 may include information indicating the date and time the medical image was captured. Communication between the camera device 3 and the display device 4 can be wired or wireless. Furthermore, this communication can be via the Internet. Alternatively, if the camera device 3 cannot communicate directly with the display device 4, a removable memory or similar device can be used to move the medical image data 432.

[0068] Display Device 4

[0069] The display device 4 generates and displays images showing the measured and estimated values ​​of the subject's bone mineral density. The display device 4 includes a communication unit 41, a control unit 42, a storage unit 43, an operation input unit 44, and a display unit 45.

[0070] <Communication Department 41>

[0071] The communication unit 41 communicates with the measuring device 2 and the imaging device 3. The communication unit 41 receives measuring information 431 from the measuring device 2. In addition, the communication unit 41 receives medical image data 432 from the imaging device 3.

[0072] <Control Department 42>

[0073] The control unit 42 comprehensively controls all parts of the display device 4. The control unit 42 includes an acquisition unit 421 (first acquisition unit), a first generation unit 422, an operation receiving unit 423, an image generation unit (generation unit) 424, an information acquisition unit (first acquisition unit, second acquisition unit, third acquisition unit) 425, a display control unit 426, and a second generation unit 427.

[0074] Acquisition Section 421

[0075] The acquisition unit 421 acquires measurement information 431 from the measurement device 2 via the communication unit 41. In addition, the acquisition unit 421 acquires medical image data 432 from the imaging device 3 via the communication unit 41.

[0076] The acquisition unit 421 stores the acquired measurement information 431 and medical image data 432 in the storage unit 43. In addition, the acquisition unit 421 sends the acquired medical image data 432 to the first generation unit 422.

[0077] Part 1, Chapter 422

[0078] The first generation unit 422 generates estimation information 433, which represents the estimated value of the bone density of the subject based on the medical image (first image) represented by the medical image data 432 of the subject received from the acquisition unit 421. The first generation unit 422 stores the generated estimation information 433 in the storage unit 43. The first generation unit 422 has an estimation model 4221.

[0079] <Presumed Model 4221>

[0080] The estimation model 4221 is a model that estimates the bone density of a subject based on medical images represented by medical image data 432. The estimation model 4221 can be set based on training data containing medical images (the second image) of the bones of the first person and teaching data containing at least one of the first person's bone density, bone mass, and bone quality. That is, the estimation model 4221 is a trained model.

[0081] For example, for each of the multiple learning image data contained in the learning data, a corresponding bone density of a person whose bone is captured in a simple X-ray image representing the learning image data can be established as teaching data. The bone density of the teaching data can also be obtained using DXA, ultrasound, or MD methods. Furthermore, the aforementioned "first person" does not necessarily equate to a human. For example, it could be an animal of the same species as the "object".

[0082] Operation Receiving Department 423

[0083] The operation receiving unit 423 accepts operation input from the user via the operation input unit 44. The operation receiving unit 423 accepts operation input instructing the display of a display image showing information related to the subject's bones. Upon receiving such operation input, the operation receiving unit 423 instructs the image generation unit 424 to generate an image.

[0084] The operation receiving unit 423 receives an operation input indicating a deduction related to the bones of the subject. Upon receiving this operation input, the operation receiving unit 423 instructs the second generation unit 427 to generate deduction information 434 representing the deduction related to the bones of the subject. Furthermore, the operation receiving unit 423 instructs the image generation unit 424 to generate a display image showing the deduction information 434.

[0085] The operation receiving unit 423 receives an operation input indicating the end of the display. If the operation receiving unit 423 receives such operation input, it instructs the display control unit 426 to end the display.

[0086] Image Generation Department 424

[0087] If the image generation unit 424 receives an instruction to generate a display image showing information related to the subject's bones, it acquires the measurement information 431 and estimation information 433 from the storage unit 43 via the information acquisition unit 425. The image generation unit 424 uses the measurement information 431 and estimation information 433 to generate a display image showing information related to the subject's bones.

[0088] If the image generation unit 424 receives an instruction to generate a display image displaying the prediction information 434, it acquires the measurement information 431, estimation information 433, and prediction information 434 from the storage unit 43 via the information acquisition unit 425. The image generation unit 424 uses the measurement information 431, estimation information 433, and prediction information 434 to generate a display image displaying information related to the subject's bones.

[0089] The image generation unit 424 sends the data of the generated display image to the display control unit 426. Details regarding examples of the display images generated by the image generation unit 424 will be described later.

[0090] Information Acquisition Department 425

[0091] The information acquisition unit 425 acquires the measurement information 431 and estimation information 433 stored in the storage unit 43 according to the instructions of the image generation unit 424, and sends them to the image generation unit 424.

[0092] The information acquisition unit 425 acquires the measurement information 431, estimation information 433 and inference information 434 stored in the storage unit 43 according to the instructions of the image generation unit 424, and sends them to the image generation unit 424.

[0093] Display Control Unit 426

[0094] The display control unit 426 displays the display image represented by the image data received from the image generation unit 424 on the display unit 45.

[0095] The display control unit 426 terminates the image display on the display unit 45 according to the display termination instruction received from the operation receiving unit 423.

[0096] Part 2, Chapter 427

[0097] The second generation unit 427 generates prediction information 434, which represents the predicted value of the bone density of the subject based on the medical image data 432 stored in the storage unit 43.

[0098] For example, the second generation unit 427 can generate prediction information 434 related to at least one of bone density, bone mass, and bone quality, which is inferred from a medical image of the subject's bone taken at a first time point based on at least one of bone density, bone mass, and bone quality. In this embodiment, an example is described where the prediction information 434 represents a predicted value of the subject's bone density, but the description of "prediction information 434" in this disclosure can also be appropriately interpreted as representing information about at least one of "bone density," "bone mass," and "bone quality."

[0099] Here, the measurement of bone density at the first time point and the acquisition of a medical image of the subject's bone at the first time point can be performed non-strictly simultaneously. For example, the measurement and acquisition can be performed within a given period. The given period is not particularly limited as long as it is a period in which it is difficult to produce a significant change in the subject's bone density; for example, half a day, one day, one week, one month, three months, and six months can be given. Furthermore, if a medical image of the subject's bone is not acquired at the first time point, the display device 4 may not receive the medical image data 432 representing the medical image of the subject's bone at the first time point. That is, the display device 4 can receive the medical image data 432 representing the medical image of the subject's bone at the first time point even if a medical image of the subject's bone is acquired at the first time point. In this specification, the time point at which the medical image is acquired in the generation of the inference information 434 is referred to as the inference base point. The inferred information 434 could also be information representing the inferred value of the subject's bone density at time point 3. Time point 3 could be a different time point than time point 1.

[0100] The second generation unit 427 can generate inference information 434 related to the subject's bone density, which is inferred by the first generation unit 422 based on medical images of the subject's bones taken at a second time point used in the estimation of the subject's estimated bone density. This inference information 434 may represent the estimated value of the subject's bone density at a third time point. The third time point may be a time point different from the second time point.

[0101] When the image generation unit 424 generates an image using the speculative information 434, the image generation unit 424 can generate a display image that shows the speculative information 434 corresponding to the establishment of the third time point.

[0102] The second generation unit 427 generates prediction information 434 based on an instruction to generate prediction information 434 representing a prediction related to the bones of the subject received from the operation receiving unit 423. The second generation unit 427 stores the generated prediction information 434 in the storage unit 43. Furthermore, the second generation unit 427 includes a prediction model 4271.

[0103] <Speculative Model 4271>

[0104] The inference model 4271 is a model that infers the bone density of a subject at a time point different from the time the medical image was taken, based on the medical image data 432 representing the subject's medical image. The inference model 4271 can be set based on learning data containing a medical image (the third image) showing the bone of a second person, and teaching data containing the bone density of the second person. The "second person" mentioned above is not specifically limited to humans. For example, it could be an animal of the same species as the "subject".

[0105] For example, for multiple learning image data contained in the learning data, a correspondence can be established using the bone density of a person whose bone is captured in a simple X-ray image as represented by the learning image data, and this correspondence can be used as teaching data. The teaching data can be corresponded to each of the multiple learning image data separately, or multiple sets can be prepared that correspond multiple learning image data to one teaching data. The teaching data can also be bone density obtained using DXA, ultrasound, or MD methods. Alternatively, the teaching data can also use values ​​estimated or inferred from other learned models.

[0106] The teaching data can be data measured at any time. The teaching data can be measured concurrently with the period of the image data used for learning, or it can be measured on a different time axis. For example, in the measurement of the teaching data, a given baseline can be set for the period of the image data used for learning. As a given baseline, for example, it can be within the past 3 months, 6 months, or 1 year, relative to the period of the image data used for learning. Furthermore, as a given baseline for the teaching data, it can also be at least one of the following: presence or absence of treatment, age, amenorrhea, and gender of the subject in the teaching data. Treatment can also be based on at least one of medication, nutritional guidance (including common use of supplements), and exercise guidance.

[0107] The learning image data can also be a series of images of the same person taken at different timelines. That is, the learning image data can include the following: first learning data and second learning data. The first learning data can be learning data containing an X-ray image of a person's bones. The second learning data can be an image of the same person, which is learning data included in X-ray images taken after the first learning data.

[0108] Image data used for learning can also be different datasets of images of the same body parts, ages, etc., taken from different people. Furthermore, image data used for learning can also be a series of images of the same person taken at different timelines. Image data used for learning can also be a series of images of the same body parts taken at different timelines.

[0109] The inference information 434 generated by the second generation unit 427 may be information related to the bone density of a subject who has undergone bone density treatment. In this case, the second person is someone who has received bone density treatment, and the inference model 4271 may be set based on learning data including medical images of the bones of the treated second person and teaching data including the bone density values ​​of the second person. For example, the learning data may be learning data including medical images before and after treatment. Furthermore, the teaching data may be teaching data including measured bone density values ​​before and after treatment.

[0110] The second generation unit 427 may have the following two models as prediction models 4271: (1) A model based on learning data including medical images of the bones of a second person who has received treatment, and teaching data including bone mineral density measurements of the second person who has received treatment. (2) A model based on learning data including medical images of the bones of a second person who has not received treatment, and teaching data including bone mineral density measurements of the second person who has not received treatment. The treatment includes drug-based treatment (e.g., also called pharmaceutical therapy), nutritional guidance, exercise guidance, etc.

[0111] The second generation unit 427 can generate speculative information 434 corresponding to the timing of the start of treatment for the subject.

[0112] Furthermore, if the medical image data 432 used in estimating the subject's bone density represents an image at time point 1, the third time point can be a time point before or after time point 1. Similarly, if the medical image data 432 used in estimating the subject's bone density represents an image at time point 2, the third time point can be a time point before or after time point 2.

[0113] The learning data of the presupposition model 4221, namely the medical image of the first person's bone (image 2), and the learning data of the inference model 4271, namely the medical image of the second person's bone (image 3), can also be the same image.

[0114] The second and third images can also be medical images taken of different body parts. The person photographed in the second image and the person photographed in the third image can also be the same person. Alternatively, the person photographed in the second image and the person photographed in the third image can also be different people.

[0115] The teaching data, which includes the bone mineral density measurements of the second person, can also be data measured at a time point different from the time point when the third image was taken. For example, the teaching data can be set as at least one of the following: 6 months later, 1 year later, several years later, and several years ago when the third image was taken. Alternatively, the learning data can be set as bone mineral density measurements measured within a given period that includes the time point when the third image was taken.

[0116] <Storage Department 43>

[0117] Storage unit 43 stores the aforementioned measurement information 431, medical image data 432, estimation information 433, and inference information 434. Furthermore, storage unit 43 may also store subject data representing information about the subject. This subject data may include data indicating whether treatment was received on the subject, the duration of treatment received, the content of treatment received, and the date and time of the subject's injury.

[0118] <Operation Input Section 44>

[0119] The operation input unit 44 accepts user operation input. For example, the operation input unit 44 may be integrated with the display unit 45. The display unit 45 may also display instructions for the processing performed by the control unit 42, such as bone density estimation processing, image generation processing, and image display processing. It may also be configured such that the user selects these display areas to perform the processing corresponding to the display areas.

[0120] <Display Unit 45>

[0121] Display unit 45 displays the image generated by image generation unit 424 to the user. Examples of display unit 45 include monitors and displays.

[0122] (Displays the processing flow)

[0123] Next, refer to Figure 2 This will explain the display processing flow in display device 4. Figure 2 This is a flowchart illustrating an example of the display processing flow in display device 4. For example... Figure 2 As shown, when the operation receiving unit 423 receives an operation input instructing the display of a display image showing information related to the bone of the subject ("Yes" in S1), the information acquisition unit 425 performs the following processing.

[0124] The information acquisition unit 425 acquires measurement information 431 (S2: first acquisition step). Furthermore, the information acquisition unit 425 acquires estimated information 433 (S3: second acquisition step). For example, the first information is information representing the bone density of the subject measured at a first time point, and the second information is information representing bone density estimated by an estimation model based on a first image of the subject's bone taken at a second time point different from the first time point. Next, the image generation unit 424 generates a display image showing the measurement value represented by the measurement information 431 and the estimated value represented by the estimated information 433 (S4: generation step). Then, the display control unit 426 displays the display image generated by the image generation unit 424 on the display unit 45 (S5).

[0125] When the operation receiving unit 423 receives an operation input indicating a bone-related prediction for the subject ("Yes" in S6), the information acquisition unit 425 acquires the prediction information 434 (S7). This operation input may include input indicating the time point from which the prediction becomes a base point, and input indicating when the prediction began. For example, without particular limitation, examples include predictions six months after the base point, predictions one year after the base point, predictions six months before the base point, and predictions one year before the base point. Furthermore, this operation input may also include input related to whether or not the subject has received treatment. For example, the prediction may include predictions of whether the subject has received treatment, predictions of whether or not treatment has been received, and predictions of both treatment and no treatment. Additionally, this operation input may include input related to the timing of the start of the subject's treatment. For example, the prediction may correspond to the timing of the start of the subject's treatment. Alternatively, if treatment has been received, multiple predictions may be made based on the content of the treatment. For example, this hypothesis can also be at least one of pharmacological treatment, nutritional guidance treatment, or exercise guidance treatment.

[0126] Next, the image generation unit 424 generates a display image showing the predicted value represented by the predicted information 434 (S8). Then, the processing returns to S5.

[0127] If the operation receiving unit 423 does not receive an operation input indicating a deduction related to the subject's bones ("No" in S6), but receives an operation input indicating the end of the display ("Yes" in S9), the process ends.

[0128] If the operation receiving unit 423 does not receive an operation input instructing the display image to display information related to the subject's bones ("No" in S1), the process of S1 is repeated.

[0129] If the operation receiving unit 423 does not receive an operation input indicating a deduction related to the subject's bones ("No" in S6) or does not receive an operation input indicating the end of the display ("No" in S9), the process returns to S6.

[0130] (Example of the display image generated by the image generation unit 424)

[0131] Next, refer to Figures 3 to 9 An example illustrating the display image generated by the image generation unit 424 will be provided. Furthermore, Figures 3 to 9 The description of “bone density” can be appropriately interpreted as at least one of “bone density”, “bone mass”, and “bone quality”.

[0132] Example of a display image showing the predicted values ​​for time points after the predicted base point.

[0133] First, an example of a display image showing the predicted values ​​for time points after the time point that serves as the base point for the prediction. Figures 3 to 5 This is an example of a display image showing the estimated value of bone density of an object at a time point after the time point that serves as the base point for estimation.

[0134] exist Figure 3 Image D1 shows the measured bone mineral density values ​​m1-m3 of the subjects at the time points when the measurements were taken, the estimated bone mineral density values ​​n1-n2 of the subjects at the time points when the medical images were taken, and the shift T1 of bone mineral density based on the measured values ​​m1-m3 and the estimated values ​​n1-n2. Furthermore, the name of the facility that took the medical images is shown in image D1.

[0135] The measured bone mineral density (BMD) value of a subject can also be displayed in a different way than the estimated BMD value. For example, the measured value can be shown as a hollow circle, while the estimated value can be shown as a black circle.

[0136] The predicted values ​​o1 and o2 are displayed in image D1. Measured values, estimated values, and predicted values ​​can be displayed in different ways. Figure 3 In the example shown, the predicted values ​​o1 and o2 are represented by hollow triangles. The predicted values ​​are connected by a dotted line, with the predicted value indicated at the end of the dotted line. The starting point of the dotted line indicates the time point that becomes the base point for the prediction.

[0137] In image D1, the estimated value o1 is an estimated value with December 21, 2021 as the reference point. The estimated value o1 is derived from medical images taken on December 21, 2021. Furthermore, the estimated value o1 is the estimated bone mineral density of the subject at a time point two months later, on February 21, 2022.

[0138] The estimated value o2 is an estimated value based on the current point in time, derived from the currently captured medical images. Furthermore, the estimated value o2 is an estimated value for the subject's bone density six months from now.

[0139] At a certain point in time, if the bone density of the subject is measured, the measured value at that point in image D1 becomes the measured value represented by the corresponding measurement information 431 obtained through the measurement. On the other hand, if the same point in time as that point in time is set as the base point for estimation, the estimated value based on that point in time displayed in image D1 becomes the estimated value estimated based on the medical image taken at that point in time.

[0140] The measured value, estimated value, and predicted value are displayed. Specifically, the labels "Measured Value," "Estimated Value," and "Predicted Value" can be displayed for each measured value, estimated value, and predicted value. These labels can also be displayed in different ways. By using different display methods, users can easily identify values ​​based on different methods. Different display methods can be represented, for example, by differences in text size, font, and color. Furthermore, the "Measured Value," "Estimated Value," and "Predicted Value" can be displayed using at least icons, abbreviations, and symbols. Additionally, information related to the method used to obtain the displayed value can be annotated in image D1. This method can include, for example, "measured," "estimated," and "predicted." That is, the image generation unit 424 can generate a display image that includes displays representing the first and second methods. This annotation method improves user convenience. In addition, the YAM (Young Adult Mean) ratio, which corresponds to bone mineral density, is displayed in image D1. In image D1, the YAM ratio represents the diagnostic benchmark value, with areas of YAM ratio between 100% and 80% marked as "normal," areas of YAM ratio between 80% and 70% marked as "osteoporosis," and areas of YAM ratio below 70% marked as "osteoporosis."

[0141] Image D1 shows the bone mineral density per unit area (g / cm³). 2 Bone salt density per unit volume (g / cm³)3 The image D1 can be composed of at least one of the following: AGE, T-score, and Z-score. For example, AGE is displayed as a value compared to the average bone mineral density of the same age or epoch. Alternatively, an image D1 can be used to determine the category corresponding to a given baseline from these values. Categories can include "normal," "osteoporosis," and "osteoporosis," etc., corresponding to these values. For example, when the T-score is displayed in image D1, a T-score of -1.0 or higher can be displayed as normal bone mineral density, a T-score of -1.0 to -2.5 as low bone mineral density or osteoporosis, and a T-score below -2.5 as osteoporosis. Furthermore, when the Z-score is displayed in image D1, a Z-score below -2.0 can be displayed as below age-appropriate, and a Z-score above -2.0 can be displayed as above age-appropriate.

[0142] The second generation unit 427 can predict the period at which the subject's bone mineral density reaches "normal," "low bone mineral density or osteopenia," or "osteoporosis" based on the YAM ratio, T score, or Z score. That is, the second generation unit 427 can also generate prediction information 434 indicating this period. For example, prediction information 434 might indicate that the subject's bone mineral density reaches a YAM ratio below 70% (osteoporosis) one year later.

[0143] That is, the image generation unit 424 generates a display image that shows information representing the criteria for determining bone density. Furthermore, the information representing the criteria may be information representing criteria corresponding to the subject's age. Alternatively, the information representing the criteria may be information representing criteria such as guide lines, which are already known based on individual criteria.

[0144] <Example 1: Displaying a presumed value for the subject's past treatment history>

[0145] Next, refer to Figure 4 This is an example of a display image showing the estimated value of a subject's bone density, indicating whether the subject has received treatment. Figure 4 This is an example of a display image showing the estimated value of bone mineral density in a subject who has received treatment. Figure 4 Image D2 shows the measured bone mineral density values ​​m1-m3 at the time points when the measurements were taken, the estimated bone mineral density values ​​n1-n2 of the subject at the time points when the medical images were taken, and the shift T1 of bone mineral density based on these measured and estimated values. Regarding image D2, the same reference numerals are used for displays identical to those described in image D1 above, and their description will not be repeated here.

[0146] Image D2 displays the speculative values ​​o1, o2, and o3. The speculative values ​​o1 and o2 displayed in image D2 are speculative values ​​for the subject's condition of not receiving treatment.

[0147] The estimated value o3 is a presumed value for the treatment received by the subject. In image D2, the "pharmaceutical therapy" is displayed at the current point in time. This display indicates that the "pharmaceutical therapy" began at the present. That is, the estimated value o3 is a presumed value six months after the "pharmaceutical therapy" treatment began on the subject from the present.

[0148] exist Figure 4 In the example shown, hollow triangles represent the predicted values ​​o1 and o2 for the subject not receiving treatment. Conversely, black triangles represent the predicted value o3 for the subject receiving treatment.

[0149] Based on the described structure, users can visually identify the effectiveness of the treatment.

[0150] This example illustrates a display image showing the estimated values ​​six months after the start of "pharmaceutical therapy" for the current patient. Alternatively, the image could also display estimated values ​​of bone density for patients who received treatment at previous points in the past. Furthermore, in pharmaceutical therapy, estimated values ​​for multiple medications can be displayed on the image. More specifically, for example, the estimated values ​​for each of the multiple medications (e.g., three types) used in pharmaceutical therapy can be displayed on the image. In this case, the estimated values ​​for the most effective medications (e.g., three types) from a pool of many medications (e.g., ten types) can also be displayed on the image.

[0151] Example 2: Displaying inferred values ​​of treatment received by the subject: Displaying the treatment period.

[0152] Next, refer to Figure 5 Other examples of display images illustrating the estimated value of a subject's bone density, showing the subject's treatment history. Figure 5 This is another example of a display image showing an estimated value of bone mineral density for a subject who has received treatment. Figure 5 Image D3 shows the measured bone mineral density values ​​m1, m11-m12 at the time points when the measurements were taken, the estimated bone mineral density values ​​n1-n2 of the subject at the time points when the medical images were taken, and the shift in bone mineral density T2 based on these measured and estimated values. Regarding image D3, the same reference numerals are used for displays identical to those described in images D1-D2 above, and their description will not be repeated here.

[0153] Image D3 displays a display p1 showing the period of treatment actually performed on the subject. Display p1 shows the treatment period from February 21, 2022 to the present. That is, the image generation unit 424 can generate a display image that displays information about the treatment period for the subject regarding bone density. In addition, information about the content of the treatment can also be displayed in image D3. Furthermore, image D3 can also display a display p2 showing the date and time of the subject's injury. The image generation unit 424 can also use the subject data stored in the storage unit 43 to generate display images such as display p1 and display p2, which display information related to the subject. Regarding the display p1 showing the treatment period and the display p2 showing the date and time of the subject's injury, the display device 4 can also accept an operation input to select whether the display is needed or not.

[0154] Image D3 displays the predicted values ​​o11, o12, and o13. Predicted value o11 is a prediction of the subject's treatment history, using the start date of treatment, February 21, 2021, as the baseline. Furthermore, predicted value o11 is the predicted value for May 13, 2021, a date following this baseline.

[0155] The projected value o12 is a projected value six months from now, based on the current situation, assuming no further treatment will be given to the subject.

[0156] The projected value o13 is a projected value six months from now, based on the current situation, regarding the continuation of treatment for the subject.

[0157] Example of a display image showing the predicted values ​​for time points prior to the predicted base point.

[0158] Next, we will explain an example of a display image showing the predicted values ​​for time points prior to the time point that becomes the base point of the prediction. Figure 6 as well as Figure 7 This is an example of a display image showing the estimated value of bone density of an object at a time point prior to the time point that serves as the base point for estimation.

[0159] <Example of a display image showing a current, past estimated bone density value with the estimated base point set to the current value>

[0160] exist Figure 6Image D4 shows the measured bone mineral density (BMD) value m1 of the subject at the time point of measurement, the estimated BMD values ​​n1, n2, and n21 of the subject at the time point of medical imaging, and the shift in BMD T3 based on these measured and estimated values. Furthermore, image D4 displays the predicted values ​​o21, o22, and o23. Regarding... Figure 6 The same reference numerals are used for the same display as those used in the above-described images D1-D3, and their description will not be repeated here.

[0161] The predicted value o21 is the predicted value for May 13, 2022, a time point after February 21, 2022, which is the base point of the prediction.

[0162] The estimated value o22 is an estimated value six months after the current point in time, with the current point as the base point for estimation.

[0163] The predicted value o23 is the predicted value for May 13, 2021, a point in time preceding the current point in time, with the current point as the base point for prediction. In other words, the predicted value o23 is the predicted value for a point in time past the current point in time, which serves as the base point for prediction.

[0164] <Example of a display image showing the estimated past bone density value of a base point that is set to a past time point>

[0165] exist Figure 7 Image D5 shows the measured bone mineral density (BMD) values ​​m1 and m31 of the subject at the time points when the measurements were taken, the estimated BMD value n31 of the subject at the time point when the medical images were taken, and the shift in BMD T4 based on these measured and estimated values. Furthermore, image D5 shows the predicted values ​​o31, o32, and o33. Regarding... Figure 7 The same reference numerals are used for the same displays as those described in the aforementioned images D1-D4, and their descriptions will not be repeated here.

[0166] The predicted value o31 is a predicted value for December 21, 2021, a point in time preceding February 21, 2022, which is the base point of the prediction. In other words, the predicted value o31 is a predicted value for a point in time further past than the base point of the prediction.

[0167] The estimated value o32 is the estimated value for May 13, 2022, a time point prior to the current time, with the current time as the base point of estimation.

[0168] The estimated value o33 is an estimated value six months from now, based on the current time.

[0169] according to Figure 6 as well as Figure 7 The example shown can fill in the bone density values ​​of subjects at past time points where medical images were not taken or bone density measurements were not performed. Furthermore, in medical institutions, it is possible to infer the bone density values ​​of subjects at past time points during their initial medical visit.

[0170] Example of a display image showing the shift in bone density of a reference subject different from the subject.

[0171] Next, an example of a display image showing information related to the bone density of the subject and information on the shift in bone density compared to a reference representing a different subject will be described. Figure 8 This is an example of a display image showing information related to the bone density of the subject and information showing the shift in bone density of the reference subject.

[0172] exist Figure 8 Image D6 shows the measured bone mineral density (BMD) values ​​m41-m43 of the subjects at the time points when the measurements were taken, the estimated BMD values ​​n41-n42 of the subjects at the time points when the medical images were taken, and the shift T5 of the subjects' BMD based on these measured and estimated values. Furthermore, image D6 shows the inferred value o41. The inferred value o41 is the inferred BMD value of the subjects at a time point six months from the current inferred point. Regarding... Figure 8 Image D6 shown uses the same reference numerals as those used in images D1-D5 described above, and their description will not be repeated here.

[0173] Image D6 displays p3, which shows the shift in bone density of a reference person different from the subject. That is, image generation unit 424 generates image D6, which displays information indicating the shift in bone density of a reference person different from the subject. For example, reference data showing the shift in bone density of a reference person can be stored in storage unit 43. Image generation unit 424 can use the reference data stored in storage unit 43 to generate a display image that shows p3, which represents information indicating the shift in bone density of the reference person.

[0174] For example, the reference person is someone with the same attributes as the subject. Here, "same attributes" can refer to at least one of the following: age, weight, sex, disease, blood relation, type of osteoporosis, values ​​based on blood markers, and assessments based on the Fracture Risk Assessment Tool (FRAX). The reference person's shift in bone mineral density serves as a comparison point for the subject's shift in bone mineral density. For example, the reference person can be someone who has received bone mineral density-related treatment or someone who has not.

[0175] Example of a display image showing predicted values ​​based on changes in the timing of the start of treatment for the subject.

[0176] Next, an example of a display image showing the predicted values ​​corresponding to changes in the timing of the start of treatment for the subject will be explained.

[0177] Figure 9 This is an example of a display image that transitions based on a change in the timing of the start of treatment for a patient. In this example, it illustrates the transition of the image when the timing of the start of treatment for a patient changes from six months in advance to the current time, based on user input. Figure 9 As shown, if this change is made, the displayed image transitions from image D7 to image D8. Furthermore, the timing for the start of treatment can be set to any point in time via user input.

[0178] Image D7 is an example of an image showing the estimated value of bone mineral density (BMD) of a subject who began treatment six months later. Image D7 shows the measured BMD values ​​m51-m52 at the time the measurements were taken, the estimated BMD values ​​n51-n52 at the time the medical images were taken, and the shift in BMD T6 based on these measured and estimated values. Furthermore, image D7 shows the estimated values ​​o51, o52, and o53. Regarding image D7, the same reference numerals are used for displays identical to those described in images D1-D6 above, and their description will not be repeated here.

[0179] The estimated value o51 is the estimated bone density of the subject six months from now, based on the current time.

[0180] The estimated value o52 is the estimated bone mineral density of the subject one year later, with the current point as the baseline, based on the condition that treatment begins six months from now. For estimated values ​​o52, o53 (described later), o62 (image D8), and o64 (image D8), the estimated values ​​are shown at the endpoints of the dotted lines connecting them to the estimated bone mineral density of the subject six months from now. The starting point of the dotted lines connecting these estimated values ​​does not indicate the time point that becomes the baseline for the estimation.

[0181] The estimated value o53 is the estimated bone density of the subject one year later, based on the current state, before treatment has begun.

[0182] Image D8 is an example of a display image showing the estimated values ​​of bone mineral density of a subject at the current stage of treatment. Regarding image D8, the same reference numerals are used for displays identical to those described in images D1-D7 above, and their description will not be repeated here. Estimated values ​​o51, o52, o63, and o64 are displayed in image D8.

[0183] The estimated value o61 is the estimated bone density of the subject six months later, based on the current state, before treatment has begun.

[0184] The estimated value o62 is the estimated bone mineral density of the subject one year later, with the current state as the estimation base, assuming no treatment has been started.

[0185] The estimated value o63 is the estimated bone density of the subject six months after the current start of treatment, using the current situation as the baseline.

[0186] The estimated value o54 is the estimated bone density of the subject one year after the current start of treatment, using the current state as the estimation point.

[0187] This example illustrates the transition of the displayed image from image D7 to image D8 based on a change in the timing of the treatment start, but images D7 and D8 can also be displayed simultaneously.

[0188] Individual estimated values ​​corresponding to the timing of treatment initiation at different points can also be displayed in a single image, such as a single graph.

[0189] (Other examples)

[0190] Next, we will describe other examples of images D1-D8 mentioned above.

[0191] (Examples of bone names captured in medical images used in the estimation of the estimated values)

[0192] Images D1-D8 may also display the names of bones captured in medical images used to estimate each estimate. That is, the image generation unit 424 can generate a display image showing information indicating the names of bones captured in medical images used to estimate estimate information 433. Furthermore, the first generation unit 422 can generate estimate information 433 based on images of bones from multiple locations captured in medical images. The image generation unit 424 can generate a display image showing information indicating the names of multiple bones captured in medical images used to estimate estimate information 433. The names of bones may include, for example, at least one of the following: skull, clavicle, sternum, scapula, ribs, humeral bones, radius, ulna, hand bones, spine, sacrum, coccyx, femur, patella, tibia, fibula, and foot bones.

[0193] In images D1-D8, the names of the bones captured in the medical images used to infer each predicted value can be represented by the following formula. That is, the image generation unit 424 can generate a display image showing information indicating the names of the bones captured in the medical images used to infer the predicted information 434. Furthermore, the second generation unit 427 can generate the predicted information 434 based on images of bones from multiple locations captured in the medical images. The image generation unit 424 can generate a display image showing information indicating the names of multiple bones captured in the medical images used to infer the predicted information 434.

[0194] (Examples of medical images used in showing the estimation of the estimated value and the inference of the predicted value)

[0195] Images D1-D8 can display the estimation of each presumed value and the medical images used in the estimation of each presumed value. These medical images can be, for example, images obtained by taking pictures of the head, neck, chest, lumbar region (lumbar vertebrae: L1-L4, etc.), hip joint, knee joint, foot joint, foot, toes, shoulder joint, elbow joint, hand joint, hand, fingers, or jaw joint. The types of sites from which medical images are taken are not limited to these. Furthermore, medical images can be frontal images of the entire object area taken from a single X-ray exposure, or side images of the object area taken from the side. The X-ray images can be images of at least one of the cortical bone and the cavernous bone.

[0196] (Examples showing the estimation or inference of bone density from multiple sites)

[0197] When estimating or inferring bone density from multiple sites captured in medical images, images D1-D8 can display individual graphs showing the shift in bone density at each site. For example, the images of each site used in estimating and inferring each value can be overlaid with the corresponding graphs. Multiple sites can be defined as regions for each vertebra. Multiple sites can be defined as regions encompassing multiple vertebrae. Multiple sites can be defined as regions of different areas. Furthermore, multiple sites can be defined as regions dividing a specific portion of a site. Multiple sites can be arbitrarily defined based on the subject's attributes. For example, in cases where the subject has a history of fractures, a region with a narrower area compared to subjects without a fracture history can be defined.

[0198] (Example of displaying the judgment criteria)

[0199] In a display image showing information indicating criteria for normality, osteopenia, and osteoporosis, the criteria may be overlaid on the chart. These criteria may correspond to the subject's age at the time of measurement, the subject's age at an estimated time, or the subject's age at a presumed time. The presumed time, for example, could be the third time point mentioned above.

[0200] (Examples showing the error range or probability of the predicted value)

[0201] The second generation unit 427 can generate prediction information 434 containing information about the error range of the predicted values. The image generation unit 424 can generate a display image that shows information about the error range of each predicted value.

[0202] In detail, the second generation unit 427 can generate prediction information 434 containing prediction probability about the predicted value, or information representing the error range about the predicted value. The prediction probability can be, for example, a prediction probability. At least one of this prediction probability and the error range can be displayed in the images D1-D8 described above.

[0203] (Example showing the error range of the predicted value)

[0204] Here is an example of a display image showing the error range of the predicted value. Figure 10 This is an example of a display image showing the error range of the estimated value of the bone density of the displayed subject. Figure 10Image D9 shows the measured bone mineral density values ​​m71-m72 of the subjects at the time points when the measurements were taken, the estimated bone mineral density values ​​n71-n72 of the subjects at the time points when the medical images were taken, and the shift T7 of the subjects' bone mineral density based on these measured and estimated values. Furthermore, the predicted values ​​o71 and o72 are also shown in image D9.

[0205] Predicted value o71 is the estimated bone density of the subject within a given period, taking the current point as the base point and including a time point six months from the base point. Predicted value o72 is the estimated bone density of the subject within a given period, taking the current point as the base point and including a time point one year from the base point. That is, predicted values ​​o71 and o72 can be displayed as estimated bone density values ​​of the subject within a given period, taking the current point as the base point and including a time point after a given period has elapsed from the base point.

[0206] exist Figure 10 In the example shown, the predicted values ​​o71 and o72 are represented by circles containing the predicted values ​​and their error ranges. Furthermore, the error ranges of predicted values ​​o71 and o72 can be displayed using error bars or box plots. Additionally, the upper limits of the error ranges of predicted values ​​o71 and o72 can be displayed by connecting them with dotted lines, etc. Furthermore, the lower limits of the error ranges of predicted values ​​o71 and o72 can be displayed by connecting them with dotted lines, etc.

[0207] (Example of a range of presumed values)

[0208] The first generation unit 422 can generate estimation information 433 containing information representing the magnitude of each estimated value. The image generation unit 424 can generate a display image showing the magnitude of each estimated value. As a specific example of the magnitude of the estimated value, an estimated bone mineral density range of 0.79-0.81 (median value 0.80) g / cm³ can be given. 2 As an example, etc.

[0209] (Example showing the predicted probability of the predicted value)

[0210] Next, we will illustrate an example of a display image showing the probability of a predicted value. Figure 11 This is an example of a display image showing the predicted probability of a person's bone density. Figure 11 Image D10 shows the measured bone mineral density values ​​m81-m83 of the subjects at the time points when the measurements were taken, the estimated bone mineral density values ​​n81-n82 of the subjects at the time points when the medical images were taken, and the shift T8 of the subjects' bone mineral density based on these measured and estimated values. Furthermore, the inferred value o82 is also shown in image D10.

[0211] The predicted value o82 is a predicted value of the bone density of the subject at a time point six months from the current prediction point. Furthermore, in image D10, above the predicted value o82, the display p4 indicating the predicted probability of the predicted value o82 is displayed as "69%". That is, the second generation unit 427 can generate prediction information 434 representing the prediction probability associated with the predicted value. Furthermore, the image generation unit 424 can generate a display image showing the prediction probability for each predicted value.

[0212] (Example of measurement results showing the measured values)

[0213] Images D1-D10 can further display medical images taken during the measurement of each value. These medical images can be, for example, images of the head, neck, chest, lumbar region (lumbar vertebrae: L1-L4, etc.), hip joint, knee joint, foot joint, foot, toes, shoulder joint, elbow joint, hand joint, hand, fingers, or jaw joint. The types of sites from which medical images are taken are not limited to these. Furthermore, the medical images can be frontal images of the subject area taken from the front, or side images of the subject area taken from the side. The X-ray images can be images of at least one of the cortical bone and the cavernous bone. Additionally, images D1-D10 also display the measurement results corresponding to the measured values ​​being displayed. For example, the displayed measurement results can be a display representing the measurement results obtained by the DXA method.

[0214] (Other examples of medical images used in showing the estimation of the estimated value and the inference of the predicted value)

[0215] In images D1-D10, if the medical images used in the estimation of each presumption and the prediction of each conjecture are further displayed, the medical images can be displayed so that the area used in the estimation can be known.

[0216] For example, the first generation unit 422 can generate a heatmap for the region used in the estimation of the estimated value. In this case, for example, the outer edge of the heatmap can represent the segmented region used in the estimation. The heatmap can represent the magnitude of bone density with any color concentration. For example, the first generation unit 422 can generate a heatmap representing the degree of interest. In addition, a heatmap representing the numerical value of bone density can also be generated. Furthermore, the first generation unit 422 can generate a heatmap representing the probability of fracture. The display image can display an image overlaid with the heatmap on a medical image. The image used in the heatmap can be a still image or a dynamic image. By representing it with a dynamic image, for example by making various heatmaps fade in sequence, the relationship between the heatmaps can be easily identified visually. In addition, if the analysis result is a heatmap that also includes bone density outside the segmented region, a portion of its segmented region can be surrounded by a frame.

[0217] (Methods for displaying medical images and measurement results)

[0218] In images D1-D10, when further displaying the medical images used in the estimation of each estimated value and the prediction of each predicted value, if the mouse cursor or the like overlaps with the display area of ​​the estimated value or the display area of ​​the predicted value, a medical image corresponding to that estimated value or predicted value can be displayed. Here, the corresponding medical image can be the medical image used in the estimation of the estimated value or the prediction of the predicted value. For example, this medical image can be displayed so that the area used in the estimation of the estimated value or the prediction of the predicted value in the medical image can be known. Furthermore, if the mouse cursor or the like overlaps with the display area of ​​the measured value, the measurement result corresponding to that measured value can be displayed. For example, the displayed measurement result can be a display representing the measurement result measured by the DXA method. The display of these medical images or the display of measurement results can be displayed in a dialog box from the display area of ​​the measured value or the display area of ​​the predicted value that overlaps with the mouse cursor or the like. Furthermore, when a click operation is performed on the display area of ​​the estimated value, the predicted value, or the measured value, a display image containing the above-mentioned medical image or measurement result can be displayed.

[0219] This describes an example of a medical image display that shows the corresponding estimated and measured values. Figure 12 This is an example of a diagram showing the medical images used in the estimation of the estimated value and the display images of the measurement results corresponding to the measured value. (Regarding...) Figure 12 Regarding the above Figure 3 The same display notes and reference numerals are shown in the illustrations, and their descriptions will not be repeated here. Figure 12Image D11 shows a graph (P6) illustrating the shift in bone mineral density (T1), a medical image (P7) corresponding to the estimated value (n2), and a measurement result (P8) corresponding to the measured value (m3). Figure 12 As shown, in image D11, if the mouse cursor P5a overlaps with the display area of ​​the estimated value n2, the medical image corresponding to the estimated value n2, i.e., display P7, will be displayed. For example, if the operation receiving unit 423 receives an operation to overlap the display area of ​​the estimated value n2 with the mouse cursor P5a, the image generation unit 424 generates a display image including the display of P7, and the display control unit 426 displays this display image on the display unit 45. Furthermore, as... Figure 12 As shown, if the mouse cursor P5b is overlapped with the display area of ​​the measured value m3, the measurement result P8 corresponding to the measured value m3 will be displayed.

[0220] Figure 13 This is a diagram showing other examples of the display of a medical image corresponding to the estimated value n2. In this example, if the mouse cursor P5a is... Figure 12 When the display areas of the estimated value n2 overlap, the medical image corresponding to the estimated value n2 is displayed, i.e., P7a. The frame line of the area used to represent the estimated value n2 in the medical image is displayed in P7a, i.e., P710.

[0221] (Example showing the shift in bone density in multiple areas)

[0222] The display image shown on the display unit 45 can show the shifts in bone density at various sites captured in a medical image. An example of a display image showing the shifts in bone density at various sites will be described. Figure 14 This is an example of a display image showing the shift in bone density at multiple sites. (Regarding...) Figure 14 Regarding the above Figure 3 The same display notes and reference numerals are shown in the illustrations, and their descriptions will not be repeated here. Figure 14 Image D12 shows a graph (P11) illustrating the shift in bone density (T1) and a medical image (P9). Within the medical image (P9), outlines indicating the areas photographed at different points in the lumbar spine (L1-L4) are shown (P91 to P94). For example, as... Figure 14As shown, if the mouse cursor P10 is located within the display area of ​​lumbar vertebra L1 (P91), an image representing the shift in bone mineral density within lumbar vertebra L1 can be displayed. Furthermore, images simultaneously displaying the shift in bone mineral density of each of the lumbar vertebrae L1-L4 can also be displayed. In such an image, for example, if the mouse cursor P10 is located within the display area of ​​lumbar vertebra L1 (P91), the shift in bone mineral density within lumbar vertebra L1 can be emphasized.

[0223] (The suspected location of the fracture is shown as an example of the suspected result)

[0224] The operation receiving unit 423 can accept operation input indicating a prediction related to the fracture of the subject. If the operation receiving unit 423 receives such operation input, it instructs the second generation unit 427 to generate prediction information 434 representing the prediction related to the fracture of the subject. Furthermore, the operation receiving unit 423 instructs the image generation unit 424 to generate a display image representing the prediction related to the fracture of the subject represented by the prediction information 434. The operation receiving unit 423 accepts operation input indicating the end of display. If the operation receiving unit 423 receives such operation input, it instructs the display control unit 426 to end the display.

[0225] Other examples of Generation Section 2, 427

[0226] The second generation unit 427 of the control unit 42 can perform the following processing. The second generation unit 427 generates prediction information 434, which includes information about the likelihood of a future fracture in a subject, based on medical images represented by medical image data 432 stored in the storage unit 43. For example, the second generation unit 427 can generate prediction information 434, which includes information about the likelihood of a future fracture, based on medical images of the subject's bones taken at a first time point. The prediction information 434 may include, for example, at least one of the following: the probability of a future fracture in the subject, the predicted time of fracture, and the location of the fracture.

[0227] The second generation unit 427 can generate inference information 434, which includes information related to the fracture of the subject, inferred from a medical image of the subject's bones taken at the second time point. The inference information 434 may include information related to the fracture of the subject at the fourth time point. The fourth time point may be a time point different from the second and third time points.

[0228] When the image generation unit 424 generates an image using the speculative information 434, the image generation unit 424 can generate a display image that shows the speculative information 434 corresponding to the establishment of the fourth time point.

[0229] <Other examples of speculative model 4271>

[0230] The prediction model 4271 of the second generation unit 427 involved in this example can perform the following processing. The prediction model 4271 can predict information related to the probability of a future fracture of the subject at a time different from the time when the medical image was taken, based on the medical image data 432 representing the subject's medical image. The prediction model 4271 can be set based on learning data containing medical images (the fourth image) of a third person's bone and teaching data containing information related to the fracture of the third person. The aforementioned "third person" may not specifically refer to a human. For example, it may be an animal of the same species as the "subject".

[0231] For example, for each of the multiple learning image data contained in the learning data, information related to a fracture of a person whose bone is captured in a simple X-ray image representing the bone can be established as teaching data. The teaching data can be at least one of bone evaluation information or a medical image (the fifth image) showing the fracture site of a third person. The fifth image can be an image showing the same location as the fourth image. The fifth image can also be an image showing a different location than the fourth image.

[0232] Teaching data can be data measured at a different time than the time period for capturing the learning image data. Teaching data can also be data measured at approximately the same time period as the time period for capturing the learning image data.

[0233] The prediction model 4271 can be composed of multiple models. For example, the prediction model 4271 can be composed of a model that predicts the bone density of a subject at a time point different from the time point when the medical image was taken, or a model that predicts the probability of a subject at a time point different from the time point when the medical image was taken. The prediction results output from multiple models can be displayed together in the display image displayed on the display unit 45, or only one of them can be displayed.

[0234] The inferred information 434 could also be a method where the first and second methods represent different types of information. For example, the first method could represent information related to bone mineral density, and the second method could represent information related to fractures. For instance, the measured value of the subject's bone mineral density and the estimated value of the subject's bone mineral density can be displayed in the same way on the display image displayed on the display unit 45. Information related to the subject's fractures can be displayed in a different way on the display image displayed on the display unit 45 than the displayed value of the subject's measured bone mineral density and the estimated value of the subject's bone mineral density.

[0235] The display image shown on the display unit 45 can display the locations where a fracture is highly likely to occur in the future, as indicated by the prediction information 434. Figure 15 This diagram illustrates an example of a location shown in the displayed image on display unit 45 that has a high probability of fracture. Figure 15 In the example shown in the left figure, the display image displayed on the display unit 45 includes an X-ray image, i.e., display P13. Display P13 includes a frame indicating an area with a high probability of future fracture, i.e., display P131.

[0236] exist Figure 15 In the example shown in the diagram on the right, the display image displayed on the display unit 45 includes an X-ray image, namely display P14. Display P14 includes a frame line representing the area used in the estimation of bone density, namely display P141, and a frame line representing the area with a high probability of future fracture, namely display P142. For example, labels such as "estimated" can be displayed around the frame line representing the area with a high probability of future fracture, namely display P141 and display P142.

[0237] Display device 4 may have a user interface that accepts user operations to switch between displaying and not displaying predicted values ​​in the displayed image. This interface can accept user operations to select which point in time the predicted value should be displayed. Furthermore, this interface can accept user operations to switch between displaying and not displaying medication, nutritional guidance, and exercise guidance in the displayed image.

[0238] [Implementation Method 2]

[0239] refer to Figure 16 Other embodiments of this disclosure will be described below. Furthermore, for ease of explanation, components having the same function as those described in the above embodiments will be marked with the same reference numerals, and their descriptions will not be repeated.

[0240] Figure 16 This is a block diagram illustrating the main structural components of an example of the information processing system 1a according to this embodiment. For example... Figure 16 As shown, the information processing system 1a includes a measuring device 2, a camera device 3, a display device (image generating device) 4a, a first generating device 4220a, and a second generating device 4270a. Each device communicates via wired or wireless means. Furthermore, this communication can be as follows: Figure 16 As shown, it is done via network 5a.

[0241] The first generation device 4220a performs the same or similar processing as that performed by the first generation unit 422 described in Embodiment 1. The first generation device 4220a transmits the generated estimation information 433 to the display device 4 via the communication unit 4221a.

[0242] The second generation device 4270a performs the same or similar processing as that performed by the second generation unit 427 described in Embodiment 1. The second generation device 4270a receives instructions for generating the prediction information 434 via the communication unit 4271a. Furthermore, the second generation device 4270a transmits the generated prediction information 434 to the display device 4 via the communication unit 4271a.

[0243] Display device 4a includes a communication unit 41a, a control unit 42a, a storage unit 43a, an operation input unit 44, and a display unit 45. The acquisition unit 421a of the control unit 42a receives measurement information 431, estimation information 433, and inference information 434 via the communication unit 41a. Display device 4a performs processing similar to or the same as the image generation processing described in Embodiment 1. Furthermore, if the operation receiving unit 423a receives an instruction to accept an operation input indicating an inference related to the subject's bones, display device 4a sends an instruction to generate inference information 434 to the second generation device 4270a via the communication unit 41a. Operation receiving unit 423a accepts operations similar to or the same as those performed by operation receiving unit 423. Furthermore, the operation input device that performs the processing of operation input unit 44 and operation receiving unit 423a may also be included in the information processing system 1a as a device different from display device 4a. In this case, the operation input device can communicate with display device 4a, the second generation device 4270a, etc., via network 5a.

[0244] [Implementation Method 3]

[0245] refer to Figure 17 Other embodiments of this disclosure will be described below. Furthermore, for ease of explanation, components having the same function as those described in the above embodiments will be marked with the same reference numerals, and their descriptions will not be repeated.

[0246] Figure 17 This is a block diagram illustrating the main structural components of an example of the information processing system 1b according to this embodiment. For example... Figure 17 As shown, the information processing system 1b includes a measuring device 2, a camera device 3, a display device 4b, a first generating device 4220a, a second generating device 4270a, and a server device (image generating device) 6b. Each device communicates via wired or wireless means. Furthermore, this communication can also be as follows: Figure 17 As shown, this is done via network 5a. Furthermore, server device 6b can also be a cloud server.

[0247] The first generation device 4220a performs the same or similar processing as that performed by the first generation unit 422 described in Embodiment 1. The first generation device 4220a sends the generated estimation information 433 to the server device 6b via the communication unit 4221a.

[0248] The second generation device 4270a performs the same or similar processing as that performed by the second generation unit 427 described in Embodiment 1. The second generation device 4270a receives instructions for generating the prediction information 434 via the communication unit 4271a. Furthermore, the second generation device 4270a transmits the generated prediction information 434 to the server device 6b via the communication unit 4271a.

[0249] Server device 6b includes a communication unit 61b, a control unit 62b, and a storage unit 63b. The acquisition unit 621b of the control unit 62b stores the measurement information 431, estimation information 433, and prediction information 434 received via the communication unit 61b into the storage unit 63b. The information acquisition unit 625b and image generation unit 624b of the control unit 62b perform image generation processing in the same or similar manner as the information acquisition unit 425 and image generation unit 424 described in Embodiment 1. Server device 6b transmits the generated image data 430b to display device 4b via the communication unit 61b.

[0250] Display device 4b includes a communication unit 41b, a control unit 42b, a storage unit 43b, an operation input unit 44, and a display unit 45. If the operation receiving unit 423b receives an instruction to accept an operation input indicating a prediction related to the subject's bones, display device 4b sends an instruction to the second generation device 4270a via the communication unit 41b to generate prediction information 434. The operation receiving unit 423b accepts operations similar to or the same as those of operation receiving unit 423. The acquisition unit 421b of control unit 42b acquires image data 430b from server device 6b via the communication unit 41b. The acquisition unit 421b stores the acquired image data 430b in the storage unit 43b and sends a signal indicating that the image data 430b is stored to display control unit 426b. If display control unit 426b receives this signal from acquisition unit 421b, it acquires the image data 430b from storage unit 43b and displays the image represented by the image data 430b on display unit 45.

[0251] [Implementation Method 4]

[0252] refer to Figure 18 Other embodiments of this disclosure will be described below. Furthermore, for ease of explanation, components having the same function as those described in the above embodiments will be marked with the same reference numerals, and their descriptions will not be repeated.

[0253] Figure 18This is a block diagram illustrating the main structural components of an example of the information processing system 1c according to this embodiment. For example... Figure 18 As shown, the information processing system 1c includes a measuring device 2, a camera device 3, and a display device 4c. Hereinafter, only the functions added to the display device 4c as described in Embodiment 1 will be explained. Descriptions of functions in the display device 4c that are identical to those in the display device 4 will not be repeated.

[0254] The display device 4c includes a communication unit 41, a control unit 42c, a storage unit 43c, an operation input unit 44, and a display unit 45.

[0255] <Control Unit 42c>

[0256] The control unit 42c encompasses all parts that control the display device 4. The control unit 42c includes an acquisition unit 421 (first acquisition unit), a first generation unit 422, an operation receiving unit 423c, an image generation unit (generation unit) 424c, an information acquisition unit (first acquisition unit, second acquisition unit, third acquisition unit, and fourth acquisition unit) 425c, a display control unit 426, a second generation unit 427, and a third generation unit 428c.

[0257] In addition to the processing performed by the operation receiving unit 423c, the image generating unit 424c, and the information acquiring unit 425c, the following processing is also performed by each of them.

[0258] Operation Receiving Unit 423c

[0259] The operation receiving unit 423c receives an operation input instructing the display of a display image of a third piece of information, which is bone-related information different from the first and / or second information of the subject. Here, the third information can be bone-related information calculated based on the first and / or second information. Furthermore, the third information can be bone-related information different from the first and / or second information, calculated based on the subject's measurement data and / or medical images used when obtaining the first and / or second information. For example, if the first and / or second information is the subject's bone density, the third information can be the subject's fracture risk information. In this embodiment, the case where the first and second information are the subject's bone density and the third information is fracture risk information is described, but it is not limited to this. Upon receiving this operation input, the operation receiving unit 423c instructs the third generation unit 428c to generate fracture risk information (third information) 435c representing the subject's fracture risk. Furthermore, the operation receiving unit 423c instructs the image generation unit 424c to generate a display image for displaying fracture risk information 435c.

[0260] Part 3, Generation Section 428c

[0261] The third generation unit 428c generates fracture risk information 435c based on an instruction received from the operation receiving unit 423c to generate fracture risk information 435c representing the fracture risk of the subject. The third generation unit 428c stores the generated fracture risk information 435c in the storage unit 43c. For example, the third generation unit 428c may generate fracture risk information 435c as follows.

[0262] The third generation unit 428c generates fracture risk information 435c representing the fracture risk of the subject at the first time point based on the measurement information 431 representing the measured value of the bone density of the subject measured at the first time point.

[0263] Furthermore, the third generation unit 428c generates fracture risk information 435c representing the fracture risk of the subject at the second time point based on the estimated value of bone density estimated from the medical image of the subject's bones taken at the second time point.

[0264] Furthermore, the third generation unit 428c generates fracture risk information 435c representing the fracture risk of the subject at the third time point based on the prediction information 434 representing the predicted value of the bone density of the subject at the third time point.

[0265] That is, the third generation unit 428c generates bone-related fracture risk information 435c of the subject calculated based on at least one of the measured information 431, the estimated information 433, and the inferred information 434.

[0266] The third generation section 428c can calculate the fracture risk using a fracture risk assessment tool based on at least one of the measured, estimated, and inferred values ​​of bone mineral density. For example, a fracture risk assessment tool such as FRAX (registered trademark) can be used.

[0267] On the other hand, the third generation unit 438c can generate fracture risk information 435c for the subject at the first time point based on the bone-related measurement data of the subject measured by the measuring device 2 at the first time point. Furthermore, the third generation unit 428c can generate fracture risk information 435c for the subject at the second time point based on medical images of the subject's bones taken at the second time point. Additionally, the third generation unit 428c can generate fracture risk information 435c for the subject at the third time point based on medical images of the subject's bones taken at the second time point.

[0268] That is, the third generation unit 428c can generate bone-related fracture risk information 435c of the subject calculated based on at least one of the bone-related measurement data of the subject at the first time point measured by the measuring device 2, the fracture risk information 435c of the subject at the second time point, or the medical image of the subject's bone taken at the second time point.

[0269] Image Generation Unit 424c

[0270] If the image generation unit 424c receives an instruction to generate a display image showing fracture risk information 435c, it acquires the measurement information 431, estimation information 433, and fracture risk information 435c from the storage unit 43c via the information acquisition unit 425c. The image generation unit 424c uses the fracture risk information 435c to generate a display image showing the fracture risk information 435c of the subject. The image generation unit 424c sends the generated display image data to the display control unit 426. Details regarding examples of display images generated by the image generation unit 424c will be described later.

[0271] In this embodiment, an example is described whereby the third generation unit 428c calculates fracture risk based on bone mineral density (BMD) values. As another example, the third generation unit 428c can calculate YAM (%), T-score, or Z-score based on BMD values ​​represented by at least one of measurement information 431, estimated information 433, and inferred information 434. That is, the third generation unit 428c can generate information representing YAM (%), T-score, or Z-score. The image generation unit 424c can generate a display image showing the YAM (%), T-score, or Z-score calculated based on at least one of measurement information 431, estimated information 433, and inferred information 434.

[0272] <Example showing the fracture risk of the subject>

[0273] Next, refer to Figure 19 Here is an example of a display image illustrating the fracture risk of a subject. Figure 19 This is an example of an image displaying the fracture risk of a person being shown. In Figure 19 Image D13 shows the measured bone mineral density values ​​m1-m3 at the time points when the measurements were taken, the estimated bone mineral density values ​​n1-n2 at the time points when the medical images were taken, and the shift T1 of bone mineral density based on these measured and estimated values. Furthermore, the predicted values ​​o1 and o2 are shown in image D13. Regarding image D13, the same reference numerals are used for displays identical to those described in image D1 above, and their description will not be repeated here.

[0274] Image D13 shows fracture risk q1, fracture risk q2, fracture risk q3, and fracture risk q4, representing fracture risk. Fracture risk q1 represents the fracture risk calculated based on the estimated value n1. Fracture risk q2 represents the fracture risk calculated based on the estimated value n2. Fracture risk q3 represents the fracture risk calculated based on the measured value m3. Fracture risk q4 represents the fracture risk calculated based on the predicted value o2.

[0275] Fracture risk calculated from measured values, fracture risk calculated from estimated values, and fracture risk calculated from predicted values ​​can be displayed in different ways. Figure 19 In the example shown, fracture risks q1 and q2 calculated based on estimated values ​​are represented by black circles, fracture risks q3 calculated based on measured values ​​are represented by hollow circles, and fracture risks q4 calculated based on predicted values ​​are represented by hollow triangles.

[0276] [Implementation Method 5]

[0277] refer to Figure 20 Other embodiments of this disclosure will be described below. Furthermore, for ease of explanation, components having the same function as those described in the above embodiments will be marked with the same reference numerals, and their descriptions will not be repeated.

[0278] Figure 20 This is a block diagram illustrating the main structural components of an example of the information processing system 1d according to this embodiment. For example... Figure 20 As shown, the information processing system 1d includes a measuring device 2, a camera device 3, a display device (image generation device) 4d, a first generation device 4220a, a second generation device 4270a, and a third generation device 4280d. Each device communicates via wired or wireless means. Furthermore, this communication can be as follows: Figure 20 As shown, this is done via network 5a. The processing described below can also be performed by other devices included in the information processing system 1d, which are not limited to the individual functions performed by the devices included in the information processing system 1d.

[0279] The first generating apparatus 4220a performs the same or similar processing as that performed by the first generating apparatus 4220a described in Embodiment 2.

[0280] The second generation device 4270a performs the same or similar processing as that performed by the second generation device 4270a described in Embodiment 2. The second generation device 4270a receives instructions for generating the prediction information 434 via the communication unit 4271a. Furthermore, the second generation device 4270a transmits the generated prediction information 434 to the display device 4d via the communication unit 4271a.

[0281] The third generation device 4280d performs the same or similar processing as that performed by the third generation unit 428c described in Embodiment 4. The storage unit 4283d provided in the third generation device 4280d stores measurement information 431, estimation information 433, and prediction information 434. For example, the third generation device 4280d obtains this information via network 5a.

[0282] The third generating device 4280d receives instructions to generate fracture risk information 435c via the communication unit 4281d. Furthermore, the third generating device 4280d transmits the generated fracture risk information 435c to the display device 4d via the communication unit 4281d.

[0283] The display device 4d includes a communication unit 41d, a control unit 42d, a storage unit 43d, an operation input unit 44, and a display unit 45. The acquisition unit 421d of the control unit 42d receives measurement information 431, estimation information 433, prediction information 434, and fracture risk information 435c via the communication unit 41d. The display device 4d performs processing similar to or the same as the image generation processing described in Embodiment 4. Furthermore, if the operation receiving unit 423d receives an operation input indicating the display of fracture risk for the subject, the display device 4d sends an instruction to generate fracture risk information 435c to the third generation device 4280d via the communication unit 41d. The operation receiving unit 423d accepts the operation received by the operation receiving unit 423c. Furthermore, the operation input device that performs the processing of the operation input unit 44 and the operation receiving unit 423c can be included as a device different from the display device 4d in the information processing system 1d. In this case, the operation input device can communicate with the display device 4d, the second generation device 4270a, the third generation device 4280d, etc. via the network 5a.

[0284] [Implementation Method 6]

[0285] refer to Figure 21 Other embodiments of this disclosure will be described below. Furthermore, for ease of explanation, components having the same function as those described in the above embodiments will be marked with the same reference numerals, and their descriptions will not be repeated.

[0286] Figure 21This is a block diagram illustrating the main structural components of an example of the information processing system 1e according to this embodiment. For example... Figure 21 As shown, the information processing system 1e includes a measuring device 2, a camera device 3, a display device 4e, a first generating device 4220a, a second generating device 4270a, a third generating device 4270d, and a server device (image generating device) 6e. Each device communicates via wired or wireless means. Furthermore, this communication can also be as follows: Figure 21 As shown, this is done via network 5a. Furthermore, server device 6e can also be a cloud server. Other devices that include information processing system 1e may also perform the processing described below, as performed by the devices included in information processing system 1e. That is, the partial existence of the functions performed by the devices included in information processing system 1e is not limited.

[0287] The first generation device 4220a performs the same or similar processing as that performed by the first generation unit 422 described in Embodiment 1. The first generation device 4220a sends the generated estimation information 433 to the server device 6e via the communication unit 4221a.

[0288] The second generation device 4270a performs the same or similar processing as that performed by the second generation unit 427 described in Embodiment 1. The second generation device 4270a receives instructions for generating the prediction information 434 via the communication unit 4271a. Furthermore, the second generation device 4270a transmits the generated prediction information 434 to the server device 6e via the communication unit 4271a.

[0289] The third generation device 4280d performs the same or similar processing as that performed by the third generation unit 428c described in Embodiment 4. The third generation device 4280d receives instructions to generate fracture risk information 435c via the communication unit 4281d. Furthermore, the third generation device 4280d sends the generated fracture risk information 435c to the server device 6e via the communication unit 4281d.

[0290] Server device 6e includes a communication unit 61b, a control unit 62e, and a storage unit 63e. The acquisition unit 621e of the control unit 62e stores the measurement information 431, estimation information 433, prediction information 434, and fracture risk information 435c received via the communication unit 61b into the storage unit 63e. The information acquisition unit (fourth acquisition unit) 625e and the image generation unit 624e of the control unit 62e perform the same or similar image generation processing as described in Embodiment 4, performed by the information acquisition unit 425c and the image generation unit 424c. Server device 6e transmits the generated image data 430b to display device 4e via the communication unit 61b.

[0291] In addition to the processing performed by display device 4b, display device 4e performs the following processing. Display device 4e includes a communication unit 41b, a control unit 42e, a storage unit 43b, an operation input unit 44, and a display unit 45. If operation receiving unit 423e receives an operation input instructing the display of fracture risk information for the subject, display device 4e sends an instruction to the third generation device 4280d to generate fracture risk information 435c via communication unit 41b. Furthermore, display device 4e instructs server device 6e to generate display image data 430b showing the fracture risk of the subject. Operation receiving unit 423e accepts the operation received by operation receiving unit 423. Acquisition unit 421b of control unit 42e acquires image data 430b from server device 6e via communication unit 41b. Acquisition unit 421b stores the acquired image data 430b in storage unit 43b and sends a signal indicating that image data 430b has been stored to display control unit 426b. If the display control unit 426b receives the signal from the acquisition unit 421b, it acquires the image data 430b from the storage unit 43b and displays the image represented by the image data 430b on the display unit 45.

[0292] [Software-based implementation example]

[0293] The functions of the display devices (4, 4a, 4b, 4c, 4d, 4e), the first generation device (4220a), the second generation device (4270a), the third generation device (4280d), and the server devices (6b, 6e) (hereinafter referred to as "devices") can be realized by a program that enables the computer to function as the device. This program is used to enable the computer to function as each control block (part in particular the part contained in the control unit) of the device.

[0294] In this case, the aforementioned apparatus includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory), serving as hardware for executing the aforementioned program. The functions described in the above embodiments are realized by executing the program using the control device and the storage device.

[0295] The above-described program can be recorded on one or more non-transitory, computer-readable recording media. The device may or may not include such a recording medium. In the latter case, the program can also be supplied to the device via any wired or wireless transmission medium.

[0296] Furthermore, some or all of the functions of the aforementioned control blocks can also be implemented through logic circuits. For example, integrated circuits that form the logic circuits that enable the functions of the aforementioned control blocks are also included within the scope of this disclosure. In addition, the functions of the aforementioned control blocks can also be implemented, for example, through a quantum computer.

[0297] Furthermore, the processes described in the above embodiments can also be executed by AI (Artificial Intelligence). In this case, the AI ​​can operate in the aforementioned control device or in other devices (such as edge computers or cloud servers).

[0298] The invention disclosed herein has been described above based on the accompanying drawings and embodiments. However, the invention disclosed herein is not limited to the embodiments described above. That is, the invention disclosed herein can be modified in various ways within the scope shown in this disclosure, and embodiments obtained by combining appropriate combinations of the technologies disclosed in different embodiments are also included within the scope of the technology of the invention disclosed herein. In other words, it is desirable to note that those skilled in the art can easily make various modifications or alterations based on this disclosure. Furthermore, it is desirable to note that these modifications or alterations are also included within the scope of this disclosure.

[0299] [Summary]

[0300] The information processing system according to Embodiment 1 of this disclosure includes: a first acquisition unit that acquires first information representing bone-related information of a subject obtained based on a first method; a second acquisition unit that acquires second information representing bone-related information of the subject obtained based on a second method different from the first method; and a generation unit that generates a display image displaying the first information and the second information.

[0301] The information processing system involved in Method 2 of this disclosure may also be, in Method 1, information related to at least one of the subject's bone density, bone mass, and bone quality, where the first information and the second information are related.

[0302] In the information processing system of Method 3 of this disclosure, in Method 2, the first acquisition unit acquires the first information measured at a first time point, the second acquisition unit acquires the second information representing at least one of bone density, bone mass, and bone quality estimated by a estimation model based on a first image of the subject's bone taken at a second time point different from the first time point, and the generation unit generates the display image in which the first information and the second information are arranged in the time sequence order of the first time point and the second time point.

[0303] The information processing system involved in Method 4 of this disclosure may also be described in Method 3, wherein the estimation model is set based on learning data including a second image of the bones of the first person and teaching data including at least one of the bone density, bone mass and bone quality of the first person.

[0304] The information processing system according to Embodiment 5 of this disclosure may, in Embodiment 3 or 4, further include: a third acquisition unit that acquires speculative information related to at least one of bone density, bone mass, and bone quality, which is speculatively derived from an image of the subject's bones taken at the first or second time point by a speculative model; if the image of the subject's bones taken is taken at the first time point, the speculative information is information from a third time point different from the first time point; if the image of the subject's bones taken is taken at the second time point, the speculative information is information from a third time point different from the second time point; and the generation unit generates a display image that displays the speculative information corresponding to the third time point.

[0305] In the present disclosure, the information processing system of method 6 may also be described in method 5, wherein the inference model is set based on learning data including a third image of the bones of the second person and teaching data including at least one of the bone density, bone mass and bone quality of the second person.

[0306] The information processing system involved in Method 7 of this disclosure may also be described in Method 6, wherein the inferred information is information related to the bone density of the subject who has received treatment regarding at least one of bone density, bone mass, and bone quality, and the second person is a person who has received treatment regarding at least one of bone density, bone mass, and bone quality, and the inferred model is set based on learning data including a third image of the bones of the second person who has received the treatment, and teaching data including at least one of the bone density, bone mass, and bone quality of the second person.

[0307] In the information processing system of Method 8 of this disclosure, in any of Methods 5 to 7, if the image of the subject's bones is captured at the first time point, the third time point may be a time point before or after the first time point; if the image of the subject's bones is captured at the second time point, the third time point may be a time point before or after the second time point.

[0308] The information processing system according to Embodiment 9 of this disclosure, in any one of Embodiments 5 to 8, further comprises: a fourth acquisition unit that acquires third information related to the bone of the subject calculated based on at least one of the first information, the second information, and the inferred information, and the generation unit generates the display image displaying the third information.

[0309] In any of embodiments 3 to 9 of the information processing system according to embodiment 10 of this disclosure, the generation unit may generate the display image that displays information representing the name of the bone captured in the first image used to estimate the second information.

[0310] In any of the embodiments 1 to 10, the information processing system according to Embodiment 11 of this disclosure may also generate a display image that shows the second information in a display mode different from the first information.

[0311] In any of the methods 11 of this disclosure, the information processing system may also generate a display image that shows information about the duration of treatment for the subject regarding at least one of bone density, bone mass, and bone quality.

[0312] In any of the embodiments 1 to 12, the information processing system according to embodiment 13 of this disclosure may also generate a display image that shows the shift information of at least one of bone density, bone mass, and bone quality of a reference person different from the subject.

[0313] In any of the embodiments 1 to 13, the information processing system according to embodiment 14 of this disclosure may also generate the display image that shows information representing a criterion for determining at least one of bone density, bone mass, and bone quality.

[0314] In any of the embodiments 1 to 14, the information processing system according to Embodiment 15 of this disclosure may also include a generation unit that generates a display image showing the first method and the second method.

[0315] The information processing system according to Embodiment 16 of this disclosure may also include, in any of Embodiments 1 to 15, a measuring device that performs a measurement on bone, wherein the first information uses information measured by the measuring device that performs the measurement on bone.

[0316] The information processing system according to Embodiment 17 of this disclosure, in any one of Embodiments 3 to 10, includes: a camera device that captures an image of a bone, wherein the first image is an image captured using the camera device.

[0317] The information processing method disclosed in Method 18 includes: a first acquisition step of acquiring first information representing bone-related information of a subject obtained based on a first technique; a second acquisition step of acquiring second information representing bone-related information of the subject obtained based on a second technique different from the first technique; and a generation step of generating a display image displaying the first information and the second information.

[0318] The image generation apparatus according to Embodiment 19 of this disclosure includes: a first acquisition unit that acquires first information representing bone-related information of a subject obtained based on a first method; a second acquisition unit that acquires second information representing bone-related information of the subject obtained based on a second method different from the first method; and a generation unit that generates a display image displaying the first information and the second information.

[0319] The image generation program involved in Embodiment 20 of this disclosure is an image generation program for enabling a computer to function as the information processing system described in Embodiment 19, and for enabling the computer to function as the first acquisition unit, the second acquisition unit, and the generation unit.

[0320] Symbol Explanation

[0321] 1, 1a, 1b, 1c, 1d, 1e Information processing systems

[0322] 4, 4a, 4c, 4d Display devices (image generating devices)

[0323] 6b, 6e Server Devices (Image Generation Devices)

[0324] Acquisition sections 421, 421a, 421c, and 621b (First Acquisition Section)

[0325] Image generation unit (generation unit) of 424, 424c, 624b, 624e

[0326] 425, 425c, 625b, 625e Information Acquisition Department (First Acquisition Department, Second Acquisition Department, Third Acquisition Department, Fourth Acquisition Department)

[0327] 431 Measurement Information (Information 1)

[0328] 433 Presumed Information (Second Information)

[0329] 434 Speculated Information

[0330] 435c Fracture Risk Information (Information 3)

[0331] 4221 Presumed Model

[0332] 4271 Speculative Model

[0333] S2 Step 1

[0334] S3 Step 2

[0335] S4 generation steps.

Claims

1. An information processing system, comprising: The first acquisition section, whose acquisition refers to the first information obtained based on the first method, which is related to the bone of the subject; and The generation unit generates a display image that displays the first information and information related to the first method together.

2. The information processing system according to claim 1, wherein, The information related to the first method is related to presumed or speculative information. The presumed information is estimated by a presuming model based on an image of the subject's bones taken at a certain time point and is related to at least one of bone density, bone mass, and bone quality. The speculative information is inferred by a speculative model based on an image of the subject's bones taken at a certain time point and is related to at least one of bone density, bone mass, and bone quality.

3. The information processing system according to claim 1, wherein, The information processing system further includes a second acquisition unit, which acquires second information representing bone-related information of the subject obtained from the subject using a second method different from the first method. The generation unit displays the first information and the second information.

4. The information processing system according to claim 3, wherein, The first and second information are information related to at least one of the subject's bone density, bone mass, and bone quality.

5. The information processing system according to claim 4, wherein, The first acquisition unit acquires the first information measured at the first time point. The second acquisition unit acquires the second information, which is inferred from the estimation model based on a first image of the subject's bones taken at a second time point different from the first time point. The generation unit generates a display image that arranges the first information and the second information in a time sequence order according to the first time point and the second time point.

6. The information processing system according to claim 5, wherein, The inference model is set based on learning data including a second image of the bones of the first person and teaching data including at least one of the first person's bone density, bone mass, and bone quality.

7. The information processing system according to claim 5 or 6, wherein, The information processing system also has: The third acquisition unit acquires inferred information related to at least one of bone density, bone mass, and bone quality, inferred from an inference model based on an image of the subject's bones taken at the first or second time point. If the image of the subject's bones was captured at the first time point, the inferred information is information from a third time point, different from the first time point. If the image of the subject's bones was captured at the second time point, the inferred information is information from a third time point different from the second time point. The generation unit generates a display image that shows the speculative information corresponding to the third time point.

8. The information processing system according to claim 7, wherein, The inference model is set based on learning data including a third image of the second person's bones and teaching data including at least one of the second person's bone density, bone mass, and bone quality.

9. The information processing system according to claim 8, wherein, The inferred information is information related to the bone mineral density of the subject, assuming that the subject has undergone treatment regarding at least one of bone mineral density, bone mass, and bone quality. The second person is someone who has received treatment for at least one of the following: bone mineral density, bone mass, and bone quality. The inference model is set based on learning data including a third image of the bones of the second person who received the treatment, and teaching data including at least one of the second person's bone density, bone mass, and bone quality.

10. The information processing system according to any one of claims 7 to 9, wherein, If the image of the subject's bones was captured at the first time point, then the third time point is either a time point before or after the first time point. If the image of the subject's bones is captured at the second time point, then the third time point is either a time point before or after the second time point.

11. The information processing system according to any one of claims 7 to 10, wherein, The information processing system also has: The fourth acquisition unit acquires third information related to the bones of the subject, calculated based on at least one of the first information, the second information, and the inferred information. The generation unit generates the display image that displays the third information.

12. The information processing system according to any one of claims 5 to 11, wherein, The generation unit generates a display image that displays information indicating the name of the bone captured in the first image used to estimate the second information.

13. The information processing system according to any one of claims 3 to 12, wherein, The generation unit generates a display image that shows the second information in a display mode different from the first information.

14. The information processing system according to any one of claims 1 to 13, wherein, The generating unit generates the display image that shows information about the duration of treatment for the subject regarding at least one of bone density, bone mass, and bone quality.

15. The information processing system according to any one of claims 1 to 14, wherein, The generation unit generates the display image that shows the shift information of at least one of bone density, bone mass, and bone quality in a reference person different from the subject.

16. The information processing system according to any one of claims 1 to 15, wherein, The generation unit generates the display image that shows information representing the criteria for determining at least one of bone density, bone mass, and bone quality.

17. The information processing system according to any one of claims 3 to 13, wherein, The generation unit generates the display image, which includes a display representing the first technique and the second technique.

18. The information processing system according to any one of claims 3 to 13, wherein, The information processing system has the following features: The measuring device performs measurements on bone. The second piece of information includes information measured using the measuring device.

19. The information processing system according to any one of claims 5 to 12, wherein, The information processing system has the following features: The camera device captures images of the bones. The first image is an image captured using the camera device.

20. An information processing method, comprising: The first acquisition step involves acquiring first information representing bone-related information about the subject obtained based on the first method. The second acquisition step involves acquiring second information representing bone-related information about the subject obtained using a second technique different from the first technique; and The generation step generates a display image that shows the first information and the second information.

21. An image generation apparatus comprising: The first acquisition section, whose acquisition refers to the first information related to the bone of the subject obtained based on the first method; The second acquisition section acquires second information related to the bones of the subject, obtained using a second method different from the first method; and The generation unit generates a display image that shows the first information and the second information.

22. An image generation program for enabling a computer to function as the image generation apparatus of claim 21. The image generation program is used to enable the computer to function as the first acquisition unit, the second acquisition unit, and the generation unit.

Images