Information processing system, terminal device, method for controlling information processing system, control program, and recording medium

Abstract

The present invention accurately estimates bone state information pertaining to the bone state in the future of a subject. This information processing system comprises: an acquisition unit that acquires medical information comprising first medical information including a medical image showing a bone of a subject, and second medical information including feature information of the subject at a point in time different from a point in time at which the medical image was captured; and an estimating unit that inputs the medical information into a learning model to estimate bone state information pertaining to a state of the bone in the future or past of the subject, the learning model having been trained using teacher data which includes a plurality of items of information including a medical image showing a bone of a predetermined person and feature information of the predetermined person at a point in time different from the point in time at which the medical image was captured.

Description

Information processing system, terminal device, control method for information processing system, control program, and recording medium

[0001] The present disclosure relates to an information processing system, a terminal device, a control method for an information processing system, and the like that estimate bone condition information regarding a subject's future bone condition.

[0002] Patent Document 1 discloses a technology that uses a prediction model to output information regarding future symptoms of a subject's joints.

[0003] International Publication No. 2021 / 182595

[0004] An information processing system according to one aspect of the present disclosure includes an acquisition unit that acquires medical information including first medical information having a medical image showing the bones of a subject and second medical information having characteristic information of the subject at a time different from the time the medical image was taken, and an estimation unit that estimates bone condition information regarding the future or past bone condition of the subject by inputting the medical information into a learning model that has been trained using training data including a plurality of pieces of information having medical images showing the bones of a specific person and the characteristic information of the specific person at a time different from the time the medical image was taken.

[0005] Furthermore, a control method for an information processing system according to one aspect of the present disclosure includes an acquisition step of acquiring medical information including first medical information having a medical image showing the bones of a subject and second medical information having characteristic information of the subject at a time point different from the time point when the medical image was taken, and an estimation step of estimating bone condition information regarding the future or past bone condition of the subject by inputting the medical information into a learning model trained using training data including a plurality of pieces of information having medical images showing the bones of a specific person and characteristic information of the specific person at a time point different from the time point when the medical image was taken.

[0006] 1 is a diagram illustrating an example configuration of an information processing system according to an aspect of the present disclosure; FIG. 2 is a block diagram illustrating an example configuration of an information processing device according to an aspect of the present disclosure; FIG. 3 is a diagram illustrating an example configuration of a learning model executed by an estimation unit; FIG. 4 is a flowchart illustrating an example flow of learning processing by a learning unit; FIG. 5 is a flowchart illustrating an example flow of processing performed by an information processing device; FIG. 6 is a diagram illustrating an example configuration of an information processing system according to another aspect of the present disclosure; FIG. 7 is a flowchart illustrating an example flow of learning processing by a learning unit; and FIG.

[0007] Each embodiment of the present disclosure will be described below. In the following description, the subject for which bone condition information regarding future or past bone conditions is to be estimated will be a human (i.e., a "subject"), but the subject is not limited to a human. That is, the "subject" according to the present disclosure may be, for example, a non-human mammal or vertebrate, such as an equine, feline, canine, bovine, or porcine. The present disclosure also includes, among the following embodiments, embodiments in which "subject" is replaced with "animal" if the embodiment is applicable to these animals.

[0008] First Embodiment Hereinafter, an information processing device 1 according to an embodiment of the present disclosure will be described in detail as an example.

[0009] The information processing device 1 outputs estimated bone condition information relating to the future or past bone condition of a subject, estimated using a learning model based on input information including medical information of the subject. The medical information of the subject includes first medical information having medical images depicting the subject's bones and second medical information having feature information of the subject at a time point different from the time point when the medical images were captured. The learning model is trained using training data including multiple pieces of information having medical images depicting the bones of a specific person and the feature information of the specific person at a time point different from the time point when the medical images were captured.

[0010] (Configuration of Information Processing System 100a) First, the configuration of the information processing system 100a according to one aspect of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an example configuration of the information processing system 100a in a medical facility 8 in which an information processing device 1 has been introduced.

[0011] The information processing system 100a includes an information processing device 1 and one or more terminal devices 30 communicably connected to the information processing device 1. The information processing device 1 estimates bone condition information regarding a subject's future or past bone condition from medical information including first medical information having a medical image of the subject's bones and second medical information having characteristic information of the subject at a time different from the time the medical image was captured. In the following description, an example of estimating (predicting) bone condition information regarding a subject's future bone condition will be described, and "bone condition information regarding a future bone condition" will also be simply referred to as "future bone condition information." The information processing device 1 is a computer that transmits estimated information regarding the subject's future bone condition information to the terminal device 30. The information processing device 1 may be a cloud-based device installed on a cloud. In this case, the terminal device 30 transmits the first medical information to the information processing device 1 on the cloud via a communication network 9 and receives the estimated information estimated by the information processing device 1 via the communication network 9. The terminal device 30 may be an on-premise device installed in a medical facility 8 or a company providing analysis services.

[0012] The terminal device 30 functions as an output unit in the information processing system 100a and presents information received from the information processing device 1. The terminal device 30 is a computer used by medical personnel such as doctors (medical personnel) belonging to the medical facility 8. The terminal device 30 is, for example, a personal computer, a tablet terminal, a smartphone, etc. The terminal device 30 has a communication unit that transmits and receives data to and from other devices, an input unit such as a keyboard and a microphone, a display unit that can display information transmitted from the information processing device 1, an output unit such as a speaker, etc.

[0013] 1 illustrates an example in which a local area network (LAN) is installed within the medical facility 8, and the information processing device 1 and the terminal device 30 are connected to the LAN. However, this is not limiting. For example, the network within the medical facility 8 may be the Internet, a telephone communication line network, an optical fiber communication network, a cable communication network, a satellite communication network, or the like. The information processing system 100a is compatible with a hospital information system (HIS), a radiology information system (RIS), a picture archiving and communication system (PACS), or the like within the medical facility 8. Furthermore, communication in the information processing system 100a complies with international standards such as DICOM (Digital Imaging and Communications in Medicine).

[0014] In addition to the information processing device 1 and terminal device 30, a medical image management device 3, an attribute information management device 4, a test value management device 5, a diagnosis result management device 6, and a bone information management device 7 may be communicatively connected to the LAN within the medical facility 8.

[0015] The medical image management device 3 is a computer that functions as a server for managing medical images taken at a medical facility 8. The medical image management device 3 may be installed, for example, within the medical facility 8, or may be a cloud. When the medical image management device 3 is a cloud, medical images can be acquired via a communication network 9.

[0016] The medical image may be an image showing at least a portion of a bone. The medical image may be an image showing at least a portion of at least one of trabecular bone, cortical bone, and / or cancellous bone. The medical image may also be an image that does not show bone. The medical image may be any of an X-ray image, a computed tomography (CT) image, a magnetic resonance imaging (MRI) image, an ultrasound image taken by an ultrasound diagnostic device, a positron emission tomography (PET) image, and a dual-energy X-ray absorptiometry (DXA) image. The medical image may be, for example, an inspection device image acquired from an inspection device (e.g., an X-ray inspection device), or an image in which noise has been reduced from an inspection device image acquired by the inspection device. Noise reduction can be achieved, for example, by machine learning using training data that classifies portions of training images as having or not having noise. The information processing device 1 may acquire medical images showing the subject's bones from the medical image management device 3. The X-ray image may include, for example, a panoramic X-ray image used for dentistry. A panoramic X-ray image is, for example, an image that includes multiple teeth (for example, all teeth).

[0017] The medical image may be at least one of a frontal image showing the target region from the front (e.g., an image obtained by irradiating the target region with X-rays in the front-back direction) and a lateral image showing the target region from the side (e.g., an image obtained by irradiating the target region with X-rays in the left-right direction). For example, a frontal chest X-ray image including a person's chest or a frontal lumbar X-ray image including a person's lumbar region may be used as the medical image. The chest X-ray image is, for example, an image showing at least one of the ribs, clavicle, and sternum. The lumbar X-ray image is, for example, an image showing at least one of the lumbar vertebrae, pelvis, and femur. The medical image is not limited to the lumbar region or chest, and may also be, for example, an image showing the teeth, jaw, arm, hand, shoulder joint, knee joint, heel, skull, or foot bones.

[0018] When a CT image is used as a medical image, information about trabeculae based on a three-dimensionally constructed image may be used, or information about trabeculae based on a two-dimensionally captured image may be used. When a CT image is used as a medical image, for example, at least one of a three-dimensional image, a cross-sectional image in a direction perpendicular to the body axis connecting the head and legs (e.g., horizontal section), and a cross-sectional image in a direction parallel to the body axis (e.g., sagittal section or coronal section) may be used.

[0019] The area shown in the medical image may be any of the head, neck, chest, waist, hip joints, knee joints, ankle joints, feet, toes, shoulder joints, elbow joints, wrist joints, hands, and fingers, and the medical image may also be a dental image.

[0020] The attribute information management device 4 is a computer that functions as a server for managing the subject's attribute information. The information processing device 1 may acquire the subject's attribute information from the attribute information management device 4. If the acquired medical images are accompanied by this attribute information, the information processing device 1 may extract the attribute information from the medical images. The attribute information includes at least one of age, sex, height, weight, race, lifestyle information, medication information, occupational information, blood test information, urine test information, saliva test information, medical history, the subject's family's medical history, genetic information, birth information, menopausal information, items from the Fracture Risk Assessment Tool (FRAX (registered trademark)), and menopausal stage estimated based on hormone information. The birth information includes at least one of whether the subject has given birth, the number of births, etc. The lifestyle may be, for example, sleep time, wake-up time, sleep duration, daily exercise amount, dietary content, meal times, meal durations, blood glucose levels, etc. The dietary content may include, for example, at least one of the name of a dish, ingredients consumed, and intake amount. The dietary content may be, for example, an estimated intake of at least one of calcium, vitamin B, vitamin D, and vitamin K. The blood glucose level may be, for example, a designated value estimated from parameters acquired by a wearable device.

[0021] The medication information may include, for example, information such as the name of the medication, the amount taken, and the duration of medication. The information about medications taken may include information about steroid drugs being used. The blood test information may be, for example, information about the results of at least one of a biochemistry test, a glucose metabolism test, and an endocrine system test.

[0022] The test value management device 5 is a computer that functions as a server for managing test values ​​obtained from tests performed at the medical facility 8. The test values ​​include at least one of bone density, Kellgren-Lawrence (KL) classification, bone morphology angle, muscle mass, MMSE (Mini Mental State Examination), blood test values ​​(e.g., at least one of bone formation marker, bone resorption marker, and vitamin K value), liver function marker, uric acid level, and malignant tumor marker. The bone evaluation information may include information evaluated by a fracture risk assessment tool. The information processing device 1 may obtain the test values ​​of the subject from the test value management device 5.

[0023] The diagnostic results management device 6 is a computer that functions as a server for managing diagnostic results obtained by diagnoses performed at the medical facility 8. The information processing device 1 may acquire the diagnostic results of the subject from the diagnostic results management device 6. The diagnostic results may include at least one of the presence or absence of a fracture and the progression of osteoporosis. For the fracture, information on the cause, such as a fragility fracture, a stress fracture, or a traumatic fracture, may be added.

[0024] The bone information management device 7 is a computer that functions as a server for managing bone condition information acquired at the medical facility 8. The bone condition information may include, for example, at least one of the following: presence or absence of fracture, possibility of osteoporosis, drug efficacy, occurrence of incident, bone mass, bone density, trabecular number, trabecular spacing, trabecular width, trabecular orientation, trabecular connectivity density, and trabecular bone structure index (trabecular bone score). The bone condition information may be obtained by analyzing bone strength from the state of cortical bone and / or trabecular bone. The information processing device 1 may acquire the subject's past bone condition information from the bone information management device 7.

[0025] The possibility of osteoporosis may be a classification result based on at least one of the presence or absence of a fracture, the possibility of a fracture, and a change in bone mineral density. The possibility of osteoporosis may include "no osteoporosis," "suspected osteoporosis," or "yes, osteoporosis." More specifically, when there is no disease that reduces bone mass, secondary osteoporosis is not observed, and a fracture is present or is highly likely, the possibility of osteoporosis may be classified as primary osteoporosis. The drug effect may include, for example, the name of a drug that improves bone condition when taken or / and administered for a certain period of time, or may include information indicating at least one of bone mineral density, bone mass, and trabecular bone condition after a certain period of time. The occurrence of an incident may include, for example, implant loosening, implant dislodgement, or infection around an implant. Examples of implants include artificial hip joints, artificial knee joints, spinal implants, bolts implanted in bone, and dental implants.

[0026] The LAN in the medical facility 8 may be communicably connected to an external communication network. In the medical facility 8, the information processing device 1 and the terminal device 30 may be directly connected without going through a LAN.

[0027] (Configuration of Information Processing Device 1) Next, the configuration of the information processing device 1 applied to the information processing system 100a shown in Fig. 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the information processing device 1.

[0028] The information processing device 1 includes a control unit 2 that performs overall control of each unit of the information processing device 1, and a storage unit 10 that stores various data used by the control unit 2. The control unit 2 includes an acquisition unit 21, an estimation unit 23, an output unit 24, and a learning unit 25. The storage unit 10 stores a control program 31, which is a program for performing various controls of the information processing device 1, as well as teacher data 32 and a trained learning model 33.

[0029] <Acquisition Unit 21> The acquisition unit 21 acquires input information including medical information of the subject. The input information is data input to the estimation unit 23. The input information includes first medical information having a medical image showing the subject's bones and second medical information having feature information of the subject at a time different from the time the medical image was captured. In the following description, the medical image showing the subject's bones included in the first medical information may be referred to as the first medical image. The second medical information may be information obtained, for example, 15 days, 1 month, 3 months, 6 months, 1 year, 3 years, or 5 years after the medical image of the first medical information was captured. The second medical information may be past information before the time the medical image included in the first medical information was captured, or may be future information after the time.

[0030] The characteristic information included in the second medical information includes at least one of a medical image of the subject, past bone condition information, attribute information, test values, and a diagnosis result. In the following description, a medical image of the subject's bones included in the second medical information may be referred to as a second medical image. The second medical information may be information at multiple points in time, or may be information at a single point in time.

[0031] The acquisition unit 21 may, for example, acquire a first medical image and / or a second medical image from the medical image management device 3, acquire attribute information of the subject from the attribute information management device 4, acquire test values ​​of the subject from the test value management device 5, acquire diagnostic results of the subject from the diagnostic result management device 6, or acquire past bone condition information of the subject from the bone information management device 7. For example, the acquisition unit 21 acquires, as input information, an X-ray image taken at a first time point as the first medical information and an X-ray image taken at a time point different from the first time point, for example, a second time point earlier than the first time point, as the second medical information. The acquisition unit 21 may further acquire, as input information, an X-ray image taken at a third time point earlier than the second time point as the second medical information.

[0032] The first medical information may include the same information elements as the feature information contained in the second medical information. In other words, the first medical information may include, in addition to the first medical image, the same information as the feature information contained in the second medical information. For example, if the second medical information includes the subject's height information as feature information, the first medical information may include, in addition to the first medical image, the subject's height information at the time the first medical image was captured.

[0033] <Estimation unit 23> The estimation unit 23 estimates future bone condition information of the subject by inputting the input information acquired by the acquisition unit 21 into the learning model 33. Here, the learning model 33 is trained in advance using training data 32. The training data 32 is data including medical information of a specific person. The specific person may be a patient suffering from a bone-related disease or a person without a disease. The medical information included in the training data 32 may include a plurality of pieces of information including medical images showing the bones of the specific person and characteristic information of the specific person at a time point different from the time point at which the medical images were captured.

[0034] The estimation unit 23 may estimate future or past bone condition information of the subject by inputting, as the input information, input information to the learning model 33, including first medical information having medical images of the subject's bones and second medical information having characteristic information of the subject at a time different from the time the medical images were captured. The estimation unit 23 may also output information such as values ​​related to the teacher data 32 used to train the learning model 33. By inputting the input information to the learning model 33, the estimation unit 23 may estimate, as the bone condition information, at least one of the following: the probability of the subject's future or past fracture; the time at which a future or past fracture will occur or has occurred in the subject (e.g., 4 to 5 years later); the subject's future or past bone density; and the subject's future or past bone quality. More specifically, when the bone condition information is bone density, the estimation unit 23 can estimate, for example, what the bone density was when the first medical information was acquired in the past, even if bone density has not been measured in the past. The estimation unit 23 may also estimate the subject's bone density at multiple future and past time points from the first medical information. The estimation unit 23 may also estimate a transition in the subject's bone density from a certain time point in the past to a certain time point in the future. The information processing device 1 may graph the estimated transition and display it on the display unit.

[0035] The bone density may be measured by actually measuring bone density from at least one of the hand, lumbar vertebrae, proximal femur, tibia, heel, and arm (e.g., radius, etc.). For bone density measurement, for example, single-energy X-ray absorptiometry, dual-energy X-ray absorptiometry, ultrasound, microdensitometry (MD), or quantitative computed tomography (CT) may be used. In a DXA device that measures bone density using the DXA method, when measuring bone density of the lumbar vertebrae, X-rays are irradiated from the front of the lumbar vertebrae of the subject. In a DXA device that measures bone density of the proximal femur, X-rays are irradiated from the front of the proximal femur of the subject. Here, "front of the lumbar vertebrae" and "front of the proximal femur" refer to the direction that correctly faces the imaging site, such as the lumbar vertebrae and the proximal femur, and may be on the ventral side of the subject's body or on the dorsal side of the subject. The proximal femur includes, for example, at least one of the neck, trochanter, shaft, and the entire proximal femur (neck, trochanter, and shaft). In the MD method, for example, the hand is irradiated with X-rays.

[0036] Bone mineral density may be a value related to the density of bone. Bone mineral density is defined as bone mineral density per unit area (g / cm ２ ), bone mineral density per unit volume (g / cm ３ The bone mineral density may be expressed by at least one of the following: YAM (%), T-score, and Z-score. YAM (%) is an abbreviation for "Young Adult Mean" and is sometimes called the young adult average percent. The bone mineral density may be an index determined by guidelines or a unique index. For example, the bone mineral density may be a value used in osteoporosis guidelines (for example, but not limited to, the 2015 edition of the Prevention and Treatment Guidelines of the Japan Osteoporosis Society).

[0037] The estimation unit 23 may estimate, as the bone condition information, what kind of bone condition information will be obtained or how the bone condition information will change if the subject undergoes treatment. More specifically, when the bone condition information is the subject's future bone density, the estimation unit 23 may estimate, as the bone condition information, the bone density or change in bone density when the subject undergoes treatment. Alternatively, the estimation unit 23 may estimate the details of the treatment in addition to the bone condition information. The information processing device 1 may then display the estimated details of the treatment on the display unit. The treatment may include, for example, taking a drug, taking a nutritional supplement, or improving lifestyle habits over a predetermined period of time. The drug may include information about a drug that has at least one of an effect on bone formation and an effect on bone resorption. Drugs that have an effect on bone formation include, but are not limited to, active vitamin D3 preparations (e.g., calcitriol, eldecalcitol, or alfacalcidol), teriparatide acetate, and teriparatide (genetically recombinant). Furthermore, examples of drugs that have an effect on bone resorption include, but are not limited to, calcitonin preparations, bisphosphonate preparations, and anti-RANKL monoclonal antibodies.

[0038] The learning model 33 is a computational model used by the estimation unit 23 when performing computations based on input data. The learning unit 25 performs machine learning on an untrained neural network using teacher data 32, which will be described later, to generate the learning model 33. Here, the learning model 33 may also be applied to non-human animals. In this case, the "predetermined person" in the teacher data 32 may be of the same biological species as the "subject." In other words, the information processing device 1 according to the present disclosure is also capable of estimating the onset and / or progression of diseases in non-human animals. Specific examples of the teacher data 32, the configuration of the neural network, and the learning process will be described later.

[0039] <Output unit 24> The output unit 24 transmits the information estimated by the estimation unit 23 to the terminal device 30. The information processing device 1 may be configured to include a display unit (not shown). In that case, the output unit 24 causes the display unit to display the information estimated by the estimation unit 23.

[0040] <Learning Unit 25> The learning unit 25 controls the learning process for an untrained neural network. The learning unit 25 executes the learning process for the untrained neural network to create a trained neural network that functions as the estimation unit 23. Teacher data 32 (described later) is used for this learning. Specific examples of the learning performed by the learning unit 25 will be described later.

[0041] (Configuration of Estimation Unit 23) The configuration of the estimation unit 23 will be described below with reference to Fig. 3. The configuration shown in Fig. 3 is an example, and the configuration of the estimation unit 23 is not limited to this.

[0042] 3 , the estimation unit 23 performs calculations based on the learning model 33 on input data input to the input layer 210, and outputs output data from the output layer 230. The output data is future bone condition information of the subject. The future bone condition information may be, for example, the probability of fracture in the subject's future, the subject's future bone mineral density, or the subject's future bone quality.

[0043] The estimation unit 23 in FIG. 3 includes a neural network 200 having an input layer 210 and an output layer 230. FIG. 3 illustrates a case where the neural network 200 is a convolutional neural network (CNN). As shown in FIG. 3, the neural network 200 includes, for example, an input layer 210, a hidden layer 220, and an output layer 230. The hidden layer 220 is also called an intermediate layer. The hidden layer 220 includes, for example, multiple convolutional layers 240, multiple pooling layers 250, and a fully connected layer 260. In the neural network 200, the fully connected layer 260 is located before the output layer 230. In the neural network 200, the convolutional layers 240 and the pooling layers 250 are alternately arranged between the input layer 210 and the fully connected layer 260. The configuration of the neural network 200 is not limited to the example in FIG. 7. For example, the neural network 200 may include one convolutional layer 240 and one pooling layer 250 between the input layer 210 and the fully connected layer 260.

[0044] The neural network 200 is not limited to a CNN and may be an LSTM. The neural network 200 may be, for example, a ConvLSTM network that combines a CNN and an LSTM. In this case, the input layer 210 can extract features related to changes in the input data. The output layer 230 can calculate new features based on the features extracted by the input layer 210, the time changes in the input data, and the initial values. The input layer 210 and the output layer 230 each have multiple LSTM layers. Each of the input layer 210 and the output layer 230 may have three or more LSTM layers.

[0045] (Learning Process by Learning Unit 25) The learning process for generating the learning model 33 will be described below with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the flow of the learning process by the learning unit 25.

[0046] The learning unit 25 acquires training data 32 from the storage unit 10 (step S1). The training data 32 includes multiple pieces of information, including bone condition information regarding the future bone condition of a specific person, and includes explanatory variables and response variables to be input to the input layer 210. Here, the explanatory variables are medical images depicting the specific person's bones and characteristic information about the specific person at a time different from the time the medical images were captured, and the response variables are bone condition information for each person. The training data 32 may also include details of the specific person's treatment. The treatment may include, for example, taking medication, taking nutritional supplements, or improving lifestyle habits over a specific period of time. The medication may include information about a medication that has at least one of an effect on bone formation and an effect on bone resorption. Examples of medications that have an effect on bone formation include, but are not limited to, active vitamin D3 preparations (e.g., calcitriol, eldecalcitol, or alfacalcidol), teriparatide acetate, and teriparatide (genetically recombinant). Furthermore, examples of drugs that have an effect on bone resorption include, but are not limited to, calcitonin preparations, bisphosphonate preparations, and anti-RANKL monoclonal antibodies.

[0047] Next, the learning unit 25 inputs into the input layer 210 a medical image showing the bones of a person (referred to as person A) and characteristic information of person A at a time different from the time the medical image was taken, as information including future bone condition information (step S2).

[0048] Next, the learning unit 25 acquires output data related to the bone condition information of person A from the output layer 230 (step S3). This output data contains the same content as the objective variable of the training data 32. In Fig. 4, the order of steps S2 and S3 may be reversed. Alternatively, in Fig. 4, steps S2 and S3 may be executed simultaneously.

[0049] Next, the learning unit 25 acquires the objective variable for person A included in the teacher data 32. Then, the learning unit 25 compares the output data acquired in step S3 with the objective variable for person A to calculate an error (step S4), and adjusts the learning model 33 during learning so as to reduce the error (step S5).

[0050] Any known method can be applied to adjust the learning model during learning. For example, the backpropagation method may be adopted as a method for adjusting the learning model. The adjusted learning model becomes a new learning model, and the estimation unit 23 uses the new learning model in subsequent calculations. During the learning model adjustment stage, parameters used by the estimation unit 23 (e.g., filter coefficients, weighting coefficients, etc.) may be adjusted.

[0051] If the error is not within the predetermined range and explanatory variables for all people included in the teacher data 32 have not been input (NO in step S6), the learning unit 25 returns to step S2 and repeats the learning process. If the error is within the predetermined range and explanatory variables for all people included in the teacher data 32 have been input (YES in step S6), the learning unit 25 ends the learning process.

[0052] When the above-described learning process is adopted, the estimation unit 23 can estimate future bone condition information of the subject from medical information including first medical information having a medical image showing the subject's bones and second medical information having characteristic information of the subject at a time different from the time the medical image was taken.

[0053] In one aspect of the present disclosure, the learning model 33 may be a learning model trained using training data including medical information of a specific person acquired at multiple different time points. For example, the learning model 33 may be a learning model trained using training data including medical images of the specific person acquired at multiple different time points as explanatory variables and bone density of the specific person acquired at the multiple different time points as a target variable. In this way, by inputting medical information of the subject acquired at at least two different time points into the learning model 33, it is possible to improve estimation accuracy.

[0054] (Processing Performed by Information Processing Device 1) The flow of processing performed by the information processing device 1 will be described below with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of processing performed by the information processing device 1.

[0055] First, the acquisition unit 21 acquires medical information including first medical information having a medical image showing the subject's bones and second medical information having characteristic information of the subject at a time different from the time the medical image was taken (step S11: acquisition step).

[0056] Next, the estimation unit 23 inputs the acquired medical information into the learning model 33 to estimate the subject's future bone condition information (step S12: estimation step).

[0057] The output unit 24 outputs each piece of information including the bone condition information estimated in step S12 (step S13: output step).

[0058] According to this configuration, the information processing device 1 and the information processing system 100a estimate future bone condition information of a subject from medical information including first medical information having a medical image showing the subject's bones and second medical information having characteristic information of the subject at a time point different from the time point at which the medical image was captured. This improves the estimation accuracy compared to when bone condition information is estimated using input information including medical information acquired at a single time point.

[0059] The estimation unit 23 may estimate the bone condition information of the subject after a time shorter than the time interval between any two of the time points when the first medical information was acquired and the at least one time point when the second medical information was acquired. This enables the bone condition information to be estimated more accurately based on information about changes in the subject's condition over a time interval longer than the time interval from the current time point to the estimation time point.

[0060] The estimation unit 23 may estimate the future bone condition information of the subject by inputting input information including medical information at three or more points in time, i.e., first medical information having medical images showing the bones of the subject, and second medical information having characteristic information of the subject at two or more points in time different from the points in time at which the medical images were captured, into the learning model 33. This allows the degree of acceleration of the change in the bone condition over time up to the present time to be known, and enables more accurate estimation of the bone condition information at the estimation point in time.

[0061] The estimation unit 23 may estimate whether the subject has undergone menopause based on the subject's attribute information and the degree of acceleration of time-dependent changes in bone condition at three or more points in time. In this case, the estimation unit 23 may use the estimated menopause information as the menopause information of the subject's attribute information.

[0062] When the second medical information is a medical image, the area depicted in the medical image of the second medical information may be different from the area depicted in the medical image of the first medical information. For example, the area depicted in the image of the first medical information may be the chest, and the medical image of the second medical information may be a dental medical image. This allows for flexible use of images taken during health checkups or dental treatment, improving versatility.

[0063] The estimation unit 23 may estimate the subject's bone condition information at a time (hereinafter referred to as the estimation time) that is longer than the time interval (hereinafter referred to as the first time interval) between the time when the first medical information was acquired and the earliest and latest of at least one time points when the second medical information was acquired. As described above, the information processing system 100a of the present disclosure estimates the subject's future bone condition information using the first medical information and the second medical information acquired at different time points. This allows for highly accurate estimation of the subject's future bone condition information within the first time interval from the current time point. Therefore, when estimating the subject's bone condition information after the first time interval from the current time point, the estimation can be performed using the subject's bone condition information estimated with high accuracy for the first time interval from the current time point. As a result, the subject's bone condition information can be estimated more accurately than when estimating the bone condition information using medical information from a single time point.

[0064] When the second medical information acquired by the acquisition unit 21 includes medical information at two or more time points, and any of the data in the second medical information and the first medical information at the two or more time points is anomalous data, the estimation unit 23 may input data excluding the anomalous data into the learning model 33. The anomalous data may be, for example, a medically impossible test value resulting from a test error or the like. This configuration reduces the possibility of inputting incorrect information into the learning model 33, thereby improving estimation accuracy. In this case, the estimation unit 23 may, for example, specify the lower and upper limits of medically possible values ​​for the test value as thresholds, and input data excluding data outside the range of the thresholds into the learning model 33.

[0065] When the second medical information acquired by the acquisition unit 21 includes medical information at two or more points in time, and any data of the second medical information at the two or more points in time and the first medical information is idiosyncratic data, the estimation unit 23 may replace the idiosyncratic data with data that approximates the other data and input the idiosyncratic data to the learning model 33. This configuration can reduce the possibility of erroneous information being input to the learning model 33, thereby improving estimation accuracy.

[0066] It has been known that bone mineral density significantly decreases with menopause. Therefore, if the subject is a woman who has not yet undergone menopause, the estimation unit 23 may use the menopause information included in the attribute information to output at least one of a first estimation result assuming that the subject has undergone menopause and a second estimation result assuming that the subject has not yet undergone menopause. This allows for the output of more accurate estimation results that take into account the presence or absence of menopause. If the subject is a woman who has not yet undergone menopause, the estimation unit 23 may simultaneously output the first estimation result assuming that the subject has undergone menopause and the second estimation result assuming that the subject has not yet undergone menopause.

[0067] It is also known that whether or not a subject has given birth has a significant effect on bone mineral density. Therefore, when the subject is a woman who has not given birth, the estimation unit 23 may use the birth information included in the attribute information to output at least one of a first estimation result assuming that the subject has given birth and a second estimation result assuming that the subject has not given birth. This allows for the output of a more accurate estimation result that takes into account the presence or absence of childbirth experience.

[0068] [Embodiment 2] (Configuration of information processing system 100b) The information processing device 1 may be communicably connected to a LAN disposed in each of a plurality of medical facilities 8 via a communication network 9, rather than being a computer installed in a predetermined medical facility 8. Fig. 6 is a diagram showing an example configuration of an information processing system 100b according to another aspect of the present disclosure. For convenience of explanation, components having the same functions as those described in the above embodiment are denoted by the same reference numerals, and their description will not be repeated.

[0069] In addition to one or more terminal devices 30a, a medical image management device 3a, an attribute information management device 4a, a test value management device 5a, a diagnostic result management device 6a, and a bone information management device 7a may be communicably connected to the LAN in medical facility 8a. Furthermore, in addition to terminal device 30b, a medical image management device 3b, an attribute information management device 4b, a test value management device 5b, a diagnostic result management device 6b, and a bone information management device 7b may be communicably connected to the LAN in medical facility 8b. In the present disclosure, when there is no particular need to distinguish between medical facilities 8a and 8b, they will be referred to as "medical facility 8." Furthermore, when no particular distinction is made between the terminal devices 30a and 7b, the medical image management devices 3a and 3b, the attribute information management devices 4a and 4b, the test value management devices 5a and 5b, the diagnostic result management devices 6a and 6b, and the bone information management devices 7a and 7b, they will be referred to as the "terminal device 30," the "medical image management device 3," the "attribute information management device 4," the "test value management device 5," the "diagnostic result management device 6," and the "bone information management device 7," respectively.

[0070] 6 shows an example in which the LANs of medical facilities 8a and 8b are connected to a communication network 9. The information processing device 1 is only required to be communicably connected to the medical image management device 3, attribute information management device 4, test value management device 5, diagnosis result management device 6, and bone information management device 7 in each medical facility via the communication network 9, and is not limited to the configuration shown in FIG. 6. For example, the information processing device 1 may be installed in the medical facility 8a or the medical facility 8b.

[0071] In the information processing system 100b having such a configuration, the information processing device 1 can acquire medical images, attribute information, test values, and diagnosis results of a subject Pa who has been examined at a medical facility 8a from the medical image management device 3a, attribute information management device 4a, test value management device 5a, and diagnosis result management device 6a of the medical facility 8a. The information processing device 1 then transmits the estimated results of future bone condition information of the subject Pa to a terminal device 30a installed in the medical facility 8a. Similarly, the information processing device 1 transmits future bone condition information of the subject Pb to a terminal device 30b.

[0072] [Embodiment 3] Like the information processing device 1, the device may be configured to output estimated information on bone condition information related to the bone condition of a subject, estimated using the learning model 33 based on input information including third medical information different from the first medical information and the second medical information. In this case, the third medical information on the subject includes at least one of a first medical image showing the subject's trabecular bones and a first virtual image representing the color shading in the first medical image for each predetermined area (pixel). The learning model 33 may be trained using training data including at least one of a second medical image showing the trabecular bones of a specific person and information including trabecular bone information related to the trabecular bone condition of the specific person, and a second virtual image representing the color shading in the second medical image for each predetermined area. The first medical image and the second medical image may be images similar to the medical images described above.

[0073] An information processing system 100a having such a configuration will be described below. For convenience of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0074] (Configuration of Information Processing System 100a) As shown in FIG. 1 , the information processing system 100a according to this embodiment includes an information processing device 1 and one or more terminal devices 30 communicably connected to the information processing device 1. The information processing device 1 estimates bone condition information related to the bone condition of a subject from medical information including at least one of a medical image showing the subject's bone trabeculae and a first virtual image that represents color shading in the medical image for each predetermined area. The information processing device 1 may estimate the subject's future bone condition information or the subject's current bone condition information. The information processing device 1 is a computer that transmits estimated information of the subject's bone condition information to the terminal device 30.

[0075] The medical image management device 3 generates a virtual image that represents the color shading in a medical image for each specified area. The virtual image is generated from the shading of cancellous bone and / or cortical bone and non-bone parts of bone image information that includes bone parts in the medical image. More specifically, the virtual image can be generated by, for example, converting the bone shading from the bone image information for each specified area into a numerical value and representing the numerical value as a color shading.

[0076] The predetermined area may be, for example, one pixel of the image or multiple pixels of the image. The predetermined area may take into account at least one of the following: trabeculae, trabecular number (e.g., the number of trabeculae per unit length), trabecular spacing (e.g., the spatial distance between trabeculae), trabecular width (e.g., the width or thickness of trabeculae), trabecular orientation (e.g., the degree of trabecular alignment within trabeculae), and bone connectivity density (e.g., the number of paths connecting the ends of trabeculae per unit area). The numerical value may be, for example, at least one of bone mass (e.g., a numerical value in grams), bone density, and cancellous bone structure index. The color shading may be, for example, two-tone white and black, grayscale (e.g., two-tone white and black plus 254 shades of gray between white and black), or a heat map using multiple colors. The information processing device 1 may acquire, from the medical image management device 3, medical images showing the subject's trabecular bones, and virtual images generated using the medical images.

[0077] 1 shows an example of a configuration including a medical image management device 3, an attribute information management device 4, an examination value management device 5, a diagnosis result management device 6, and a bone information management device 7, but is not limited to this. For example, a configuration may be provided that includes a management device having the functions of any two or more or all of these devices.

[0078] (Configuration of Information Processing Apparatus 1) Next, the configuration of the information processing apparatus 1 applied to the information processing system 100a according to this embodiment will be described with reference to FIG.

[0079] <Acquisition Unit 21> The acquisition unit 21 acquires input information including medical information of the subject. The input information is data input to the estimation unit 23. The input information includes at least one of a medical image showing the subject's trabecular bones and a virtual image representing the color shading in the medical image for each specified area. In the following description, the medical image showing the subject's trabecular bones may be referred to as the first medical image, and the virtual image representing the color shading in the medical image for each specified area may be referred to as the first virtual image. The acquisition unit 21 may acquire the first medical image and / or the first virtual image from the medical image management device 3. The acquisition unit 21 may acquire the first virtual image generated by a virtual image generation device other than the medical image management device 3 using the first medical image from the virtual image generation device. In addition to the first medical image and the first virtual image, the acquisition unit 21 may acquire, as input information, attribute information of the subject from the attribute information management device 4, may acquire test values ​​of the subject from the test value management device 5, may acquire diagnostic results of the subject from the diagnostic result management device 6, or may acquire bone condition information of the subject from the bone information management device 7.

[0080] <Estimation Unit 23> The estimation unit 23 estimates the subject's bone condition information by inputting the input information acquired by the acquisition unit 21 into the learning model 33. In this embodiment, an example will be described in which the estimation unit 23 estimates the subject's future bone condition information. However, the subject's past bone condition information may also be estimated. Here, the learning model 33 is trained in advance using training data 32. The training data 32 is data including medical information of a specific person. The specific person may be a patient suffering from a bone-related disease or a person without a disease. The medical information included in the training data 32 includes at least one of: a second medical image showing the specific person's trabecular bones; information including trabecular bone information regarding the state of the specific person's trabecular bones; and a second virtual image representing the color shading in the second medical image for each specific area. In the following description, the medical image showing the specific person's trabecular bones may be referred to as the second medical image, and the virtual image representing the color shading in the second medical image for each specific area may be referred to as the second virtual image.

[0081] The estimation unit 23 may estimate the subject's future bone condition information by inputting input information including at least one of a first medical image and a first virtual image into the learning model 33. The estimation unit 23 may (1) input the medical information including the first medical image into the learning model 33 trained using training data including at least information including a second medical image depicting the trabeculae of a specific person and trabecular information related to the state of the trabeculae of the specific person, or (2) input medical information including the first virtual image into the learning model 33 trained using training data including at least a second virtual image representing color shading in the second medical image for each specific area, thereby estimating the subject's future bone condition information. By inputting the input information into the learning model 33, the estimation unit 23 may estimate at least one of the following bone condition information: cancellous bone structure index, fracture probability, osteoporosis possibility, drug efficacy, incident occurrence, bone mineral density, trabecular number, trabecular spacing, trabecular width, trabecular orientation, and bone connectivity density. Alternatively, bone strength analyzed from the state of cortical bone and / or cancellous bone may be used as bone condition information.

[0082] (Configuration of Estimation Unit 23) As shown in FIG. 3 , the estimation unit 23 performs calculations based on the learning model 33 on input data input to the input layer 210, and outputs output data from the output layer 230. The output data is future bone condition information of the subject. The future bone condition information may be, for example, at least one of a cancellous bone structure index, a fracture occurrence probability, a possibility of osteoporosis, a drug effect, an incident occurrence, bone mineral density, trabecular number, trabecular spacing, trabecular width, trabecular orientation, and bone connectivity density.

[0083] (Learning Process by Learning Unit 25) The learning process for generating the learning model 33 will be described below with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of the learning process by the learning unit 25.

[0084] The learning unit 25 acquires training data 32 from the storage unit 10 (step S21). The training data 32 includes information including a second medical image showing the trabecular bones of a specific person and trabecular bone information related to the state of the trabecular bones of the specific person, at least one of the second virtual images, and data related to the state of the trabecular bones of the specific person a predetermined time after the second medical image is captured, and also includes explanatory variables and objective variables to be input to the input layer 210. Here, the explanatory variables are the information including the second medical image showing the trabecular bones of the specific person and trabecular bone information related to the state of the trabecular bones of the specific person, and at least one of the second virtual images, and the objective variable is data related to the state of the trabecular bones of the specific person a predetermined time after the second medical image is captured.

[0085] Next, the learning unit 25 inputs information including a second medical image showing the trabeculae of a certain person (referred to as person A) and trabecular bone information regarding the state of the trabeculae of the specified person, as well as information including at least one of the second virtual images, into the input layer 210 (step S22).

[0086] Next, the learning unit 25 acquires output data related to the bone condition information of person A from the output layer 230 (step S23). This output data contains the same content as the objective variable of the training data 32. In FIG. 7, the order of steps S22 and S23 may be reversed. Alternatively, in FIG. 7, steps S22 and S23 may be executed simultaneously.

[0087] Next, the learning unit 25 acquires the objective variable for person A included in the teacher data 32. Then, the learning unit 25 compares the output data acquired in step S23 with the objective variable for person A to calculate an error (step S24), and adjusts the learning model being learned so as to reduce the error (step S25).

[0088] If the error is not within the predetermined range and explanatory variables for all people included in the teacher data 32 have not been input (NO in step S26), the learning unit 25 returns to step S2 and repeats the learning process. If the error is within the predetermined range and explanatory variables for all people included in the teacher data 32 have been input (YES in step S26), the learning unit 25 ends the learning process.

[0089] When the above-described learning process is employed, the estimation unit 23 can estimate the subject's future bone condition information from medical information including at least one of a first medical image showing the subject's bone trabeculae and a first virtual image that represents the color shading in the first medical image for each specified area. When the training data 32 includes multiple first medical images and / or first virtual images of the same person, the learning model 33 may be constructed based on temporal changes in the feature quantities of characteristic regions. This allows the subject's future bone condition information to be estimated taking into account temporal changes in the state of the trabeculae, thereby improving the accuracy of the estimation.

[0090] (Processing Performed by Information Processing Device 1) The flow of processing performed by the information processing device 1 will be described below with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the flow of processing performed by the information processing device 1.

[0091] First, the acquisition unit 21 acquires medical information including at least one of a first medical image showing the subject's bone trabeculae and a first virtual image that represents the color shading in the first medical image for each specified area (step S31: acquisition step).

[0092] Next, the estimation unit 23 inputs the acquired medical information into the learning model 33 to estimate the subject's future bone condition information (step S32: estimation step).

[0093] The output unit 24 outputs each piece of information including the bone condition information estimated in step S32 (step S33: output step).

[0094] According to this configuration, the information processing device 1 and the information processing system 100a estimate future bone condition information of a subject from medical information including at least one of a first medical image showing the subject's bone trabeculae and a first virtual image representing the color shading of the first medical image for each predetermined area. Here, the first medical image shows the bone trabeculae, which are the internal structure of the bone. Therefore, the estimation result output from the learning model 33 by the estimation unit 23 using the first medical image or the first virtual image generated using the first medical image as input information reflects the internal structure of the bone, resulting in high estimation accuracy. In other words, the information processing device 1 and the information processing system 100a can estimate bone condition information regarding the subject's future bone condition with high accuracy.

[0095] In the above description, the information processing device 1 estimates bone condition information related to the subject's future bone condition, but the information processing device 1 of the present disclosure is not limited to this. The information processing device 1 may also estimate bone condition information related to the subject's current bone condition. In this case, the learning unit 25 may generate the learning model 33 using data related to the state of the trabecular bone of the specified person at the time the second medical image was captured as the objective variable. As a result, the estimation unit 23 can estimate bone condition information related to the subject's current bone condition by inputting medical information including at least one of the first medical image and the first virtual image into the learning model 33.

[0096] Here, it is considered that bone condition changes significantly due to menopause. Therefore, when estimating bone condition information regarding the subject's future bone condition, and the subject is a female who has not yet undergone menopause, the estimation unit 23 may output at least one of a first estimation result assuming that the subject has undergone menopause and a second estimation result assuming that the subject will not undergo menopause, if the subject is a female who has not yet undergone menopause. This makes it possible to output estimation results with higher accuracy that take into account the presence or absence of menopause. When the subject is a female who has not yet undergone menopause, the estimation unit 23 may simultaneously output the first estimation result assuming that the subject has undergone menopause and the second estimation result assuming that the subject will not undergo menopause.

[0097] [Special Notes] It is not essential that all of the functional blocks included in the control unit 2 of the information processing device 1 are included in the information processing device 1. For example, in the information processing system 100a, the terminal device 30 may have the function of the acquisition unit 21, and the information processing device 1 may receive input information from the terminal device 30. In the information processing system 100b, the information processing device 1 may also receive input information from the terminal device 30.

[0098] For example, in the information processing systems 100a and 100b, a learning model 33 in which the function of the learning unit 25 has been subjected to learning processing by a computer other than the information processing device 1 may be installed in the information processing device 1. For example, in the information processing systems 100a and 100b, the function of the learning unit 25 may be provided by a computer or terminal device 30 other than the information processing device 1. Furthermore, in the information processing systems 100a and 100b, the function of the estimation unit 23 may be provided by a computer or terminal device 30 other than the information processing device 1.

[0099] [Example of Software Implementation] The control block (particularly the control unit 2) of the information processing device 1 may be implemented by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be implemented by software.

[0100] In the latter case, the information processing device 1 includes a computer that executes instructions from a program, which is software that realizes each function. This computer includes, for example, one or more processors and a computer-readable recording medium storing the program. The object of the present disclosure is achieved when the processor in the computer reads and executes the program from the recording medium. The processor may be, for example, a central processing unit (CPU). The recording medium may be a "non-transitory tangible medium," such as a read-only memory (ROM), tape, disk, card, semiconductor memory, or programmable logic circuit. The information processing device 1 may also include a random access memory (RAM) for expanding the program. The program may also be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). One aspect of the present disclosure may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0101] The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to the above-described embodiments. In other words, the invention according to the present disclosure can be modified in various ways within the scope of the present disclosure, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the invention according to the present disclosure. In other words, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. It should also be noted that these modifications or corrections are included in the scope of the present disclosure.

[0102] [Summary] The information processing system according to aspect 1 of the present disclosure includes an acquisition unit that acquires medical information including first medical information having a medical image showing the bones of a subject and second medical information having characteristic information of the subject at a time point different from the time point when the medical image was captured, and an estimation unit that estimates bone condition information regarding the future or past bone condition of the subject by inputting the medical information into a learning model that has been trained using training data including a plurality of pieces of information having medical images showing the bones of a specific person and the characteristic information of the specific person at a time point different from the time point when the medical image was captured.

[0103] The information processing system according to aspect 2 of the present disclosure may include, in the above-described aspect 1, at least one of the subject's fractures, bone density, bone quality, cancellous bone structure index, fracture occurrence probability, osteoporosis possibility, drug efficacy, incident occurrence, trabecular number, trabecular spacing, trabecular width, trabecular orientation, and bone connectivity density.

[0104] In the information processing system according to aspect 3 of the present disclosure, in the above-mentioned aspect 1 or 2, the estimation unit may estimate the bone condition information of the subject after a time shorter than the time interval between any two of the time points at which the first medical information was acquired and at least one time point at which the second medical information was acquired.

[0105] In the information processing system according to aspect 4 of the present disclosure, in the above-mentioned aspect 1 or 2, the estimation unit may estimate bone condition information after a time longer than the time interval between the earliest and latest of the time points at which the first medical information was acquired and at least one of the time points at which the second medical information was acquired.

[0106] In an information processing system according to aspect 5 of the present disclosure, in any of aspects 1 to 4 above, the learning model may be a learning model trained using training data including medical information of the specified person obtained at multiple different points in time.

[0107] In an information processing system according to aspect 6 of the present disclosure, in any one of aspects 1 to 5 above, the estimation unit may input data excluding the idiosyncratic data into the learning model when the second medical information acquired by the acquisition unit includes medical information at two or more points in time, and when any of the data in the second medical information and the first medical information at the two or more points in time is idiosyncratic data.

[0108] In the information processing system according to aspect 7 of the present disclosure, in any one of aspects 1 to 6 above, the estimation unit may, when the second medical information acquired by the acquisition unit includes medical information at two or more points in time, and any of the data in the second medical information and the first medical information at the two or more points in time is idiosyncratic data, replace the idiosyncratic data with data that approximates other data and input the idiosyncratic data into the learning model.

[0109] In the information processing system according to aspect 8 of the present disclosure, in any one of aspects 1 to 7 above, the estimation unit may input data contained in the medical information acquired by the acquisition unit, excluding data that falls outside a predetermined threshold range, into the learning model.

[0110] In an information processing system according to aspect 9 of the present disclosure, in any of aspects 1 to 8 above, the estimation unit may output at least one of a first estimation result assuming that the subject has undergone menopause and a second estimation result assuming that the subject has not undergone menopause, if the subject is a woman who has not yet undergone menopause.

[0111] In an information processing system according to aspect 10 of the present disclosure, in any of aspects 1 to 9 above, the feature information may include at least one of medical images, past bone condition information, attribute information, test values, and diagnostic results.

[0112] In the information processing system according to aspect 11 of the present disclosure, in the above aspect 10, the past bone condition information may include at least one of bone mineral density and a cancellous bone structure index.

[0113] In the information processing system according to aspect 12 of the present disclosure, in the above-mentioned aspects 10 or 11, the attribute information may include at least one of age, sex, height, weight, lifestyle habits, medication information, occupational information, blood test information, urine test information, saliva test information, medical history, medical history of the subject's family, genetic information, menopausal information, and menopausal stage estimation estimated based on hormone information.

[0114] In the information processing system according to aspect 13 of the present disclosure, in any one of aspects 10 to 12, the estimation unit may input at least the attribute information as the medical information to the learning model.

[0115] In the information processing system according to aspect 14 of the present disclosure, in any one of aspects 10 to 13 above, the test values ​​may include at least one of bone density, Kellgren-Lawrence (KL) classification, bone morphology angle, muscle mass, Mini Mental State Examination (MMSE), blood test values, liver function markers, uric acid levels, and malignant tumor markers.

[0116] In the information processing system according to aspect 15 of the present disclosure, in any one of aspects 1 to 14 above, the first medical information may include the same information elements as the characteristic information contained in the second medical information.

[0117] In an information processing system according to aspect 16 of the present disclosure, in any one of aspects 1 to 15 above, the medical image may be at least one of an X-ray image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, an ultrasound image, and a PET (Positron Emission Tomography) image.

[0118] In an information processing system according to aspect 17 of the present disclosure, in any one of aspects 1 to 16 above, the estimation unit may estimate bone condition information regarding the subject's bone condition at multiple points in the future and past.

[0119] In the information processing system of aspect 18 of the present disclosure, in the above-mentioned aspect 1, when the first medical information including a first medical image showing the trabecular bone of the subject is input to the learning model, the learning model may be trained using training data including a second medical image showing the trabecular bone of a specified person and trabecular bone information regarding the state of the trabecular bone of the specified person.

[0120] In the information processing system of aspect 19 of the present disclosure, in the above-mentioned aspect 1, when the first medical information including a first virtual image that represents the color shading for each specified area in a first medical image that shows the bone trabeculae of the subject is input to the learning model, the learning model may be trained using training data that includes at least a second virtual image that represents the color shading for each specified area in a second medical image that shows the bone trabeculae of a specified person.

[0121] In the information processing system according to aspect 20 of the present disclosure, in the above-mentioned aspect 18, the trabecular bone information is data regarding the state of the trabecular bone of the specified person at a time different from the time when the second medical image was captured, and the estimation unit may estimate future bone condition information of the subject.

[0122] In the information processing system of aspect 21 of the present disclosure, in aspect 18 or 19 above, the trabecular bone information is data regarding the trabecular bone of the specified person at the time the second medical image was captured, and the estimation unit may estimate the current bone condition information of the subject.

[0123] In an information processing system according to aspect 22 of the present disclosure, in any one of aspects 18 to 20 above, the medical information may include at least the first virtual image.

[0124] A terminal device according to aspect 23 of the present disclosure presents on a display unit bone condition information regarding the future or past bone condition of a subject, estimated by inputting medical information including first medical information having a medical image showing the bones of a subject and second medical information having characteristic information of the subject at a time different from the time the medical image was taken into a learning model trained using training data including a plurality of pieces of information having medical images showing the bones of a specific person and characteristic information of the specific person at a time different from the time the medical image was taken.

[0125] A control method for an information processing system according to aspect 24 of the present disclosure includes an acquisition step of acquiring medical information including first medical information having a medical image showing the bones of a subject and second medical information having characteristic information of the subject at a time point different from the time point when the medical image was taken, and an estimation step of estimating bone condition information regarding the future bone condition of the subject by inputting the medical information into a learning model trained using training data including a plurality of pieces of information having medical images showing the bones of a specific person and characteristic information of the specific person at a time point different from the time point when the medical image was taken.

[0126] A control method for an information processing system according to aspect 25 of the present disclosure is a control program for causing a computer to function as an information processing system described in any one of aspects 1 to 22 above, and is a control program for causing a computer to function as the acquisition unit and the estimation unit.

[0127] A recording medium according to aspect 26 of the present disclosure is a computer-readable recording medium on which the control program according to aspect 25 above is recorded.

[0128] REFERENCE SIGNS LIST 1 Information processing device 21 Acquisition unit 23 Estimation unit 24 Output unit 32 Teacher data 33 Learning model 100a, 100b Information processing system S11, S31 Acquisition step S12, S32 Estimation step S13, S33 Output step

Claims

1. an acquisition unit that acquires medical information including first medical information having a medical image showing the bones of a subject and second medical information having characteristic information of the subject at a time point different from the time point at which the medical image was captured; An information processing system comprising: an estimation unit that estimates bone condition information regarding the future or past bone condition of a subject by inputting the medical information into a learning model that has been trained using training data that includes a medical image showing the bones of a specific person and a plurality of pieces of information having characteristic information of the specific person at a time different from the time the medical image was taken.

2. The information processing system of claim 1, wherein the bone status information includes at least one of the subject's fractures, bone density, bone quality, cancellous bone structure index, fracture occurrence probability, osteoporosis possibility, drug efficacy, incident occurrence, trabecular number, trabecular spacing, trabecular width, trabecular orientation, and bone connectivity density.

3. 2. The information processing system according to claim 1, wherein the estimation unit estimates the bone condition information of the subject after a time shorter than the time interval between any two points in time of acquiring the first medical information and at least one point in time of acquiring the second medical information.

4. 2. The information processing system according to claim 1, wherein the estimation unit estimates bone condition information at a time longer than the time interval between the earliest and latest of the time points at which the first medical information was acquired and at least one of the time points at which the second medical information was acquired.

5. The information processing system according to claim 1 , wherein the learning model is a learning model trained using training data including medical information of the specified person acquired at a plurality of different points in time.

6. 2. The information processing system of claim 1, wherein the second medical information acquired by the acquisition unit includes medical information at two or more points in time, and when any of the data in the second medical information and the first medical information at the two or more points in time is idiosyncratic data, the estimation unit inputs data excluding the idiosyncratic data into the learning model.

7. 2. The information processing system of claim 1, wherein the second medical information acquired by the acquisition unit includes medical information at two or more points in time, and when any of the data in the second medical information and the first medical information at the two or more points in time is idiosyncratic data, the estimation unit replaces the idiosyncratic data with data that approximates other data and inputs the idiosyncratic data into the learning model.

8. The information processing system according to claim 1 , wherein the estimation unit inputs data included in the medical information acquired by the acquisition unit, excluding data outside a predetermined threshold range, into the learning model.

9. The information processing system of claim 1, wherein the estimation unit outputs at least one of a first estimation result assuming that the subject has undergone menopause and a second estimation result assuming that the subject has not undergone menopause when the subject is a woman who has not yet undergone menopause.

10. The information processing system according to claim 1 , wherein the characteristic information includes at least one of a medical image, past bone condition information, attribute information, test values, and a diagnosis result.

11. The information processing system according to claim 10 , wherein the past bone condition information includes at least one of bone mineral density and a cancellous bone structure index.

12. The information processing system of claim 10, wherein the attribute information includes at least one of age, sex, height, weight, lifestyle habits, medication information, occupational information, blood test information, urine test information, saliva test information, medical history, medical history of the subject's family, genetic information, menopausal information, and menopausal estimation estimated based on hormone information.

13. The information processing system according to claim 10 , wherein the estimation unit inputs at least the attribute information as the medical information into the learning model.

14. 11. The information processing system according to claim 10, wherein the test values ​​include at least one of bone density, Kellgren-Lawrence (KL) classification, bone morphology angle, muscle mass, Mini Mental State Examination (MMSE), blood test values, liver function markers, uric acid levels, and malignant tumor markers.

15. The information processing system according to claim 1 , wherein the first medical information includes the same information elements as the characteristic information included in the second medical information.

16. 2. The information processing system according to claim 1, wherein the medical image is at least one of an X-ray image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, an ultrasound image, and a PET (Positron Emission Tomography) image.

17. The information processing system according to claim 1 , wherein the estimation unit estimates bone condition information relating to bone conditions of the subject at multiple points in time in the future and the past.

18. When the first medical information including a first medical image showing the trabecular bone of the subject is input to the learning model, the learning model is trained using training data including a second medical image showing the trabecular bone of a specific person and trabecular bone information related to the state of the trabecular bone of the specific person. The information processing system according to claim 1 .

19. When the first medical information including a first virtual image that expresses color shading for each predetermined area in a first medical image that shows the subject's trabecular bone is input to the learning model, the learning model is trained using training data that includes at least a second virtual image that expresses color shading for each predetermined area in a second medical image that shows the trabecular bone of a predetermined person. The information processing system according to claim 1 .

20. the trabecular bone information is data relating to a state of the trabecular bone of the specified person at a time point different from a time point at which the second medical image was captured; The information processing system according to claim 18 , wherein the estimation unit estimates future bone condition information of the subject.

21. the trabecular bone information is data on the trabecular bone of the specified person at the time when the second medical image was captured; The information processing system according to claim 18 , wherein the estimation unit estimates current bone condition information of the subject.

22. The information processing system according to claim 19 , wherein the medical information includes at least the first virtual image.

23. A terminal device that presents on a display unit bone condition information regarding the future or past bone condition of a subject, estimated by inputting medical information including first medical information having a medical image of the subject's bones and second medical information having characteristic information of the subject at a time different from the time the medical image was taken into a learning model that has been trained using training data that includes multiple pieces of information having medical images of the bones of a specific person and characteristic information of the specific person at a time different from the time the medical image was taken.

24. an acquisition step of acquiring medical information including first medical information having a medical image showing the bones of the subject and second medical information having characteristic information of the subject at a time point different from the time point at which the medical image was captured; A method for controlling an information processing system, comprising: an estimation step of estimating bone condition information regarding the future or past bone condition of a subject by inputting the medical information into a learning model trained using training data including a plurality of medical images showing the bones of a specific person and characteristic information of the specific person at a time different from the time the medical image was taken.

25. A control program for causing a computer to function as the information processing system according to claim 1 , the control program causing the computer to function as the acquisition unit and the estimation unit.

26. A computer-readable recording medium on which the control program according to claim 25 is recorded.