Prediction device, prediction system, control method, and control program

Abstract

To provide a prediction device, a prediction system, a control method, and a control program for predicting joint symptoms and the like.SOLUTION: In a prediction system including a plurality of prediction devices 1 and one or more terminal devices 7 communicatively connected, a prediction device for a medical facility includes: a first prediction unit having a first prediction model that outputs, from a medical image showing a target at a first time point, first prediction information predicting a disease of the target at a second time point which is after a predetermined period from the first time point; and a second prediction unit having a second prediction model that outputs third prediction information indicating a method of intervention for the target and an effect of the intervention from the first prediction information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a prediction device, a prediction system, a control method, and a control program for predicting joint symptoms, etc. [Background technology]

[0002] Patent Document 1 discloses a technique for determining osteoporosis. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2008-36068 Summary of the Invention

[0004] A prediction device according to one embodiment of the present disclosure includes a first prediction unit having a first prediction model that outputs first prediction information that predicts a disease of a subject at a second time point that is a predetermined period of time from a first time point based on a medical image of the subject at the first time point, and a second prediction unit having a second prediction model that outputs third prediction information that indicates a method of intervention for the subject and the effect of the intervention based on the first prediction information.

[0005] Furthermore, a control method according to one embodiment of the present disclosure is a control method for a computer that causes a computer to execute a first prediction step of outputting first prediction information that predicts a disease of the subject at a second time point that is a predetermined period of time from a medical image showing the subject at a first time point, and a second prediction step of outputting third prediction information that indicates a method of intervention in the subject and the effect of the intervention from the first prediction information, wherein the computer has a first prediction model capable of estimating the first prediction information from the medical image, and a second prediction model capable of estimating the third prediction information from the first prediction information.

[0006] Furthermore, the prediction device according to each aspect of the present disclosure may be realized by a computer. In this case, the control program of the prediction device that causes the computer to operate as each unit (software element) of the prediction device to realize the prediction device, and the computer-readable recording medium on which the control program is recorded, also fall within the scope of the present disclosure. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating a configuration example of a prediction system according to an embodiment of the present disclosure. [Diagram 2] FIG. 13 is a diagram illustrating a configuration example of a prediction system according to another aspect of the present disclosure. [Diagram 3] 1 is a block diagram illustrating an example of a configuration of a prediction device according to an aspect of the present disclosure. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a neural network included in a first prediction section, a second prediction section, and a third prediction section. [Diagram 5] 10 is a flowchart showing an example of the flow of a learning process by a first learning unit. [Figure 6] 10 is a flowchart showing an example of the flow of a learning process by a second learning unit. [Figure 7] 13 is a flowchart showing an example of a flow of a process performed by the prediction device. [Figure 8] FIG. 13 is a diagram illustrating an example of a medical service realized by the prediction device. [Figure 9] FIG. 13 is a block diagram illustrating an example of a configuration of a prediction device according to another aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The present disclosure will be described in detail below.

[0009] A joint disease of a subject can significantly reduce the subject's quality of life (QOL), but in order to reduce the reduction in QOL caused by a joint disease of a subject, it is important to discover the onset of the disease and the progression of symptoms as early as possible and to start intervention with an appropriate method such as lifestyle guidance. In contrast, in one embodiment of the present disclosure, highly reliable information on the onset of a joint disease of a subject or the degree of progression of symptoms can be presented. Therefore, in one embodiment of the present disclosure, the reduction in the QOL of a subject can be reduced.

[0010] Each embodiment of the present disclosure will be described below. In the following description, the subject will be described as a "subject" when the subject is a human, but the subject is not limited to humans. The subject may be, for example, a mammal other than a human, such as an equine, feline, canine, bovine, or porcine animal. The present disclosure also includes, among the following embodiments, embodiments in which the "subject" is replaced with "animal" if the embodiment is applicable to these animals.

[0011] [Embodiment 1] (Outline of Prediction Device 1) A prediction device 1 according to an embodiment of the present disclosure is a device that outputs first prediction information regarding a future symptom of a joint of a subject from a medical image showing the joint of the subject at a first time point, using a first prediction model. Here, the first prediction model is a prediction model capable of estimating the first prediction information from the medical image.

[0012] The first time point may be, for example, a time point when a medical image of a joint of a subject is acquired. Typically, the first time point may be a time point when a medical image of a current state of a joint of a subject is acquired. In other words, the first time point may substantially mean the current time point.

[0013] The first prediction information is information on the symptoms of the joints of the subject at a second time point after a predetermined period of time has elapsed since the first time point. The predetermined period of time may be any period of time that has elapsed since the first time point, and may be half a year, one year, five years, ten years, or fifty years. That is, the second time point may be intended to be any time point in the future. The predetermined period of time is not limited to one period of time, and may include multiple periods of time. That is, the first prediction information may include information indicating the symptoms of the joints of the subject at multiple times, such as six months, one year, five years, ten years, and fifty years after the first time point. The first prediction information is, for example, information including predictions regarding symptoms that are likely to occur in the joints of the subject at a future time point, the time when the symptoms will occur, and the degree of progression of the symptoms of the joints of the subject.

[0014] More specifically, the first prediction information may include information on symptoms likely to occur in the subject's joints, such as peri-articular alignment, articular cartilage thickness, joint gap, osteophyte formation, the presence or absence of synovitis, Kellgren-Laurence (KL) classification, range of motion of the joint, joint stiffness, the presence or absence of lameness, etc.

[0015] The prediction device 1 can present information regarding symptoms of diseases that will occur in the subject's joints in the future (e.g., osteoarthritis, rheumatoid arthritis, osteonecrosis, and other joint-specific diseases) to the subject and the doctor in charge of the subject at a time when the symptoms have not yet occurred.

[0016] The prediction device 1 is capable of outputting first prediction information for a subject's joints including any one of the temporomandibular joint, spinal intervertebral joint, shoulder joint, acromioclavicular joint, elbow joint, wrist joint, hand, fingers, hip joint, sacroiliac joint, knee joint, ankle joint, foot, and toes.

[0017] The prediction device 1 can use any medical image that can analyze the shape and arrangement of bones that constitute the joint of the subject to be predicted, the angle between the bones that form the joint, and the state of the surrounding soft tissues, etc. Therefore, the medical image may include at least one of a plain X-ray image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, and an ultrasound image, for example.

[0018] (Configuration of prediction system 100a) First, a configuration of a prediction system 100a according to an embodiment of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an example of the configuration of a prediction system 100a in a medical facility 8 in which a prediction device 1 is installed.

[0019] The prediction system 100a includes a prediction device 1 and one or more terminal devices 7 communicably connected to the prediction device 1. The prediction device 1 is a computer that outputs first prediction information from a medical image showing a joint of a subject and transmits the first prediction information to the terminal device 7. The terminal device 7 receives the first prediction information from the prediction device 1 and presents the first prediction information. The terminal device 7 is a computer used by a medical professional such as a doctor belonging to a medical facility 8. The terminal device 7 is, for example, a personal computer, a tablet terminal, a smartphone, etc. The terminal device 7 has a communication unit that transmits and receives data to and from other devices, an input unit such as a keyboard and a microphone, a display unit that can display information included in the first prediction information, an output unit such as a speaker, etc. In the medical facility 8 shown in FIG. 1, a LAN (local area network) is arranged, and an example is shown in which the prediction device 1 and the terminal device 7 are connected to the LAN, but this is not limited to this. For example, the network in the medical facility 8 may be the Internet, a telephone communication line network, an optical fiber communication network, a cable communication network, a satellite communication network, etc.

[0020] In addition to the prediction device 1 and the terminal device 7, a medical image management device 6 may be communicatively connected to the LAN in the medical facility 8. The medical image management device 6 is a computer that functions as a server for managing medical images captured in the medical facility 8. In this case, the prediction device 1 may obtain a medical image showing a joint of a subject from the medical image management device 6.

[0021] The LAN in the medical facility 8 may be communicatively connected to an external communication network. In the medical facility 8, the prediction device 1 and the terminal device 7 may be directly connected without going through a LAN.

[0022] (Configuration of prediction system 100b) The prediction device 1 may not be a computer installed in a specific medical facility 8, but may be communicatively connected to a LAN installed in each of a plurality of medical facilities 8 via a communication network 9. Fig. 2 is a diagram showing a configuration example of a prediction system 100b according to another embodiment of the present disclosure.

[0023] In addition to one or more terminal devices 7a, an electronic medical record management device 5a and a medical image management device 6a may be communicably connected to the LAN in the medical facility 8a. In addition to the terminal device 7b, an electronic medical record management device 5b and a medical image management device 6b may be communicably connected to the LAN in the medical facility 8b. In the following, when there is no particular distinction between the medical facilities 8a and 8b, they will be referred to as the "medical facility 8." In addition, when there is no particular distinction between the terminal devices 7a and 7b, and the medical image management devices 6a and 6b, they will be referred to as the "terminal devices 7" and the "medical image management devices 6," respectively.

[0024] 2 shows an example in which LANs of a medical facility 8a and a medical facility 8b are connected to a communication network 9. The prediction device 1 is only required to be communicably connected to devices in each medical facility via the communication network 9, and is not limited to the configuration shown in FIG 2. For example, the prediction device 1 may be installed in the medical facility 8a or the medical facility 8b.

[0025] In the prediction system 100b having such a configuration, the prediction device 1 can acquire medical images of the subject Pa who has been examined at the medical facility 8a from the medical image management device 6a of the medical facility 8a. Then, the prediction device 1 transmits first prediction information regarding the joints of the subject Pa to the terminal device 7a installed at the medical facility 8a. Similarly, the prediction device 1 can acquire medical images of the subject Pb who has been examined at the medical facility 8b, and transmit the first prediction information regarding the joints of the subject Pb to the terminal device 7b installed at the medical facility 8b.

[0026] In this case, the medical image of each subject may include identification information (e.g., facility ID) unique to each medical facility 8 that examines each subject, and identification information (e.g., patient ID) unique to each subject that is assigned to each subject. Based on these identification information, the prediction device 1 can correctly transmit the first prediction information output from the medical image of the subject to the terminal device 7 of each medical facility 8 where the subject was examined.

[0027] (Configuration of Prediction Device 1) Next, the configuration of the prediction device 1 applied to the prediction system 100a shown in Fig. 1 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the prediction device 1.

[0028] The prediction device 1 includes a control unit 2 that comprehensively controls each unit of the prediction device 1, and a storage unit 3 that stores various data used by the control unit 2. The control unit 2 includes an image acquisition unit 21, a first prediction unit 23, an output control unit 24, a second prediction unit 25, a first learning unit 27, and a second learning unit 28. The storage unit 3 stores a control program 31, which is a program for performing various controls of the prediction device 1, first learning data 32 (patient information), and second learning data 33 (effect information).

[0029] <Image acquisition unit 21> The image acquisition unit 21 acquires a medical image, which is an image of a joint of a subject at a first time point, from the medical image management device 6. The medical image is input data to be input to the first prediction unit 23.

[0030] <First Prediction Unit 23> The first prediction unit 23 outputs first prediction information about a joint at a second time point after a predetermined period has elapsed from a first time point, from a medical image showing the joint of a subject. The first prediction unit 23 has a trained neural network including an input layer 231 and an output layer 232 (see FIG. 4).

[0031] The first prediction unit 23 has a neural network trained using first learning data 32, which is time-series data on joint symptoms acquired from each of a plurality of patients at a plurality of time points in the past. This trained neural network is used as a first prediction model capable of outputting first prediction information from medical images of a subject. The neural network is a mathematical model that imitates the human brain and nervous system, and each parameter of this mathematical model is determined by a learning process, and prediction information can be predicted by performing a calculation according to this trained mathematical model or an arithmetic formula.

[0032] In response to a medical image of a subject being input to an input layer 231 (see FIG. 4), the first prediction unit 23 performs a calculation based on a first prediction model and outputs first prediction information from an output layer 232 (see FIG. 4). As an example, the first prediction unit 23 may be configured to extract a feature amount from the medical image and use it as input data. The following known algorithms may be applied to extract the feature amount. ·Convolutional neural network (CNN) · Autoencoder Recurrent neural network (RNN) ·LSTM (Long Short-Term Memory).

[0033] The first prediction model is a calculation model used when the first prediction unit 23 performs calculations based on input data. The first prediction model is generated by executing machine learning using the first learning data 32, which will be described later, on the neural network of the first prediction unit 23. Specific examples of the first learning data 32, the configuration of the first prediction unit 23, and the learning process will be described later.

[0034] <Second Prediction Unit 25> Similar to the first prediction unit 23, the second prediction unit 25 has a trained neural network including an input layer 251 and an output layer 252 (see FIG. 4).

[0035] The second prediction unit 25 has a neural network trained using second learning data 33 including time series data on joint symptoms acquired at multiple time points in the past from each of multiple patients to whom an intervention was applied in the past. This trained neural network is used as a second prediction model capable of estimating third prediction information from the first prediction information of the subject. Here, the second prediction model is a prediction model generated based on time series data on joint symptoms acquired from each of multiple patients to whom an intervention was applied.

[0036] The third prediction information is information indicating a method of intervention for the subject and the effect of the intervention. More specifically, it is information indicating the type of intervention and information indicating the effect of the intervention. The effect of the intervention is information indicating the symptoms of the subject's joints at the second time point when the intervention is applied. Alternatively, the effect of the intervention may be information indicating the degree to which the application of the intervention improves the symptoms of the disease related to the subject's joints at the second time point or the degree to which the progression of the symptoms is suppressed compared to the case where the intervention is not applied. The third prediction information may include information indicating the time when the intervention should be applied (intervention time).

[0037] In response to the first prediction information being input to the input layer 251 (see FIG. 4), the second prediction unit 25 performs a calculation based on the second prediction model, and outputs the third prediction information from the output layer 252 (see FIG. 4). As an example, the second prediction unit 25 may be configured to extract a feature amount from the first prediction information and use it as input data. The following known algorithms may be applied to extract the feature amount. ·Convolutional neural network (CNN) · Autoencoder Recurrent neural network (RNN) ·LSTM (Long Short-Term Memory).

[0038] The second prediction model is a calculation model used when the second prediction unit 25 performs calculations based on input data. The second prediction model is generated by executing machine learning using second learning data 33, which will be described later, on the neural network of the second prediction unit 25. Specific examples of the second learning data 33, the configuration of the second prediction unit 25, and the learning process will be described later.

[0039] <Output control unit 24> The output control unit 24 transmits the first prediction information output from the first prediction unit 23 to the terminal device 7. In addition, the output control unit 24 transmits the third prediction information output from the second prediction unit 25 to the terminal device 7.

[0040] The prediction device 1 may be configured to include a display unit (not shown). In this case, the output control unit 24 causes the display unit to display the first prediction information.

[0041] <First Study Section 27> The first learning unit 27 controls a learning process for the neural network of the first prediction unit 23. For this learning, first learning data 32 (described later) is used. A specific example of the learning performed by the first learning unit 27 will be described later.

[0042] <Second Study Section 28> The second learning unit 28 controls a learning process for the neural network of the second prediction unit 25. For this learning, second learning data 33 (described later) is used. A specific example of the learning performed by the second learning unit will be described later.

[0043] (First learning data 32) The first learning data 32 is data used in machine learning to generate the first prediction model. The first learning data 32 includes first learning input data 321 used as input data and first teacher data 322 for calculating an error with the first prediction information output by the first prediction unit 23.

[0044] For example, the first learning input data 321 may include patient images of bones, joints, and soft tissues of each of a plurality of patients. The first teacher data 322 may include symptom information on the joints of each patient at the time the patient images were captured. Here, the symptom information may include information on the onset time of a joint disease or the progression of symptoms. The patient images may include at least one of X-ray images, CT images, MRI images, and ultrasound images of each of the joints of a plurality of patients.

[0045] The first learning data 32 may be data in which the first learning input data 321 and the first teacher data 322 are integrated. That is, the first learning data 32 may be time-series data in which patient images acquired from each of a plurality of patients at a plurality of time points in the past are associated with information on joint symptoms at the time the patient images were captured. For example, the first learning data 32 may include parameters indicating feature amounts extracted from information such as the alignment around the joint of the patient at a certain time point and one year after the certain time point, the thickness of the articular cartilage, osteophyte formation, the presence or absence of synovitis, the KL classification, the range of motion of the joint, the degree of pain, the degree of stiffness of the joint, and the presence or absence of lameness. The first learning data 32 may also include parameters indicating attributes of the patient. The attributes of the patient are, for example, the height, weight, age, and sex of each patient. When the first learning data is time series data, the first learning unit 27 may use a patient image at a certain point in time as first learning input data 321, and use a patient image a predetermined period of time after the certain point in time, information about the joint symptoms at the time the patient image was captured, and information about the patient as first teacher data 322.

[0046] The first learning data 32 may include information on the QOL of each of a plurality of patients in time series data. For example, information such as WOMAC, SF-36, and VAS may be included. The first prediction unit 23 having a first prediction model generated by machine learning using such first learning data 32 is capable of outputting second prediction information related to the subject's QOL at a second time point from a medical image of the subject.

[0047] (Second training data 33) The second learning data 33 (effect information) is data used in machine learning to generate a second prediction model. The second learning data 33 includes second learning input data 331 used as input data and second teacher data 332 for calculating an error with the first prediction information output by the first prediction unit 23. Specifically, the second learning input data 331 may include information indicating the time when the intervention was applied to each of a plurality of patients to whom the intervention was applied, a patient image showing the joint of each patient, and symptom information regarding the onset or progression of a joint symptom at the time when the patient image of each patient was captured. In addition, the second teacher data 332 may include a patient image showing the joint of the patient at a time later (for example, one year later) than the time when the above-mentioned patient image was captured, and symptom information regarding the onset or progression of a joint symptom of the patient.

[0048] The second learning data 33 may be time-series data in which patient images obtained at multiple time points from each of multiple patients to whom intervention was applied in the past are associated with information regarding joint symptoms at the time the patient images were captured.

[0049] (Configuration of the first prediction unit 23 and the second prediction unit 25) Hereinafter, the configuration of the first prediction unit 23 and the second prediction unit 25 will be described with reference to Fig. 4. The configuration shown in Fig. 4 is an example, and the configurations of the first prediction unit 23 and the second prediction unit 25 are not limited thereto. The first prediction unit 23 and the second prediction unit 25 may have the same configuration.

[0050] <Configuration of first prediction unit 23> 4, the first prediction unit 23 and the second prediction unit 25 perform calculations based on a first prediction model on input data input to an input layer 231, and output first prediction information from an output layer 232. Furthermore, the first prediction unit 23 may output second prediction information in addition to the first prediction information.

[0051] The first prediction unit 23 in FIG. 4 includes a neural network including an input layer 231 and an output layer 232. The neural network may be any neural network suitable for handling time-series information. For example, LSTM may be used. The neural network is preferably any neural network suitable for handling time-series information and position information in combination. For example, a ConvLSTM network combining CNN and LSTM may be used. The input layer 231 can extract a feature of the time change of the input data. The output layer 232 can calculate a new feature based on the feature extracted by the input layer 231, the time change of the input data, and an initial value. The input layer 231 and the output layer 232 have a plurality of LSTM layers. Each of the input layer 231 and the output layer 232 may have three or more LSTM layers. The first learning input data 321 for learning is input to the input layer 231 of the first prediction unit 23 during learning of the first prediction unit 23. The data on joint symptoms, which becomes the first teacher data 322, is compared with output data output from the output layer 232 of the first prediction unit 23 during learning of the first prediction unit 23. The first prediction unit 23 is optimized by known machine learning using the first learning input data 321 and the first teacher data 322. The first prediction unit 23 adjusts variable parameters in the first prediction unit 23 so that the difference between the pseudo prediction result calculated from the first learning input data 321 input to the input layer 231 and output from the output layer 232 and the first teacher data 322 becomes smaller.

[0052] Specifically, when the first learning input data 321 is input to the input layer 231, a plurality of pixel data constituting the first learning input data 321 are input to a plurality of artificial neurons, respectively. Then, parameters are adjusted so that the error of the pseudo prediction result output from the output layer 232 with respect to the first teacher data 322 is reduced. The adjusted parameters become learned parameters and are stored in the first prediction model or the second prediction model.

[0053] As a method for adjusting the parameters, for example, the backpropagation method is adopted. The parameters include, for example, parameters used in the input layer 231 and the output layer 232. Specifically, the parameters include weighting coefficients used in the input layer 231 and the LSTM layer of the output layer 232. As a result, the first prediction model performs a calculation based on the first prediction model on the medical image data input to the input layer 231, and outputs the first prediction information from the output layer 232. As described above, in the prediction device, the medical image data is used to perform learning of the first prediction unit 23 and prediction in the first prediction unit 23. Therefore, a medical image can be input to the prediction device and output as the first prediction information regarding future joint symptoms.

[0054] Specifically, the input data used during learning by the first prediction unit 23 is a medical image of a patient at a certain time point A included in the first learning input data 321. Based on the above-mentioned input data, the first prediction unit 23 outputs, as output data, a prediction result of the onset or progression degree of a disease related to a joint of the patient from time point A to time point B after a predetermined period (e.g., three years) has elapsed. Specifically, the first prediction unit 23 outputs, for example, the angle formed between the bones constituting the joint of the patient at time point B, a parameter indicating the degree of wear of the cartilage, a value indicating a rank according to the degree of deformation of the joint, the time and degree of pain that will develop in the joint of the patient, and information indicating the time when invasive treatment will be required for the joint of the patient. The output data shown here is an example and is not limited to these.

[0055] The input data used by the first prediction unit 23 when outputting the first prediction information is, for example, a parameter indicating a feature amount extracted from a medical image of the subject at the first time point. The first prediction unit 23 outputs, as the first prediction information, a prediction result of the onset or progression degree of a disease related to the subject's joint up to the second time point based on the above-mentioned input data. Specifically, the first prediction unit 23 outputs, as the first prediction information, for example, an angle formed between bones constituting the joint of the subject at the second time point, a parameter indicating the degree of wear of the cartilage, a value indicating a class according to the degree of deformation of the joint, the time and degree of pain onset in the subject's joint, and information indicating the time when invasive treatment is required for the patient's joint. The first prediction information shown here is an example and is not limited thereto.

[0056] Furthermore, the first prediction unit 23 outputs information indicating the QOL of the subject as the second prediction information based on the above-mentioned first prediction information. Specifically, the first prediction unit 23 outputs at least one of information regarding pain occurring in the joints of the subject, information regarding the catastrophic thinking of the subject, information regarding the motor ability of the subject, information indicating the life satisfaction of the subject, and information regarding the degree of stiffness in the joints of the subject as the second prediction information.

[0057] Information indicating the subject's QOL is information that includes at least one of the following: Information about the pain the subject is experiencing in their joints Information about the subject's catastrophic thinking Information about the subject's athletic ability Information indicative of said subject's life satisfaction.

[0058] Information indicative of the subject's QOL may include information regarding the subject's (1) physical functioning, (2) role-based functioning (physical), (3) bodily pain, (4) overall well-being, (5) vitality, (6) social functioning, (7) role-based functioning (mental), and (8) mental health.

[0059] The basic information may also include, for example, information such as the range of motion of the subject's joints, weight support index, the presence or absence of lameness, KL classification, degree of stiffness, etc. Information related to QOL may also include, for example, information such as WOMAC (Western Ontario and McMaster Universities Osteoarthritis Index), SF-36 (36-Item. Short-Form Health Survey), VAS (Visual analog scale), etc.

[0060] <Configuration of second prediction unit 25> The second prediction unit 25 may also have a configuration similar to that of the first prediction unit 23, and specifically includes an input layer 251 and an output layer 252. The second prediction unit 25 acquires the first prediction information from the first prediction unit 23 and uses it as input data to be input to the input layer 251. The second prediction unit 25 may further acquire medical images and use them as input data. The second prediction unit 25 performs calculations using a second prediction model based on the first prediction information, and outputs third prediction information.

[0061] Specifically, the input data used during learning of the second prediction unit 25 are information indicating the onset or progression of a disease related to the patient's joint at a certain time point B, which is included in the second learning data 33, and information indicating a method of intervention, which is included in the intervention information 34. Based on the above-mentioned input data, the second prediction unit 25 outputs information indicating the degree to which the symptoms of the disease related to the patient's joint are improved or the degree to which the progression of the symptoms is suppressed when the intervention at time point B is applied. More specifically, the second prediction unit 25 may output information such as, for example, about how much the deformation of the patient's joint is suppressed, how many years the time when pain occurs can be delayed, how many years the time when surgery is required can be delayed, how much the pain is reduced, how much walking ability (including the ability to climb and descend stairs) is improved, or how much the QOL is improved.

[0062] In addition, the second prediction unit 25 outputs, as third prediction information, information indicating the degree to which symptoms of the subject's joint disease will be improved or the degree to which the progression of the symptoms will be suppressed when the intervention is applied, based on the first prediction information and information indicating the method of intervention contained in the intervention information 34.

[0063] (Intervention information 34) Returning to FIG. 3, the memory unit 3 stores in advance information (intervention information 34) on an intervention whose effect is estimated by the second prediction unit 25. When the second prediction unit 25 acquires the first prediction information, the memory unit 3 refers to the memory unit 3, selects at least one of the intervention methods included in the intervention information 34, and estimates the effect of the intervention. Examples of the intervention information 34 whose effect is estimated include non-invasive treatments such as weight restriction, thermotherapy, ultrasound therapy, wearing an orthosis, or taking supplements (e.g., chondroitin or glucosamine). In addition, the effect of an invasive treatment such as artificial joint replacement may be estimated as the intervention information 34.

[0064] (Learning process by first learning unit 27) The learning process for generating the first prediction model will be described below with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of the learning process by the first learning unit 27.

[0065] The first learning unit 27 acquires the first learning input data 321 included in the first learning data 32 from the storage unit 3 (step S1). The first learning input data 321 includes patient images of patients A, B, . . .

[0066] Next, the first learning unit 27 inputs a patient image of patient A at time point A, which is included in the first learning input data 321, to the input layer 231 (step S2). The input layer 231 may extract parameters indicating feature amounts from the input patient image.

[0067] Next, the first learning unit 27 acquires output data related to the symptoms of the joints of the patient A from the output layer 232 (step S3). This output data includes the same content as the first teacher data 322.

[0068] Next, the first learning unit 27 acquires the first learning input data 321 included in the first learning data 32. Then, the first learning unit 27 compares the acquired output data with symptom information of patient A at time point B included in the first teacher data 322, and calculates an error (step S4), and the first learning unit 27 adjusts the first prediction model so as to reduce the error (step S5).

[0069] Any known method can be applied to adjust the first prediction model. For example, an error backpropagation method may be adopted as a method for adjusting the first prediction model. The adjusted first prediction model becomes a new first prediction model, and the first prediction unit 23 uses the new first prediction model in subsequent calculations. In the adjustment stage of the first prediction model, parameters (e.g., filter coefficients, weighting coefficients, etc.) used in the first prediction unit 23 may be adjusted.

[0070] If the error is not within the predetermined range and if the patient images of all patients included in the first learning data 32 have not been input (NO in step S6), the first learning unit 27 returns to step S2 and repeats the learning process. If the error is within the predetermined range and if the patient images of all patients included in the first learning data 32 have been input (YES in step S6), the first learning unit 27 ends the learning process.

[0071] (Learning process by second learning unit 28) Next, the learning process for generating the second prediction model will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the flow of the learning process by the second learning unit .

[0072] The second learning unit 28 acquires second learning input data 331 from the storage unit 3 (step S11). The second learning input data 331 includes symptom information on the symptoms of the joints of each patient and symptom information of a patient (patient A) to whom a certain intervention (hereinafter, intervention X) was applied.

[0073] Next, the second learning unit 28 inputs symptom information on the joint symptoms of the patient A at the time point B, which is included in the second learning input data 331, to the input layer 231 (step S12).

[0074] Next, the second learning unit 28 acquires output data related to the symptoms of the joints of the patient A from the output layer 232 (step S13). This output data includes the same content as the second teacher data 332.

[0075] Next, the second learning unit 28 acquires the second teacher data 332. Then, the second learning unit 28 compares the acquired output data with the symptom information of the patient A to which the intervention X was applied at the time point B, which is included in the second teacher data 332, and calculates an error (step S14), and the learning unit 27 adjusts the second prediction model so as to reduce the error (step S15).

[0076] Any known method can be applied to adjust the second prediction model. For example, the error backpropagation method may be adopted as a method for adjusting the second prediction model. The adjusted second prediction model becomes a new second prediction model, and the second prediction unit 25 uses the new second prediction model in the subsequent calculations. In the stage of adjusting the second prediction model, parameters (e.g., filter coefficients, weighting coefficients, etc.) used in the first prediction unit 23 may be adjusted.

[0077] If the error is not within the predetermined range and if the symptom information of all patients included in the second learning data 33 has not been input (NO in step S16), the second learning unit 28 returns to step S12 and repeats the learning process. If the error is within the predetermined range and if the symptom information of all patients included in the second learning data 33 has been input (YES in step S16), the second learning unit 28 ends the learning process.

[0078] (Processing performed by prediction device 1) The flow of processing performed by the prediction device 1 will be described below with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of processing performed by the prediction device 1.

[0079] <Output of first forecast information> First, the image acquiring unit 21 acquires a medical image (input data) from the medical image management device 6 (step S21: image acquiring step), and inputs it to the first prediction unit 23.

[0080] Next, in response to the input of the medical image, the first prediction unit 23 outputs first prediction information (step S22: prediction step).

[0081] <Output of second forecast information> When the basic information includes information related to the subject's QOL, in step S22, the first prediction unit 23 may further output second prediction information in response to the input of the medical image.

[0082] <Output of the third forecast information> When the second prediction unit 25 acquires the first prediction information, it refers to the intervention information 34 in the storage unit 3 and selects at least one of the intervention methods included in the intervention information 34 (step S23). In addition, in response to the input of the first prediction information, the second prediction unit 25 outputs third prediction information about the selected intervention (step S24).

[0083] (Medical services realized by prediction device 1) FIG. 8 is a diagram showing an example of a medical service realized by the prediction device 1. As shown in FIG.

[0084] When a subject visits a medical facility 8 due to knee joint pain or the like and is suspected of having a disease related to the joint, the doctor or the like in charge of the subject (user) acquires an X-ray (medical image) of the subject at that time (first time point). When the user inputs the acquired information to the prediction device 1, the prediction device 1 outputs the state of the subject's joint at a second time point (first prediction information). The prediction device 1 also outputs information (third prediction information) indicating the effect of an intervention when lifestyle advice or the like is applied to the subject. By using each piece of information output by the prediction device 1, the user can present an intervention such as lifestyle advice that is more preferable to the subject. It is also considered that the subject will accept the intervention more positively if he or she knows the effect of the intervention.

[0085] [Other Aspect 1] <Basic information acquisition section 22> In addition to the above-mentioned configuration, the prediction device 1 may further include a basic information acquisition unit 22 as shown in FIG. 3. The basic information acquisition unit 22 acquires basic information, which is information related to the subject, from the electronic medical record management device 5. The electronic medical record management device 5 is a computer that functions as a server for managing electronic medical record information of subjects who have been examined at the medical facility 8. The electronic medical record management device 5a may be communicably connected to the LAN in the medical facility 8a in addition to one or more terminal devices 7a and the medical image management device 6a. The electronic medical record information may include basic information and interview information of the subject. The basic information is input data input to the first prediction unit 23 in addition to the medical image. The first prediction model of the first prediction unit 23 is capable of estimating the first prediction information based on the basic information in addition to the medical image.

[0086] The basic information includes the subject's height, age, and sex. The basic information may further include at least one of the subject's body mass index (BMI), type (e.g., race, etc.), occupational history, exercise history, medical history of joint disease, and information on bone morphology or synovial fluid / synovial membrane, biomarker information, and genetic information. The basic information may include, for example, information included in the subject's electronic medical record information, etc. The basic information may be interview information obtained from the subject through an interview conducted at a medical facility 8, etc., and may include, for example, information related to the subject's QOL at a first time point.

[0087] In addition, when the first prediction unit 23 uses basic information as input data, the first learning unit 27 may, when learning the first prediction unit 23, input general symptom information and attribute information of the patient in addition to medical images of the patient at a certain point in time to the first prediction unit 23 as first learning input data 321.

[0088] According to the above configuration, the prediction device 1 acquires basic information of the subject from the electronic medical record management device 5 in addition to the medical images, and transmits first prediction information regarding the joints of the subject Pa to the terminal device 7a installed in the medical facility 8a. Similarly, the prediction device 1 can acquire medical images and basic information of the subject Pb who has been examined at the medical facility 8b, and transmit the first prediction information regarding the joints of the subject Pb to the terminal device 7b installed in the medical facility 8b.

[0089] In addition to the first prediction unit 23 and the second prediction unit 25, the prediction device 1 may further include a third prediction unit 26 that outputs fourth prediction information. The prediction device 1 may also include a third learning unit 29 that performs a learning process for the third prediction unit 26. Here, the fourth prediction information is clinical outcome information of a subject at a third time point, which is a time point after a predetermined period has elapsed since the application of the intervention when the intervention is applied. The third prediction unit 26 can be generated by machine learning using the same second learning data 33 as the second prediction unit 25. However, the third prediction unit 26 is a neural network that outputs output data different from that of the second prediction unit 25. The third prediction unit 26 has a third prediction model generated based on time-series data (clinical outcome information) regarding the intervention method that is included in the second learning data 33 and that was acquired at multiple time points in the past from each of multiple patients to whom a certain intervention was applied in the past. The third prediction unit 26 performs a calculation using the third prediction model based on the first prediction information and the intervention information 34, and outputs fourth prediction information regarding the clinical outcome of the subject after the intervention at the third time point.

[0090] Clinical outcome information is information on patients who have undergone surgical treatment, such as arthroscopic surgery, joint-preserving surgery (e.g., osteotomy), and artificial joint replacement, and examples of clinical outcome information include pain, joint stiffness, range of motion of the joint, walking ability, and alignment around the joint. Clinical outcome information for arthroscopic surgery is not particularly limited, but examples include the presence or absence of synovitis and the presence or absence of ulcerative colitis, and clinical outcome information for artificial joint replacement includes pain, range of motion of the joint, walking ability, and the like.

[0091] The third prediction unit 26 may also have a configuration similar to that of the first prediction unit 23, and specifically includes an input layer 261 and an output layer 262. The third prediction unit 26 outputs, as fourth prediction information, for example, clinical outcomes at time point B of an intervention applied at time point A.

[0092] Further, the third learning unit 29 performs learning of the third prediction unit 26. The flow of the learning process performed in the third learning unit 29 is similar to the process performed in the first learning unit 27 and the second learning unit 28. However, the third learning unit 29 inputs symptom information of a certain patient A at time point B and information indicating a certain intervention X, which are included in the second learning data 33, as input data to the third prediction unit 26. The third learning unit 29 outputs, based on the input data, the clinical outcome of the intervention X at time point B when the intervention X is applied, as output data to the third learning unit 29. The third learning unit 29 compares the output data with the clinical outcome information of the intervention X at time point B, which is included in the second learning data 33, and adjusts the third prediction model.

[0093] Furthermore, the output control unit 24 may output information at the time of imaging of the subject in addition to the first prediction information, the second prediction information, and the third prediction information. Specifically, the output control unit 24 may output a medical image of the subject at the first time point, information on symptoms of the joints, information indicating the degree of progression of the symptoms, and the like.

[0094] [Embodiment 2] Hereinafter, a case will be described in which the prediction device 1 is applied to predict information related to knee osteoarthritis (hereinafter referred to as KOA). KOA is a disease in which abnormalities occur in the shape and cartilage of the knee joint, causing the knee joint to gradually deform. The number of KOA patients in Japan is estimated to be approximately 24 million. Approximately one-third of these patients suffer from knee pain. When KOA develops and symptoms worsen, it becomes necessary to undergo knee replacement surgery.

[0095] In order to prevent the onset of KOA and suppress the progression of symptoms, it is important to start intervention such as lifestyle guidance as early as possible. By applying the prediction device 1 according to one embodiment of the present disclosure, it is possible to predict the onset time of KOA and the progression of symptoms.

[0096] Hereinafter, a specific example of each data when the prediction device 1 is applied to prediction of information related to KOA will be described.

[0097] In this case, the input data input to the input layer 231 is a medical image at a first time point. The first time point is, for example, the time point when the current condition of the subject's knee is examined at the medical facility 8. Furthermore, basic information of the subject at the first time point may be input to the input layer 231 in addition to the medical image.

[0098] The medical image may be, for example, at least one of a lower limb X-ray (X-ray image), a lower limb CT image (CT image), a lower limb MRI image (MRI image), and a lower limb ultrasound image taken of the subject at a first time point. The X-ray image may be at least one of a standing frontal image, a frontal image, a lateral image, a Rosenberg View, and a patella axial image. A positioner for fixing the knee may be used during imaging.

[0099] The basic information may also include information indicating the subject's height, weight, age, and sex at the first time point. The basic information may further include at least one of the following information: Type (e.g. race), presence or absence of injury or illness, type of injury or illness, occupation, occupational history, lifestyle habits, exercise habits, exercise history, smoking habits, information on bone morphology, genetic information, composition of knee joint fluid, lower limb muscle mass, quadriceps strength, frequency of use of walking aids, presence or absence of medication, presence or absence of use of anti-inflammatory drugs, information on pain occurring in the knee (e.g. presence or absence of pain), information indicating the range of motion of the knee joint, information on the subject's physical ability (walking ability), information indicating the subject's life satisfaction, medical history of joint diseases, biomarker information, and a value indicating a class according to the degree of deformation of the knee joint (e.g. KL classification, lumbar field classification).

[0100] The information indicating the subject's life satisfaction may be a score obtained by performing a diagnosis on the subject's QOL. The score can be evaluated using, for example, SF-36, WOMAC, IKDC (International Knee Documentation Committee), KOOS (Knee injury and Osteoarthritis Outcome Score), JKOM (Japanese knee osteoarthritis measure), etc.

[0101] The genetic information may include information on at least one of the genes encoding asporin, GDF5, and DVWA, which are information on genes thought to be associated with the progression of KOA.

[0102] The injury may be, for example, a meniscus injury, anterior cruciate ligament injury, posterior cruciate ligament injury, and trauma.

[0103] (First learning data 32) The first learning data 32 in this embodiment is time-series data on symptoms of the knee joints acquired at multiple points in time in the past from each of multiple patients with diseases related to the knee joints. The first learning data 32 may also include the following (1) to (3) for each of the multiple patients.

[0104] (1) Patient image showing knee joint (2) symptom information regarding the onset or progression of a knee joint disease at the time the patient image was taken; (3) Patient-related information related to the quality of life at the time the patient image was captured is associated with the patient-related information.

[0105] As an example, the first learning unit 27 may use a patient image at a certain point in time from the data included in the first learning data 32 as first learning input data 321, and may use the patient image, symptom information and patient-related information of the patient at a predetermined time after that point in time as first teacher data 322.

[0106] (First forecast information) The first prediction unit 23 outputs first prediction information in response to the input of the above-mentioned input data. The first prediction information may include at least one of the following pieces of information. A value indicating the classification (e.g., KL classification) according to the degree of deformation of the knee joint at the second time point of the subject Subject's knee range of motion at time point 2 -Knee joint space width at the second time point for each subject The subject's femoro-tibial angle at the second time point The time and severity of the subject's knee pain When the subject begins to have difficulty walking When the subject's knee will require invasive treatment.

[0107] The information output as the first prediction information may be any information that can be predicted by the first prediction model, and is not limited to these.

[0108] (Second training data 33) The second learning data 33 according to this embodiment is time-series data of medical images when an intervention is applied. The second learning data 33 may also include basic information of the patient to whom the intervention is applied.

[0109] The second prediction unit 25 outputs third prediction information indicating an estimated result of the effect of each intervention using the second prediction model. Here, the third prediction information output by the second prediction unit 25 is, for example, the degree of deformation of the subject's knee joint at the second time point when a certain intervention is applied, or the presence / absence and degree of pain, etc. In addition, the second prediction unit 25 may generate an image indicating the state of the subject's knee at the second time point when the intervention is applied, and output it as the third prediction information.

[0110] In addition, when a change in exercise habits is selected as the intervention method, the second prediction unit 25 may also predict and output the muscle mass, muscle strength, and muscle balance of the lower limbs required to suppress knee deformity. In addition, when arthroscopic surgery is selected as the intervention method, the second prediction unit 25 may predict and output the resection range of osteophytes required to suppress knee deformity. In addition, when osteotomy is selected as the intervention method, the second prediction unit 25 may predict and output the alignment correction required to suppress knee deformity.

[0111] (Method of intervention that is expected to be effective) In this embodiment, the intervention method included in the intervention information 34 stored in the storage unit 3 is, for example, non-invasive treatment. For example, the intervention may include weight restriction, thermotherapy, ultrasound therapy, exercise therapy, stimulation therapy, manual therapy, high frequency therapy, drug intake (drug administration), supplement intake, and wearing of an orthosis. The orthosis may include, for example, a walking aid, a supporter, and a foot plate.

[0112] The supplement may include, for example, chondroitin or glucosamine. The drug may include, for example, a nonsteroidal anti-inflammatory drug, acetaminophen, a cyclooxygenase (COX)-2 inhibitor, or sodium chondroitin sulfate. The intervention method may include intra-articular injection, surgical therapy, and invasive treatment. Examples of intra-articular injection include hyaluronic acid, steroids, or regenerative medicine products (PRP: plate rich plasma, MSC: mesenchymal stem call, or Muse cells: Multi-lineage differentiating Stress Enduring cell, etc.). Examples of surgical therapy include arthroscopic surgery, meniscal suture and resection, ligament reconstruction, osteotomy (HTO: high tibial osteotomy, etc.), cartilage regenerative medicine, or total knee replacement.

[0113] The second prediction unit 25 selects at least one of the intervention methods and estimates the effect of the intervention. The second prediction unit 25 may also estimate the effect of applying a combination of multiple interventions.

[0114] [Other aspect 2] The control unit 2 of the prediction device 1 may extract new basic information from the medical image. For example, the control unit 2 may measure at least one of the following based on the medical image: From the images, the angle between the femur and tibia (femoro-tibial angle), muscle mass of the lower limbs, thickness of the articular cartilage, abnormal bone marrow shadow, subchondral cyst, depression, osteophyte formation, meniscus injury, cruciate ligament injury, synovitis, bone sclerosis, changes in ligament tension, joint contracture, bone density of the subchondral bone on the medial or lateral side of the femur, bone density of the subchondral bone on the medial or lateral side of the tibia, trabecular structure of the subchondral bone on the medial or lateral side of the tibia, bone density of the subchondral bone on the medial or lateral side of the tibia, and blood vessel course of the subchondral bone on the medial or lateral side of the tibia.

[0115] The result of measurement by the control unit 2 may be used as basic information to be input to the first prediction unit 23.

[0116] The control unit 2 may also measure the joint space from an X-ray photograph of the subject's lower limbs and use the measurement result as basic information to be input to the first prediction unit 23. The control unit 2 may also measure the subject's external knee varus moment from a video of the subject's walking and detect the presence or absence of lateral thrust and use the detection result as basic information to be input to the first prediction unit 23.

[0117] According to this configuration, the prediction device 1 can perform prediction based on a larger amount of input data, and therefore the accuracy of the first prediction information can be further improved.

[0118] The prediction device 1 may have a third prediction model generated based on time-series data (clinical outcome information) related to the intervention. The clinical outcome information is information on patients who have undergone surgical treatment, such as arthroscopic surgery, osteotomy, and total joint replacement. The clinical outcome information includes, for example, pain, joint stiffness, range of motion of the joint, walking ability, and alignment around the joint (valgus alignment, rotational alignment). The clinical outcome information in arthroscopic surgery includes, but is not limited to, the presence or absence of synovitis and the presence or absence of a stifle symptom. The clinical outcome information in osteotomy includes, but is not limited to, femoro-tibial angle, %MA (Mikulicz line), and patellofemoral joint pressure. The clinical outcome information in total knee replacement includes, but is not limited to, pain, range of motion of the joint, and walking ability.

[0119] [Embodiment 3] Hereinafter, a case will be described in which the prediction device 1 is applied to predict information related to hip osteoarthritis (hereinafter, referred to as HOA). For convenience of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the explanations thereof will not be repeated.

[0120] HOA is a disease in which abnormalities occur in the shape and cartilage of the hip joint, gradually deforming the hip joint. The prevalence of HOA in Japan is estimated to be approximately 1-5% of the population. If HOA develops and symptoms worsen, it becomes necessary to undergo total hip replacement surgery.

[0121] In order to prevent the onset of HOA and suppress the progression of symptoms, it is important to start intervention such as lifestyle guidance as early as possible. By applying the prediction device 1 according to one embodiment of the present disclosure, it is possible to predict the onset time of HOA and the progression of symptoms.

[0122] Hereinafter, a specific example of each data when the prediction device 1 is applied to prediction of information related to HOA will be described.

[0123] In this case, the input data input to the input layer 231 is a medical image at a first time point. The first time point here is a time point at which the current condition of the subject's hip joint can be understood. For example, the first time point is a time point at which the subject visits a hospital. In addition to the medical image, basic information on the subject at the first time point may be input to the input layer 231.

[0124] In this case, the medical image is an image used as input data for performing a calculation on the deformation of the hip joint of the subject. The medical image is, for example, at least one of an X-ray photograph (X-ray image), a CT photograph (CT image), an MRI photograph (MRI image), and an ultrasound image taken around the hip joint of the subject at a first time point.

[0125] The basic information may also include information indicating the subject's height, weight, age, and sex at the first time point. The basic information may further include at least one of the following information: Type (e.g. race), presence or absence of injury or illness, type of injury or illness, occupation, occupational history, lifestyle habits, exercise habits, exercise history, smoking habits, information on bone morphology, genetic information, composition of joint fluid, lower limb muscle mass, frequency of use of walking aids, presence or absence of medication, presence or absence of use of anti-inflammatory drugs, information on pain in the hip joint (e.g. presence or absence of pain), information indicating the range of motion of the hip joint, information on the subject's physical ability (walking ability), information indicating the subject's life satisfaction, medical history of joint diseases, and biomarker information and a value indicating a class according to the degree of hip joint deformity (e.g. KL classification).

[0126] The genetic information may include information on at least one of CALM1, CALM2, and GDF5, which are genes encoding calmodulin, and a gene encoding asporin, which are information on genes thought to be involved in the progression of HOA.

[0127] The history of joint disease may be, for example, acetabular dysplasia or developmental dysplasia of the hip. The diagnosis of developmental dysplasia of the hip may be performed by X-ray or ultrasound images. When X-ray or ultrasound diagnosis is difficult, the diagnosis may be performed by photographs or videos. A neural network may be used for the diagnosis. The neural network may be trained on photographs, videos, X-ray images, or ultrasound images of multiple patients with developmental dysplasia of the hip.

[0128] (First learning data 32) The first learning data 32 in this embodiment is time-series data on hip joint symptoms acquired at multiple points in time in the past from each of multiple patients with hip joint-related diseases. The first learning data 32 may also include the following (1) to (3) for each of the multiple patients.

[0129] (1) Patient image showing hip joint (2) symptom information regarding the onset or progression of a hip joint disease at the time the patient image was taken; (3) Patient-related information relevant to quality of life at the time the patient image was taken.

[0130] As an example, the first learning unit 27 may use a patient image at a certain point in time from the data included in the first learning data 32 as first learning input data 321, and may use the patient image, symptom information and patient-related information of the patient at a predetermined time after that point in time as first teacher data 322.

[0131] (First forecast information) The first prediction unit 23 outputs first prediction information in response to the input of the above-mentioned input data. The first prediction information may include at least one of the following pieces of information. A value indicating the classification (e.g., KL classification) according to the degree of hip joint deformity at the second time point of the subject - Subject's range of motion at the hip joint at time point 2 -The time and severity of hip pain in the subjects When the subject begins to have difficulty walking When the subject's hip joint will require invasive treatment.

[0132] The information output as the first prediction information may be any information that can be predicted by the first prediction model, and is not limited to these.

[0133] (Second training data 33) The second learning data 33 according to this embodiment is time-series data of medical images when an intervention is applied. The second learning data 33 may also include basic information of the patient to whom the intervention is applied.

[0134] The second prediction unit 25 outputs third prediction information indicating an estimated result of the effect of each intervention using the second prediction model. Here, the third prediction information output by the second prediction unit 25 is, for example, the degree of deformation of the subject's hip joint at the second time point when the intervention is applied, or the presence / absence and degree of pain, etc. In addition, the second prediction unit 25 may generate an image indicating the state of the subject's hip joint at the second time point when the intervention is applied, and output it as the third prediction information.

[0135] (Method of intervention that is expected to be effective) In this embodiment, the intervention method included in the intervention information 34 stored in the storage unit 3 is, for example, non-invasive treatment. For example, the intervention method may include weight restriction, thermotherapy, ultrasound therapy, exercise therapy, stimulation therapy, manual therapy, high frequency therapy, drug intake (drug administration), supplement intake, wearing of an orthosis, patient education, etc. The orthosis may include, for example, a walking aid, a supporter, etc.

[0136] The supplement may include, for example, chondroitin or glucosamine. The drug may include, for example, nonsteroidal anti-inflammatory drugs, acetaminophen, COX-2 inhibitors, and sodium chondroitin sulfate. The intervention method may include intra-articular injection, surgical therapy, and invasive treatment. Intra-articular injection may include hyaluronic acid, steroids, or regenerative medicine products (PRP, MSC, or Muse cells, etc.). Surgical therapy may include arthroscopic surgery, rotational acetabular osteotomy, or total hip replacement.

[0137] The second prediction unit 25 selects at least one of the intervention methods and estimates the effect of the intervention. The second prediction unit 25 may also estimate the effect of applying a combination of a plurality of the intervention methods.

[0138] [Embodiment 4] Hereinafter, a case will be described in which the prediction device 1 is applied to predict information related to idiopathic osteonecrosis of femoral head (hereinafter, referred to as "ION"). For convenience of explanation, the same reference numerals are attached to members having the same functions as the members described in the above embodiment, and the explanations thereof will not be repeated.

[0139] ION is an intractable disease in which the femoral head undergoes avascular necrosis and collapses, resulting in loss of hip joint function. ION accounts for 7% of all diseases requiring total hip replacement surgery. In Japan, ION occurs in approximately 1 in 5,000 people. Systemic steroid administration, drinking habits, and smoking habits are suspected to be related factors to the onset of ION.

[0140] ION often develops in people aged 30 to 40 years, resulting in a decline in the ability to work. In addition, once ION develops, it requires long-term treatment. Patients with ION who progress to femoral head collapse will need to undergo total hip replacement surgery.

[0141] A prediction device according to one embodiment of the present disclosure can predict the period until the onset of ION.

[0142] Furthermore, a prediction device according to one embodiment of the present disclosure can predict the progression of ION symptoms and the period until the femoral head collapses.

[0143] In this case, the input data input to the input layer 231 is a medical image at a first time point. The first time point here is a time point at which the current condition of the subject's hip joint can be understood. For example, the first time point is a time point at which the subject visits a hospital. In addition to the medical image, basic information on the subject at the first time point may be input to the input layer 231.

[0144] In this case, the medical image may be, for example, at least one of an X-ray photograph (X-ray image), a CT photograph (CT image), an MRI photograph (MRI image), and an ultrasound image taken around the subject's hip joint at a first time point.

[0145] In addition, the basic information includes at least one of the subject's sex, age, height, weight, species (e.g., race), occupation, occupational history, lifestyle habits, exercise habits, exercise history, smoking habits, drinking habits, whether or not the subject is taking steroids, information on bone morphology, genetic information, information on pain and stiffness occurring in the hip joint (e.g., the presence or absence or degree of pain / stiffness), and biomarker information at the first time point.

[0146] (First learning data 32) The first learning data 32 in this embodiment is time-series data on symptoms of ION acquired at multiple points in time in the past from each of multiple patients diagnosed with ION. The first learning data 32 also includes information in which a patient image showing a hip joint, symptom information on the onset of a hip joint disease or the progression of symptoms at the time the patient image was captured, and patient-related information related to QOL at the time the patient image was captured are associated with each other for each of the multiple patients. The first learning data 32 may also include information indicating the time when femoral head collapse occurred.

[0147] As an example, the first learning unit 27 may use a patient image at a certain point in time from the data included in the first learning data 32 as first learning input data 321, and may use the patient image, symptom information and patient-related information of the patient at a predetermined time after that point in time as first teacher data 322.

[0148] (First forecast information) In response to the input of the above-mentioned input data, the first prediction unit 23 outputs first prediction information. The first prediction information may include at least one of the following pieces of information. The subject's degree of hip pain at the second time point - Subject's range of motion at the hip joint at time point 2 -Time when the subject's hip pain begins When the subject's femoral head collapses When the subject begins to have difficulty walking When the subject's hip joint will require invasive treatment.

[0149] For example, the first prediction unit 23 outputs the degree of hip joint pain of the subject five years from now as the first prediction information.

[0150] The information output as the first prediction information may be any information that can be predicted by the first prediction model, and is not limited to these.

[0151] (Second training data 33) The second learning data 33 according to this embodiment is time-series data of medical images when an intervention is applied. The second learning data 33 may also include basic information of the patient to whom the intervention is applied.

[0152] The second prediction unit 25 outputs third prediction information indicating an estimated result of the effect of each intervention using the second prediction model. Here, the third prediction information output by the second prediction unit 25 is, for example, the presence / absence and degree of pain or the presence / absence and degree of stiffness that will occur in the hip joint of the subject at the second time point when the intervention is applied. The second prediction unit 25 may also generate an image showing the state of the hip joint of the subject at the second time point when the intervention is applied, and output it as the third prediction information. The second prediction information may also output the time when femur collapse will occur when the intervention is applied as the third prediction information.

[0153] (Method of intervention that is expected to be effective) In this embodiment, the intervention method included in the intervention information 34 stored in the storage unit 3 is, for example, non-invasive treatment. The intervention method may include weight restriction, thermotherapy, ultrasound therapy, drug intake, wearing of an orthosis, etc. The orthosis may include, for example, a walking aid, a supporter, etc.

[0154] The drug may be, for example, a statin antihyperlipidemic drug.

[0155] The intervention may include intra-articular injections or surgical treatments and invasive procedures such as osteotomy, hip replacement, etc.

[0156] The second prediction unit 25 selects at least one of the intervention methods and estimates the effect of the intervention. The second prediction unit 25 may also estimate the effect of applying a combination of a plurality of the intervention methods.

[0157] [Software implementation example] The control blocks of the prediction device 1 (particularly the image acquisition unit 21, the basic information acquisition unit 22, the first prediction unit 23, the output control unit 24, the second prediction unit 25, the first learning unit 27, and the second learning unit 28) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or by software.

[0158] In the latter case, the prediction device 1 includes a computer that executes instructions of a program, which is software that realizes each function. The computer includes, for example, one or more processors, and a computer-readable recording medium that stores the program. The processor in the computer reads the program from the recording medium and executes it, thereby achieving the object of the present disclosure. The processor may be, for example, a CPU (Central Processing Unit). The recording medium may be a "non-transient tangible medium," such as a ROM (Read Only Memory), as well as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like. The device may further include a RAM (Random Access Memory) that expands the program. The program may be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) that can transmit the program. One aspect of the present disclosure may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0159] The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to the above-mentioned embodiments. In other words, the invention according to the present disclosure can be modified in various ways within the scope of the present disclosure, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments are also included in the technical scope of the invention according to the present disclosure. In other words, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. It should also be noted that these modifications or corrections are included in the scope of the present disclosure.

[0160] [Summary 1] A prediction device according to aspect 1 of the present disclosure includes a first prediction unit having a first prediction model that outputs first prediction information that predicts a disease of a subject at a second time point that is a predetermined period of time from a first time point based on a medical image of the subject at the first time point, and a second prediction unit having a second prediction model that outputs third prediction information that indicates a method of intervention for the subject and the effect of the intervention based on the first prediction information.

[0161] In a prediction device according to aspect 2 of the present disclosure, in aspect 1, the first prediction model may be generated by machine learning using first learning data including patient images showing the patient's bones, joints, and soft tissues, and first teacher data including symptom information of the patient at a predetermined time after the patient images were captured.

[0162] In a prediction device according to aspect 3 of the present disclosure, in aspect 1 or 2, the first prediction model may be generated by machine learning using first learning data including patient images showing the patient's bones, joints, and soft tissues, and first teacher data including symptom information of the patient at the time the patient images were captured.

[0163] In a prediction device according to aspect 4 of the present disclosure, in any of aspects 1 to 3, the second prediction model may be generated by machine learning using learning data including at least any of information indicating the time when the intervention was applied to the patient to whom the intervention was applied, a patient image showing the bones, joints, and soft tissues of the patient, and symptom information at the time the patient image was captured, and a patient image showing the patient at a predetermined time after the patient image was captured, and teacher data including symptom information.

[0164] In the prediction device according to aspect 5 of the present disclosure, in any one of aspects 2 to 4, the symptom information may be at least any one of information regarding the onset time of a joint disease of the patient and information regarding the progression of the symptom.

[0165] A prediction device according to aspect 6 of the present disclosure, in any one of aspects 1 to 5, may be arranged so that the medical image includes identification information unique to the medical facility examining the subject, and identification information unique to the subject.

[0166] The prediction device of aspect 7 of the present disclosure, in any of aspects 1 to 6, may further include a basic information acquisition unit that acquires basic information of the target at the first time point, and the first prediction model may be capable of estimating the first prediction information from the basic information in addition to the medical image.

[0167] The prediction device according to an eighth aspect of the present disclosure is in the seventh aspect, wherein the basic information includes at least one of height, weight, age, and sex of the subject.

[0168] A prediction device according to aspect 9 of the present disclosure, in any of aspects 1 to 8, may be such that the first prediction information includes information including a degree of onset or progression of symptoms of a disease related to the target joint up to the second time point.

[0169] A prediction device according to aspect 10 of the present disclosure, in any one of aspects 1 to 9, may have as the first prediction model a neural network trained using patient information, which is time-series data regarding symptoms of the patient's joints obtained from the patient at multiple points in time in the past.

[0170] In a prediction device according to aspect 11 of the present disclosure, in any of aspects 1 to 10, the first prediction unit may be further capable of outputting second prediction information related to the QOL of the subject at the second time point from the medical image, and the first prediction model may be capable of estimating the second prediction information from the medical image.

[0171] The prediction device according to Aspect 12 of the present disclosure, in any one of Aspects 1 to 11, may be arranged such that the third prediction information includes information relating to a timing at which the intervention should be applied.

[0172] A prediction device according to aspect 13 of the present disclosure, in any one of aspects 1 to 12, may have as the second prediction model a neural network trained using effect information, which is time-series data on joint symptoms of a patient to whom the intervention has been applied in the past, obtained at multiple points in time in the past.

[0173] A prediction device according to aspect 14 of the present disclosure is configured in such a way that, in aspect 13, the effect information may include information indicating the time when the intervention was applied to a patient to whom the intervention was applied, a patient image showing the patient's joint, and symptom information regarding the onset or progression of symptoms in the patient's joint at the time the patient image was taken.

[0174] A prediction device according to aspect 15 of the present disclosure, in any of aspects 1 to 14, may further include a third prediction unit that uses a third prediction model generated based on clinical performance information, which is time-series data regarding the intervention, obtained at multiple time points in the past from patients to whom the intervention has been applied in the past, to output, from the first prediction information and the intervention, fourth prediction information regarding clinical performance after the intervention has been applied to the target at a third time point a predetermined period of time has elapsed since the intervention was applied.

[0175] The prediction device according to a sixteenth aspect of the present disclosure is in any one of the first to fifteenth aspects, wherein the disease is a disease specific to a joint.

[0176] A prediction system according to aspect 17 of the present disclosure includes a prediction device according to any one of aspects 1 to 16, and a terminal device communicatively connected to the prediction device, the terminal device presenting the first prediction information.

[0177] A control method according to aspect 18 of the present disclosure is a control method for a computer that causes a computer to execute a first prediction step of outputting first prediction information that predicts a disease of the subject at a second time point that is a predetermined period of time from a first time point based on a medical image of the subject at the first time point, and a second prediction step of outputting third prediction information that indicates a method of intervention in the subject and the effect of the intervention from the first prediction information, wherein the computer has a first prediction model capable of estimating the first prediction information from the medical image, and a second prediction model capable of estimating the third prediction information from the first prediction information.

[0178] A control program according to aspect 19 of the present disclosure is a control program for causing a computer to function as a prediction device according to any one of aspects 1 to 16, and is a control program for causing a computer to function as the first prediction unit and the second prediction unit.

[0179] [Summary 2]

[0180] A prediction device according to one embodiment of the present disclosure includes an image acquisition unit that acquires a medical image of a target joint (including bone, cartilage and soft tissue) at a first time point, and a first prediction unit that outputs first prediction information about the joint from the medical image at a second time point a predetermined period of time has elapsed from the first time point, and the first prediction unit has a first prediction model capable of estimating the first prediction information from the medical image.

[0181] Furthermore, a control method according to one aspect of the present disclosure is a control method for a computer that causes a computer to execute an image acquisition step of acquiring a medical image of a target joint at a first time point, and a prediction step of outputting, from the medical image, first prediction information regarding the joint at a second time point a predetermined period of time has elapsed from the first time point, wherein the computer has a first prediction model capable of estimating the first prediction information from the medical image. [Explanation of symbols]

[0182] 1 Prediction device 21 Image acquisition unit 22 Basic information acquisition department 23 First Prediction Section 24 Output control section 25 Second Prediction Section 26 Third Prediction Section 32 First learning data (patient information) 33 Second learning data (effectiveness information) 100a, 100b Prediction System S21 Image acquisition step S23 Prediction step

Claims

1. a first prediction unit having a first prediction model that outputs first prediction information that predicts a disease related to a target joint at a second time point that is a predetermined period of time after the first time point, based on a medical image of the target joint at the first time point; a second prediction unit having a second prediction model that outputs third prediction information indicating a method of intervention for the subject and an effect of the intervention from the first prediction information; Equipped with said intervention being an invasive or non-invasive treatment associated with said joint condition, the intervention effect is the treatment effect, The first prediction model is The system is generated by machine learning using first learning data including a patient image showing bones, joints, and soft tissues of the patient, and first teacher data including symptom information of the patient at a time when a predetermined period has elapsed since the patient image was captured; The third prediction information includes information indicating at least one of a time when the intervention should be applied and a time when symptoms of the disease will occur if the intervention is applied. Prediction device.

2. The first prediction model is generated by machine learning using first learning data including a patient image showing bones, joints, and soft tissues of the patient, and first teacher data including symptom information of the patient at the time the patient image was captured; The prediction device according to claim 1 .

3. The second prediction model is generated by machine learning using learning data including at least one of information indicating the time when the intervention was applied to the patient to whom the intervention was applied, a patient image showing the bones, joints, and soft tissues of the patient, and symptom information at the time when the patient image was captured, and a patient image showing the patient at a time when a predetermined period has elapsed since the time when the patient image was captured, and teacher data including symptom information; The prediction device according to claim 1 .

4. The symptom information is at least one of information regarding the onset time of a joint disease of the patient and information regarding the progression of the symptom; The prediction device according to claim 1 .

5. The medical image includes identification information unique to a medical facility examining the subject and identification information unique to the subject. The prediction device according to claim 1 .

6. A basic information acquisition unit that acquires basic information of the target at the first time point, The first prediction model is capable of estimating the first prediction information from the basic information in addition to the medical image. The prediction device according to claim 1 .

7. The prediction device according to claim 6 , wherein the basic information includes at least one of height, weight, age, and sex of the subject.

8. The first prediction information is information including a degree of onset or progression of a symptom of a disease related to the joint of the subject up to the second time point. The prediction device according to claim 1 .

9. The first prediction unit has, as the first prediction model, a neural network trained using patient information, which is time-series data on symptoms of the patient's joints acquired from the patient at multiple points in time in the past. The prediction device according to claim 1 .

10. The first prediction unit is further capable of outputting second prediction information related to a QOL of the subject at the second time point from the medical image, The first prediction model is capable of estimating the second prediction information from the medical image. The prediction device according to claim 1 .

11. The third predictive information includes information regarding when the intervention should be applied. The prediction device according to claim 1 .

12. The second prediction unit has, as the second prediction model, a neural network trained using effect information, which is time-series data on symptoms of the joints of the patient, acquired at multiple time points in the past from a patient to whom the intervention was applied in the past. The prediction device according to claim 1 .

13. The effect information includes information indicating the time when the intervention was applied to the patient to whom the intervention was applied, a patient image showing the joint of the patient, and symptom information regarding the onset or progression of symptoms of the joint of the patient at the time the patient image was captured. The prediction device according to claim 12.

14. and a third prediction unit that uses a third prediction model generated based on clinical outcome information, which is time-series data related to the intervention, acquired at multiple time points in the past from a patient to whom the intervention was applied in the past, to output, from the first prediction information and the intervention, fourth prediction information related to clinical outcomes of the subject after the intervention is applied at a third time point when a predetermined period of time has elapsed since the intervention was applied. The prediction device according to claim 1 .

15. The disease is a joint-specific disease, The prediction device according to claim 1 .

16. The prediction device according to claim 1 ; a terminal device communicably connected to the prediction device, the terminal device presenting the first prediction information; Prediction system.

17. A control method for a computer that causes a computer to execute a first prediction step of outputting first prediction information that predicts a disease related to a target joint at a second time point that is a predetermined period of time after a first time point from a medical image showing the target joint at the first time point, and a second prediction step of outputting third prediction information that indicates a method of intervention for the target and an effect of the intervention from the first prediction information, the computer has a first prediction model capable of estimating the first prediction information from the medical image, and a second prediction model capable of estimating the third prediction information from the first prediction information; said intervention being an invasive or non-invasive treatment associated with said joint condition, the intervention effect is the treatment effect, The first prediction model is The system is generated by machine learning using first learning data including a patient image showing bones, joints, and soft tissues of the patient, and first teacher data including symptom information of the patient at a time when a predetermined period has elapsed since the patient image was captured; The third prediction information includes information indicating at least one of a time when the intervention should be applied and a time when symptoms of the disease will occur if the intervention is applied. Control methods.

18. A control program for causing a computer to function as the prediction device according to claim 1 , the control program causing a computer to function as the first prediction unit and the second prediction unit.

Images