Estimation device, estimation method, and system

Abstract

The invention provides an estimation device, an estimation method and a system. The estimation device includes an input unit and an approximator. The input unit inputs input information having an image in which a bone has been captured. The approximator is capable of estimating an estimation result relating to the bone mineral density of the bone on the basis of the input information input to the input unit. The approximator has a parameter for obtaining a learning completion of an estimation result relating to the bone mineral density of the bone from input information.

Description

[0001] This application is a divisional application of a patent application for invention with an application date of September 10, 2019, an application number of 201980058486.5, and an invention title of "Presumption Device, Presumption System, and Computer-Readable Non-Transitory Recording Medium Recording a Presumption Program".

[0002] Cross-reference to related applications

[0003] This application claims the priority of Japanese Patent Application No. 2018-168502 (filed on September 10, 2018) and Japanese National Application No. 2018-220401 (filed on November 26, 2018), and the entire disclosure of that application is incorporated herein by reference for reference purposes. Technical field

[0004] This disclosure relates to the presumption of bone density. Background art

[0005] In Patent Document 1, a technique for determining osteoporosis is described. In Patent Document 2, a technique for presuming the strength of bones is described.

[0006] Prior art documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2008-36068

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2002-523204 Summary of the invention

[0010] A presumption device, a presumption system, and a presumption program are disclosed. In one embodiment, the presumption device includes: an input unit that inputs input information having an image of a bone captured; an approximator that can presume a presumption result associated with the bone density of the bone based on the input information input to the input unit; and an output unit that outputs the presumption result presumed by the approximator, and the approximator has parameters that have completed learning for obtaining a presumption result associated with the bone density of the bone based on the input information.

[0011] In addition, in one embodiment, the presumption system includes: an input unit that inputs input information having an image of a bone captured; and an approximator that has parameters that have completed learning for obtaining a presumption result associated with the bone density of the bone based on the input information, and can presume a presumption result associated with the bone density of the bone based on the input information input to the input unit, and when the input information is input to the input unit, the approximator performs arithmetic processing on the input information.

[0012] In addition, in one embodiment, the estimation program is an estimation program for causing the device to function as a neural network that performs operations based on parameters that have been learned and outputs an estimated value of the bone density of the bone captured in the image, and the learned parameters are used to obtain an estimation result associated with the bone density of the bone based on input information having an image of the bone captured therein. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. is an example showing the structure of a computer device (estimation device).

[0014] Figure 2 FIG. is a diagram for explaining the operation of the estimation device.

[0015] Figure 3 FIG. is an example showing the structure of a neural network.

[0016] Figure 4 FIG. is an example showing the state in which learning image data and reference bone density are associated with each other.

[0017] Figure 5 FIG. is a diagram for explaining the operation of the estimation device.

[0018] Figure 6 FIG. is an example showing an estimation system.

[0019] Figure 7 FIG. is an example showing the structure of an estimation device.

[0020] Figure 8 FIG. is an example showing the structure of an estimation device.

[0021] Figure 9 FIG. is an example showing the structure of an estimation device.

[0022] Figure 10 FIG. is an example showing the structure of an estimation device.

[0023] Figure 11 FIG. is an example showing the structure of an estimation device.

[0024] Figure 12 FIG. is an example showing the structure of an estimation device.

[0025] Figure 13 FIG. is a conceptual diagram schematically showing the structure of the estimation system.

[0026] Figure 14 FIG. is a conceptual diagram schematically showing the structure of a part of the estimation system.

[0027] Figure 15 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0028] Figure 16 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0029] Figure 17 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0030] Figure 18 It is a conceptual diagram schematically showing the structure of a part of another embodiment of a presumption system.

[0031] Symbol Explanation

[0032] 1: Computer device (presumption device);

[0033] 20: Storage unit;

[0034] 100: Control program;

[0035] 110, 910, 960: Learned parameters;

[0036] 120: Data for presumption;

[0037] 130: Data for learning;

[0038] 140: Training data;

[0039] 200: Neural network;

[0040] 210: Input layer;

[0041] 230: Output layer;

[0042] 280, 832: Approximator;

[0043] 500: Processing device;

[0044] 600: Bone density presumption system;

[0045] 801: Presumption system;

[0046] 802: Terminal device;

[0047] 803: Presumption device;

[0048] 831: Input unit;

[0049] 833: Output unit;

[0050] 834: Control unit;

[0051] 835: Storage unit;

[0052] 900, 8322: Second neural network;

[0053] 930: Determination unit;

[0054] 950: Third neural network;

[0055] 980: Fracture prediction unit;

[0056] 8321: First neural network;

[0057] O: Estimation result;

[0058] I: Input information;

[0059] E: Encoding unit;

[0060] D: Decoding unit;

[0061] C: Transformation unit. Detailed implementation

[0062] Embodiment 1.

[0063] Figure 1 is a block diagram showing an example of the structure of the computer device 1 according to Embodiment 1. The computer device 1 functions as an estimation device for estimating bone density. Hereinafter, the computer device 1 may sometimes be referred to as the "estimation device 1".

[0064] As Figure 1 shown, the estimation device 1 includes, for example, a control unit 10, a storage unit 20, a communication unit 30, a display unit 40, and an input unit 50. The control unit 10, the storage unit 20, the communication unit 30, the display unit 40, and the input unit 50 are electrically connected to each other through a bus 60, for example.

[0065] The control unit 10 can manage the operation of the estimation device 1 uniformly by controlling other structural elements of the estimation device 1. The control unit 10 is also referred to as a control device or a control circuit. As will be described in further detail below, the control unit 10 includes at least one processor in order to provide control and processing capabilities for executing various functions.

[0066] According to various embodiments, at least one processor may be implemented as a single integrated circuit (IC) or as multiple communicatively connected integrated circuits (ICs) and / or discrete circuits. At least one processor can be implemented according to various known techniques.

[0067] In one embodiment, a processor includes, for example, one or more circuits or units configured to execute one or more data calculation programs or processes by executing instructions stored in an associated memory. In another embodiment, the processor may also be firmware (e.g., discrete logic elements) configured to execute one or more data calculation programs or processes.

[0068] According to various embodiments, the processor may also include one or more processors, controllers, microprocessors, microcontrollers, application-specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, field programmable logic devices, or any combination of these devices or structures, or a combination of other known devices and structures, and perform the functions described below. In this example, the control unit 10 includes, for example, a CPU (Central Processing Unit).

[0069] The storage unit 20 includes non-transitory storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory) that can be read by the CPU of the control unit 10. A control program 100 for controlling the estimation device 1 is stored in the storage unit 20. Various functions of the control unit 10 are realized by the CPU of the control unit 10 executing the control program 100 in the storage unit 20. It can also be said that the control program 100 is a bone density estimation program for causing the computer device 1 to function as an estimation device. In this example, by the control unit 10 executing the control program 100 in the storage unit 20, as Figure 2 shown, an approximator 280 capable of outputting an estimated value 300 of bone density is formed in the control unit 10. The approximator 280 includes, for example, a neural network 200. The control program 100 can also be said to be a program for causing the computer device 1 to function as the neural network 200. Thereafter, the estimated value of bone density is sometimes referred to as the "bone density estimated value". In addition, a structural example of the neural network 200 will be described in detail later.

[0070] In addition to the control program 100, learned parameters 110 related to the neural network 200, estimation data 120 (hereinafter, also referred to as "input information"), learning data 130, and training data 140 are also stored in the storage unit 20. The learning data 130 and the training data 140 are data used when the neural network 200 is learning. The learned parameters 110 and the estimation data 120 are data used by the learned neural network 200 when estimating bone density.

[0071] The learning data 130 is the data input to the input layer 210 of the neural network 200 during the learning of the neural network 200. The learning data 130 is also referred to as learning data. The training data 140 is the data representing the correct value of bone density. The training data 140 is compared with the output data output from the output layer 230 of the neural network 200 during the learning of the neural network 200. The learning data 130 and the training data 140 are sometimes collectively referred to as learning data with training.

[0072] The data 120 for estimation is the data input to the input layer 210 of the neural network 200 when the bone density is estimated by the learned neural network 200. The learned parameters 110 are the learned parameters within the neural network 200. The learned parameters 110 can also be referred to as the parameters adjusted through the learning of the neural network 200. Among the learned parameters 110, there are included weighting coefficients representing the weights of the connections between artificial neurons. As shown in Figure 2 the learned neural network 200 performs operations based on the learned parameters 110 for the data 120 for estimation input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230.

[0073] In addition, the data input to the input layer 210 can be input to the input layer 210 either via the input unit 50 or directly to the input layer 210. In the case of directly inputting to the input layer 210, the input layer 210 can also be set as part or all of the input unit 50. Thereafter, the bone density estimation value 300 is sometimes referred to as the estimation result 300.

[0074] The communication unit 30 is connected to a communication network including the Internet or the like by wire or wirelessly. The communication unit 30 can communicate with other devices such as a cloud server and a network server through the communication network. The communication unit 30 can input the information received from the communication network to the control unit 10. In addition, the communication unit 30 can output the information received from the control unit 10 to the communication network.

[0075] The display unit 40 is, for example, a liquid crystal display or an organic EL display. The display unit 40 can display various information such as characters, symbols, and graphics by being controlled by the control unit 10.

[0076] The input unit 50 can accept input from the user for the estimation device 1. The input unit 50 includes, for example, a keyboard and a mouse. The input unit 50 can also include a touch screen capable of detecting the user's operation on the display surface of the display unit 40.

[0077] In addition, the structure of the estimation device 1 is not limited to the above examples. For example, the control unit 10 may also include multiple CPUs. In addition, the control unit 10 may also include at least one DSP. In addition, all functions of the control unit 10 or some of the functions of the control unit 10 may also be implemented by a hardware circuit that does not require software for the implementation of its functions. In addition, the storage unit 20 may also include a non-transitory storage medium readable by a computer other than a ROM and a RAM. For example, the storage unit 20 may also include a small hard disk drive and an SSD (Solid State Drive), etc. In addition, the storage unit 20 may also include a memory such as a USB (Universal Serial Bus) memory that can be plugged into and unplugged from the estimation device 1. Hereinafter, a memory that can be plugged into and unplugged from the estimation device 1 is sometimes referred to as a "plug-in memory".

[0078] <Example of the Structure of the Neural Network>

[0079] Figure 3 FIG. is an example showing the structure of the neural network 200. In this example, the neural network 200 is, for example, a convolutional neural network (CNN (Convolutional Neural Network)). As Figure 3 shown, the neural network 200 includes, for example, an input layer 210, a hidden layer 220, and an output layer 230. The hidden layer 220 is also referred to as an intermediate layer. The hidden layer 220 includes, for example, multiple convolutional layers 240, multiple pooling layers 250, and a fully connected layer 260. In the neural network 200, a fully connected layer 260 exists before the output layer 230. Moreover, in the neural network 200, the convolutional layer 240 and the pooling layer 250 are alternately arranged between the input layer 210 and the fully connected layer 260.

[0080] In addition, the structure of the neural network 200 is not limited to Figure 3 the example. For example, the neural network 200 may also include one convolutional layer 240 and one pooling layer 250 between the input layer 210 and the fully connected layer 260. In addition, the neural network 200 may also be a neural network other than a convolutional neural network.

[0081] <An Example of the Estimation Data, Learning Data, and Training Data>

[0082] The data 120 for estimation includes image data of a simple X-ray image of a bone that is the object of bone density estimation. The object of bone density estimation is, for example, a human. Therefore, it can be said that the data 120 for estimation includes image data of a simple X-ray image of a human bone. In the data 130 for learning, it includes image data of a plurality of simple X-ray images of a human bone. A simple X-ray image is also referred to as a two-dimensional image, a general X-ray image, or a Roentgen image. In addition, the object of bone density estimation can also be other than a human. For example, the object of bone density estimation can be an animal such as a dog, a cat, or a horse. Furthermore, the bone that is the object is mainly cortical bone and cancellous bone from a living body, but among the bones that are the object, it can also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine or the like.

[0083] After that, sometimes the image data included in the data 120 for estimation is referred to as "estimation-use image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 120 for estimation is referred to as "estimation-use simple X-ray image". In addition, sometimes the image data included in the data 130 for learning is referred to as "learning-use image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 130 for learning is referred to as "learning-use simple X-ray image". In the data 130 for learning, it includes a plurality of learning-use X-ray image data respectively representing a plurality of learning-use simple X-ray images.

[0084] As the imaging site for the simple X-ray image for estimation, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers, or jaw joint can be used. In other words, as the estimation data 120, the following can be used: the image data of the simple X-ray image obtained by irradiating the head with X-rays, the image data of the simple X-ray image obtained by irradiating the neck with X-rays, the image data of the simple X-ray image obtained by irradiating the chest with X-rays, the image data of the simple X-ray image obtained by irradiating the waist with X-rays, the image data of the simple X-ray image obtained by irradiating the hip joint with X-rays, the image data of the simple X-ray image obtained by irradiating the knee joint with X-rays, the image data of the simple X-ray image obtained by irradiating the ankle joint with X-rays, the image data of the simple X-ray image obtained by irradiating the foot with X-rays, the image data of the simple X-ray image obtained by irradiating the toes with X-rays, the image data of the simple X-ray image obtained by irradiating the shoulder joint with X-rays, the image data of the simple X-ray image obtained by irradiating the elbow joint with X-rays, the image data of the simple X-ray image obtained by irradiating the wrist joint with X-rays, the image data of the simple X-ray image obtained by irradiating the hand with X-rays, the image data of the simple X-ray image obtained by irradiating the fingers with X-rays, or the image data of the simple X-ray image obtained by irradiating the jaw joint with X-rays. Among the simple X-ray images obtained by irradiating the chest with X-rays, there are simple X-ray images showing the lungs and simple X-ray images showing the thoracic vertebrae. In addition, the types of the imaging sites for the simple X-ray image for estimation are not limited to this. Furthermore, the simple X-ray image for estimation can be a front image of the target site taken from the front or a side image of the target site taken from the side.

[0085] Among the imaging regions of the plurality of learning X-ray images respectively represented by the plurality of pieces of learning data 130, there is included at least one of, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers, and jaw joint. In other words, in the learning data 130, there is included at least one of the 15 types of image data, namely, the image data of a simple X-ray image obtained by irradiating the head with X-rays, the image data of a simple X-ray image obtained by irradiating the neck with X-rays, the image data of a simple X-ray image obtained by irradiating the chest with X-rays, the image data of a simple X-ray image obtained by irradiating the waist with X-rays, the image data of a simple X-ray image obtained by irradiating the hip joint with X-rays, the image data of a simple X-ray image obtained by irradiating the knee joint with X-rays, the image data of a simple X-ray image obtained by irradiating the ankle joint with X-rays, the image data of a simple X-ray image obtained by irradiating the foot with X-rays, the image data of a simple X-ray image obtained by irradiating the toes with X-rays, the image data of a simple X-ray image obtained by irradiating the shoulder joint with X-rays, the image data of a simple X-ray image obtained by irradiating the elbow joint with X-rays, the image data of a simple X-ray image obtained by irradiating the wrist joint with X-rays, the image data of a simple X-ray image obtained by irradiating the hand with X-rays, the image data of a simple X-ray image obtained by irradiating the fingers with X-rays, and the image data of a simple X-ray image obtained by irradiating the jaw joint with X-rays. In the learning data 130, there may be included image data of some of the 15 types of image data or all types of image data. In addition, the types of imaging regions of the learning simple X-ray images are not limited to this. Further, among the plurality of learning simple X-ray images, there may be included a front image or a side image. In addition, among the plurality of learning simple X-ray images, there may also be included both a front image and a side image of the same imaging region.

[0086] In the training data 140, for each of the plurality of learning image data included in the learning data 130, it includes the measured value of the bone density of a person having a bone photographed in the learning plain X-ray image represented by the learning image data. Among the plurality of measured values of bone density included in the training data 140, for example, it includes at least one of the measured value of bone density measured by irradiating X-rays to the lumbar spine, the bone density measured by irradiating X-rays to the proximal part of the femur, the bone density measured by irradiating X-rays to the radius, the bone density measured by irradiating X-rays to the metacarpal bone, the bone density measured by applying ultrasonic waves to the wrist, and the bone density measured by applying ultrasonic waves to the heel. After that, sometimes the measured value of bone density included in the training data 140 is referred to as the "reference bone density".

[0087] Here, as a method for measuring bone density, the DEXA (dual-energy X-ray absorptiometry) method is known. In a DEXA device that uses the DEXA method to measure bone density, when measuring the bone density of the lumbar spine, X-rays (specifically, two types of X-rays) are irradiated to the lumbar spine from its front. In addition, in a DEXA device, when measuring the bone density of the proximal part of the femur, X-rays are irradiated to the proximal part of the femur from its front.

[0088] In the training data 140, it may include the bone density of the lumbar spine measured by a DEXA device, or may include the bone density of the proximal part of the femur measured by a DEXA device. In addition, in the training data 140, it may also include the bone density measured by irradiating X-rays to the target part from its side. For example, in the training data 140, it may also include the bone density measured by irradiating X-rays to the lumbar spine from its side.

[0089] In addition, as another method for measuring bone density, the ultrasonic wave method is known. In a device that uses the ultrasonic wave method to measure bone density, for example, ultrasonic waves are applied to the wrist to measure the bone density of the wrist, and ultrasonic waves are applied to the heel to measure the bone density of the heel. In the training data 140, it may also include the bone density measured by the ultrasonic wave method.

[0090] In the plurality of learning plain X-ray images represented by the plurality of learning image data included in the learning data 130, bones of different people are photographed. And, as Figure 4As shown, in the storage unit 20, for each of the multiple learning image data included in the learning data 130, a correspondence is established with the reference bone density of a person having bones photographed in the learning simple X-ray image represented by the learning image data. It can also be said that for each of the multiple learning simple X-ray images used in the learning of the neural network 200, a correspondence is established with the reference bone density of a person having bones photographed in the learning simple X-ray image. The reference bone density corresponding to the learning image data is the bone density measured for the same person as the person having bones photographed in the learning simple X-ray image represented by the learning image data at a time almost the same as the time when the learning simple X-ray image was photographed.

[0091] In the part photographed in the learning simple X-ray image represented by the learning image data (in other words, the photographing part of the learning simple X-ray image), the part (i.e., the bone) for which the reference bone density corresponding to the learning image data is measured may or may not be included. In other words, in the part photographed in the learning simple X-ray image, the part for which the reference bone density corresponding to the learning simple X-ray image is measured may or may not be included. As an example of the former, consider the case where a correspondence is established between the learning image data representing the learning simple X-ray image of the waist being photographed and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the hip joint being photographed and the reference bone density of the proximal part of the femur. On the other hand, as an example of the latter, consider the case where a correspondence is established between the learning image data of the chest being photographed and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the knee joint being photographed and the reference bone density of the heel.

[0092] In addition, the orientation of the part imaged in the learning simple X-ray image represented by the image data for learning, and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning image data may be the same or different. In other words, the orientation of the part imaged in the learning simple X-ray image, and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning simple X-ray image may be the same or different. As an example of the former, consider the case of establishing correspondence between the learning image data representing a simple X-ray image of the chest taken from the front (hereinafter sometimes referred to as "front chest simple X-ray image"), and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing correspondence between the learning image data representing a simple X-ray image of the waist taken from the front (hereinafter sometimes referred to as "front waist simple X-ray image"), and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front. On the other hand, as an example of the latter, consider the case of establishing correspondence between the learning image data representing a simple X-ray image of the waist taken from the side (hereinafter sometimes referred to as "side waist simple X-ray image"), and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing correspondence between the learning image data representing a simple X-ray image of the knee joint taken from the side (hereinafter sometimes referred to as "side knee simple X-ray image"), and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front.

[0093] In addition, among the multiple learning simple X-ray images respectively represented by the multiple learning image data included in the learning data 130, there may be included a simple X-ray image of a part of the same type as the part imaged in the estimation simple X-ray image, or a simple X-ray image of a part of a different type from the part imaged in the estimation simple X-ray image. As an example of the former, consider the case where the estimation simple X-ray image is a front chest simple X-ray image, and a front chest simple X-ray image is included among the multiple learning simple X-ray images. As another example, consider the case where the estimation simple X-ray image is a simple X-ray image of the knee joint taken from the front (hereinafter sometimes referred to as "front knee simple X-ray image"), and a side knee simple X-ray image is included among the multiple learning simple X-ray images. On the other hand, as an example of the latter, consider the case where the estimation simple X-ray image is a front waist simple X-ray image, and a front chest simple X-ray image is included among the multiple learning simple X-ray images. As another example, consider the case where the estimation simple X-ray image is a side waist simple X-ray image, and a front knee simple X-ray image is included among the multiple learning simple X-ray images.

[0094] In addition, among the multiple learning plain X-ray images, there may be included plain X-ray images of parts taken in the same orientation as the target plain X-ray image, or plain X-ray images of parts taken in an orientation different from that of the target plain X-ray image. As an example of the former, consider the case where the target plain X-ray image is a lumbar anterior plain X-ray image and a lumbar anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the target plain X-ray image is a knee anterior plain X-ray image and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images. On the other hand, as an example of the latter, consider the case where the target plain X-ray image is a knee lateral plain X-ray image and a knee anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the target plain X-ray image is a lumbar lateral plain X-ray image and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images.

[0095] In addition, in the training data 140, there may be included reference bone densities measured from parts (bones) included in the part imaged in the target plain X-ray image, or reference bone densities measured from parts (bones) not included in the part imaged in the target plain X-ray image. As an example of the former, consider the case where the target plain X-ray image is a lumbar anterior plain X-ray image and the reference bone density of the lumbar vertebra is included in the training data 140. On the other hand, as an example of the latter, consider the case where the target plain X-ray image is a chest anterior plain X-ray image and the reference bone density of the head of the metacarpal bone is included in the training data 140.

[0096] In addition, in the training data 140, there may be included reference bone densities measured by irradiating the target part with X-rays in the same orientation as the orientation of the part imaged in the target plain X-ray image, or reference bone densities measured by irradiating the target part with X-rays in an orientation different from the orientation of the part imaged in the target plain X-ray image. As an example of the former, consider the case where the target plain X-ray image is a lumbar anterior plain X-ray image and the reference bone density measured by irradiating the lumbar vertebra with X-rays from its anterior side is included in the training data 140. On the other hand, as an example of the latter, consider the case where the target plain X-ray image is a lumbar lateral plain X-ray image and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its anterior side is included in the training data 140.

[0097] In this example, image data representing the gray scale of a simple X-ray image obtained by a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device) is compressed, and the image data obtained by reducing the number of gray scales is used as learning image data and estimation image data. For example, when the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is more than (1024×640) and the number of bits of the pixel data is 16 bits. In this case, the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is compressed to, for example, (256×256), (1024×512), or (1024×640), and the number of bits of the pixel data is reduced to 8 bits, and then used as learning image data and estimation image data. In this case, each of the learning simple X-ray image and the estimation simple X-ray image includes (256×256), (1024×512), or (1024×640) pixels, and the value of the pixel is expressed in 8 bits.

[0098] For the learning image data and the estimation image data, they can be generated by the control unit 10 of the estimation device 1 based on the image data obtained by the simple X-ray imaging device, or can be generated by a device other than the estimation device 1 based on the image data obtained by the simple X-ray imaging device. In the former case, for the image data obtained by the simple X-ray imaging device, it can be received by the communication unit 30 through a communication network, or can be stored in the pluggable memory included in the storage unit 20. In the latter case, it can also be that the communication unit 30 receives the learning image data and the estimation image data from other devices through the communication network, and the control unit 10 stores the learning image data and the estimation image data received by the communication unit 30 in the storage unit 20. Or, the learning image data and the estimation image data generated by other devices can be stored in the pluggable memory included in the storage unit 20. In addition, for the training data 140, it can also be that the communication unit 30 receives it through the communication network, and the control unit 10 stores the training data 140 received by the communication unit 30 in the storage unit 20. Or, the training data 140 can be stored in the pluggable memory included in the storage unit 20. In addition, the number and number of bits of the pixel data of the learning image data and the estimation image data are not limited to the above.

[0099] <Example of learning of neural network>

[0100] Figure 5 This is a diagram for explaining an example of the learning of the neural network 200. When the control unit 10 performs the learning of the neural network 200, as Figure 5As shown, learning data 130 is input to the input layer 210 of the neural network 200. Then, the control unit 10 adjusts the variable parameter 110a in the neural network 200 so that the error of the output data 400 output from the output layer 230 of the neural network 200 with respect to the training data 140 is small. More specifically, the control unit 10 inputs each piece of learning image data in the storage unit 20 to the input layer 210. When inputting the learning image data to the input layer 210, the control unit 10 inputs the multiple pixel data constituting the learning image data to the multiple artificial neurons constituting the input layer 210 respectively. Then, when the learning image data is input to the input layer 210, the control unit 10 adjusts the parameter 110a so that the error of the output data 400 output from the output layer 230 with respect to the reference bone density corresponding to the learning image data is small. As a method for adjusting the parameter 110a, for example, the error backpropagation method can be adopted. The adjusted parameter 110a becomes the learned parameter 110 and is stored in the storage unit 20. In the parameter 110a, for example, parameters used in the hidden layer 220 are included. Specifically, the parameter 110a includes the filter coefficients used in the convolutional layer 240 and the weighting coefficients used in the fully connected layer 260. In addition, the method for adjusting the parameter 110a, in other words, the learning method of the parameter 110a is not limited to this.

[0101] In this way, in the storage unit 20, the learned parameter 110 is stored, which has learned the relationship between the learning data 130 of the image data including multiple learning simple X-ray images and the measured values of the bone density as the training data 140 using the neural network 200.

[0102] In the above example, the estimation device 1 performs the learning of the neural network 200, but other devices can also perform the learning of the neural network 200. In this case, the learned parameter 110 generated by other devices is stored in the storage unit 20 of the estimation device 1. In addition, it becomes unnecessary to store the learning data 130 and the training data 140 in the storage unit 20. For the learned parameter 110 generated by other devices, it can also be received by the communication unit 30 through the communication network, and the control unit 10 stores the learned parameter 110 received by the communication unit 30 in the storage unit 20. Alternatively, the learned parameter 110 generated by other devices can be stored in the pluggable memory included in the storage unit 20.

[0103] In the neural network 200 learned as above, the image data of multiple learning simple X-ray images is input to the input layer 210 as the learning data 130, and it includes the learned parameter 110a learned using the reference bone density as the training data 140. As described above Figure 2As shown, the neural network 200 performs operations based on the learned parameters 110a on the estimation data 120 input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230. When the estimation image data as the estimation data 120 is input to the input layer 210, the multiple pixel data constituting the estimation image data are respectively input to the multiple artificial neurons constituting the input layer 210. Then, the convolutional layer 240 performs operations using the filter coefficients included in the learned parameters 110a, and the fully connected layer 260 performs operations using the weighting coefficients included in the learned parameters 110a.

[0104] For example, when the estimation image data representing the frontal chest plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the bones of the chest photographed in the frontal chest plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the frontal lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the lumbar vertebrae included in the lumbar region photographed in the frontal lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the lateral lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the lumbar vertebrae included in the lumbar region photographed in the lateral lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the frontal knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the bones of the knee joint photographed in the frontal knee plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the lateral knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the bones of the knee joint photographed in the lateral knee plain X-ray image represented by the estimation image data is output from the output layer 230.

[0105] The estimation value 300 output from the output layer 230 can be represented by at least one of the bone mineral density per unit area (g / cm 2 ), the bone mineral density per unit volume (g / cm 3 ), YAM, T-score, and Z-score. YAM is an abbreviation for "Young Adult Mean" and is sometimes referred to as the young adult average percentage. For example, from the output layer 230, it is possible to output the bone mineral density per unit area (g / cm 2The estimated value 300 expressed as ) and the estimated value 300 expressed as YAM can also output the estimated value 300 expressed as YAM, the estimated value 300 expressed as T-score, and the estimated value 300 expressed as Z-score.

[0106] In addition, the storage unit 20 may also store a plurality of estimation data 120. In this case, among the plurality of estimation-only X-ray images respectively represented by the plurality of estimation data 120 in the storage unit 20, there may be included a plurality of X-ray images of the same type of part, or a plurality of X-ray images of different types of parts. In addition, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images of a part taken from the same direction, or a plurality of X-ray images of a part taken from different directions. In other words, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images in which the orientation of the part captured therein is the same, or a plurality of X-ray images in which the orientation of the part captured therein is different. The control unit 10 inputs each of the plurality of estimation data 120 in the storage unit 20 to the input layer 210 of the neural network 200, and outputs a bone density estimated value 300 corresponding to each estimation data 120 from the output layer 230 of the neural network 200.

[0107] As described above, in this example, the image data of the plain X-ray image is used for the learning of the neural network 200 and the estimation of the bone density in the neural network 200. For the image data of the plain X-ray image, in other words, the image data of the Roentgen image, it can be easily obtained because it is used in various examinations and the like in many hospitals. Therefore, the bone density can be simply estimated without using an expensive device such as a DEXA device.

[0108] In addition, by using the image data of the plain X-ray image taken for examinations and the like as the estimation image data, the bone density can be simply estimated by taking advantage of the opportunity of such examinations. Thus, by using the estimation device 1, the service for hospital users can be improved.

[0109] In addition, in a frontal plain X-ray image such as of the chest, it may be affected by internal organs, making it difficult to capture bones. On the other hand, the possibility of taking a frontal plain X-ray image is relatively high in many hospitals. In this example, even when using a frontal plain X-ray image in which it may be difficult to capture bones as a presumptive plain X-ray image or a learning plain X-ray image, bone density can still be estimated. Therefore, it is possible to simply estimate bone density using the image data of easily obtainable frontal plain X-ray images. Moreover, for a frontal chest plain X-ray image, it is often taken in health examinations and the like, and can be said to be a particularly easily obtainable plain X-ray image. By using the frontal chest plain X-ray image as a presumptive plain X-ray image or a learning plain X-ray image, bone density can be estimated even more simply.

[0110] In addition, in this example, even when the learning plain X-ray images include a plain X-ray image that captures a part of a different type from the presumptive plain X-ray image, bone density can be estimated based on the image data of the presumptive plain X-ray image. Thus, the convenience of the estimation device 1 (in other words, the computer device 1) can be improved.

[0111] In addition, in this example, even when the learning plain X-ray images include a plain X-ray image that captures a part in a different orientation from the presumptive plain X-ray image, bone density can be estimated based on the image data of the presumptive plain X-ray image. Thus, the convenience of the estimation device 1 can be improved.

[0112] In addition, in this example, even when the part captured in the learning plain X-ray image does not include the part (bone) for which the reference bone density corresponding to the learning plain X-ray image has been measured, the neural network 200 can estimate the bone density based on the learned parameters 110. Thus, the convenience of the estimation device 1 can be improved.

[0113] In addition, in this example, even when the orientation of the part captured in the learning plain X-ray image and the orientation of the X-ray irradiation of the object part in the measurement of the reference bone density corresponding to the learning plain X-ray image are different from each other, the neural network 200 can estimate the bone density based on the learned parameters 110. Thus, the convenience of the estimation device 1 can be improved.

[0114] In addition, in this example, even when the training data 140 includes a reference bone density measured from a part that is not included in the part captured in the presumptive plain X-ray image, bone density can be estimated based on the image data of the presumptive plain X-ray image. Thus, the convenience of the estimation device 1 can be improved.

[0115] In addition, the bone density estimated value 300 obtained by the estimation device 1 can also be displayed by the display unit 40. Furthermore, the bone density estimated value 300 obtained by the estimation device 1 can also be used by other devices.

[0116] Figure 6 FIG. is an example of a bone density estimation system 600 including an estimation device 1 and a processing device 500 that processes using the bone density estimated value 300 obtained by the estimation device 1. In Figure 6 this example, the estimation device 1 and the processing device 500 can communicate with each other via a communication network 700. The communication network 700 includes at least one of a wireless network and a wired network, for example. In the communication network 700, for example, a wireless LAN (Local Area Network) and the Internet are included.

[0117] In the estimation device 1, the communication network 700 is connected to the communication unit 30. The control unit 10 causes the communication unit 30 to send the bone density estimated value 300 to the processing device 500. The processing device 500 processes using the bone density estimated value 300 received from the estimation device 1 via the communication network 700. For example, the processing device 500 is a display device such as a liquid crystal display device, and displays the bone density estimated value 300. At this time, the processing device 500 can also display the bone density estimated value 300 in a table or as a graph. In addition, when a plurality of estimation devices 1 are connected to the communication network 700, the processing device 500 can also display the bone density estimated values 300 obtained by the plurality of estimation devices 1. The structure of the processing device 500 can be the same as Figure 1 the structure of the estimation device 1 shown, or can be different from the structure of the estimation device 1.

[0118] In addition, the processing using the bone density estimated value 300 performed by the processing device 500 is not limited to the above example. Furthermore, the processing device 500 can also communicate directly with the estimation device 1 wirelessly or by wire without going through the communication network 700.

[0119] <Other examples of estimation data and learning data>

[0120] <First other example>

[0121] In this example, in the learning data 130, for each piece of learning image data, information related to the health status of the person with the bone photographed in the learning simple X-ray image represented by the learning image data is included. In other words, in the learning data 130, for each piece of learning image data, information related to the health status of the subject (examined subject) of the learning simple X-ray image represented by the learning image data is included. Thereafter, the information related to the health status of the subject of the learning simple X-ray image is sometimes referred to as "learning health-related information". In addition, the information related to the health status of the subject of the learning simple X-ray image represented by the learning image data is sometimes referred to as the learning health-related information corresponding to the learning image data.

[0122] The learning health-related information includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and information on the presence or absence of a fracture history. The learning health-related information is databaseized for each person and generated as a CSV (Comma-Separated Value) format file or a text format file. Each of the age information, height information, and weight information is represented as numerical data of multiple bits, for example. In addition, regarding the gender information, for example, "male" or "female" is represented as 1-bit data, regarding the drinking habit information, "has a drinking habit" or "has no drinking habit" is represented as 1-bit data. In addition, regarding the smoking habit information, 1-bit data is represented as "has a smoking habit" or "has no smoking habit", and regarding the information on the presence or absence of a fracture history, "has a fracture" or "has no fracture" is represented as 1-bit data. In addition, the learning health-related information may also include the body fat percentage or subcutaneous fat percentage of the examinee.

[0123] When the learning data 130 includes the learning image data and the learning health-related information corresponding thereto, the reference bone density corresponding to the learning image data (refer to Figure 4)(It) is associated with the learning health-related information corresponding to the learning image data. That is, with respect to the learning image data representing a learning simple X-ray image of a person's bone and the information related to the health status of the person (learning health-related information), the measured value of the bone density of the person (reference bone density) is associated. Then, in the learning of the neural network 200, the learning image data and the learning health-related information corresponding thereto are input to the input layer 210 simultaneously. Specifically, the learning image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the learning health-related information is input to the other part of the multiple artificial neurons. Then, when the learning image data and the learning health-related information corresponding thereto are input to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data and the learning health-related information.

[0124] In addition, in this example, the data 120 for estimation includes the estimation image data and the information related to the health status of the person having the bone photographed in the estimation simple X-ray image represented by the estimation image data. In other words, the data 120 for estimation includes the estimation image data and the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data. Hereinafter, the information related to the health status of the subject of the estimation simple X-ray image may sometimes be referred to as "estimation health-related information (hereinafter, in another embodiment, may sometimes be referred to as 'individual data')." In addition, the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data may sometimes be referred to as the estimation health-related information corresponding to the estimation image data.

[0125] Similar to the learning health-related information, the estimation health-related information includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and the presence or absence information of fracture history. The estimation health-related information includes the same types of information as the learning health-related information. In addition, the estimation health-related information may also include the body fat percentage or subcutaneous fat percentage of the subject, similar to the learning health-related information.

[0126] In this example, when estimating bone density, the image data for estimation and the health-related information for estimation corresponding thereto are input to the input layer 210 at the same time. Specifically, the image data for estimation is input to a part of a plurality of artificial neurons constituting the input layer 210, and the health-related information for estimation is input to other parts of the plurality of artificial neurons. If the image data for estimation and the health-related information for estimation for a certain person are input to the input layer 210, the estimated value of the bone density of the certain person is output from the output layer 230.

[0127] In this way, by using not only the image data of a simple X-ray image but also the information related to the health state of the subject of the simple X-ray image, the accuracy of bone density estimation can be improved.

[0128] <Other Example 2>

[0129] In this example, the learning data 130 includes the image data of N (N≥2) learning simple X-ray images that are taken of the parts of the same person and in which the orientations of the parts taken are different from each other. After that, the N learning simple X-ray images are sometimes collectively referred to as the "learning simple X-ray image group".

[0130] In the learning simple X-ray image group, for example, a front image and a side image of the same person are included. In the learning simple X-ray image group, for example, a chest front simple X-ray image and a waist side simple X-ray image of a certain person are included. The image sizes of the front image and the side image included in the learning simple X-ray image group may also be different from each other. For example, the horizontal width of the image size of the side image may also be smaller than the horizontal width of the image size of the front image. After that, the image data of each learning simple X-ray image in the learning simple X-ray image group is sometimes collectively referred to as the "learning image data group".

[0131] The learning data 130 includes learning image data groups for a plurality of different people. Thus, the learning data 130 includes a plurality of learning image data groups. Moreover, one learning image data group corresponds to one reference bone density. That is, one learning image data group for a certain person corresponds to the measured value (reference bone density) of the bone density of the certain person.

[0132] In the learning of the neural network 200 in this example, each learning image data set is input relative to the input layer 210. When one learning image data set is input relative to the input layer 210, the N learning image data constituting the one learning image data set are simultaneously input to the input layer 210. For example, it is assumed that the learning image data set includes the first learning image data and the second learning image data. In this case, the first learning image data (for example, the image data of a frontal plain X-ray image of the chest) is input to a part of the multiple artificial neurons constituting the input layer 210, and the second learning image data (for example, the image data of a lateral plain X-ray image of the waist) is input to another part of the multiple artificial neurons. Then, when the learning image data set is input relative to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data set.

[0133] In addition, in this example, the estimation data 120 includes the image data of N estimation plain X-ray images that capture parts of the same person and have different orientations of the captured parts. After that, the N estimation plain X-ray images are sometimes collectively referred to as the "estimation plain X-ray image group".

[0134] In the estimation plain X-ray image group, for example, it includes a frontal image and a lateral image of the same person. In the estimation plain X-ray image group, for example, it includes a frontal plain X-ray image of a person's waist and a lateral plain X-ray image of the knee. The image sizes of the frontal image and the lateral image included in the estimation plain X-ray image group may also be different from each other. For example, the horizontal width of the image size of the lateral image may be smaller than the horizontal width of the image size of the frontal image. After that, the image data of each estimation plain X-ray image in the estimation plain X-ray image group is sometimes collectively referred to as the "estimation image data set".

[0135] In this example, when using the estimation data 120 to estimate the bone density, the N estimation image data constituting the estimation image data set are simultaneously input relative to the input layer 210. For example, it is assumed that the estimation image data set includes the first estimation image data and the second estimation image data. In this case, the first estimation image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the second estimation image data is input to another part of the multiple artificial neurons. If the estimation image data set for a certain person is input to the input layer 210, the estimated value of the bone density of the certain person is estimated from the output layer 230.

[0136] In this way, by using the image data of a plurality of simple X-ray images that capture parts of the same subject and have different orientations of the captured parts, the accuracy of bone density estimation can be improved.

[0137] In addition, the learning data 130 may also include a learning image data group and learning health-related information. In this case, in the learning of the neural network 200, the learning image data group and the learning health-related information for the same person are input to the input layer 210 simultaneously. Similarly, the estimation data 120 may also include an estimation image data group and estimation health-related information. In this case, the estimation image data group and the estimation health-related information are input to the input layer 210 simultaneously.

[0138] In each of the above examples, the same learned parameter 110 is used regardless of the type of bone captured in the X-ray image represented by the estimation image data. However, learned parameters 110 corresponding to the type of bone captured in the X-ray image represented by the estimation image data may also be used. In this case, the neural network 200 has a plurality of learned parameters 110 corresponding to multiple types of bones respectively. The neural network 200 uses the learned parameter 110 corresponding to the type of bone captured in the input estimation image data to estimate the bone density. For example, when the X-ray image represented by the input estimation image data captures the lumbar vertebrae, the neural network 200 uses the learned parameter 110 for estimating the bone density of the lumbar vertebrae to estimate the bone density. In addition, when the X-ray image represented by the input estimation image data captures the proximal part of the femur, the neural network 200 uses the learned parameter 110 for estimating the bone density of the proximal part of the femur to estimate the bone density. The neural network 200 uses, for example, the learned parameter 110 indicated by the user through the input unit 50 among the plurality of learned parameters 110. In this case, the user indicates the learned parameter 110 used by the neural network 200 according to the type of bone captured in the X-ray image represented by the estimation image data input to the neural network 200.

[0139] In the learning of the neural network 200, a plurality of learning image data respectively representing a plurality of X-ray images of the same type of bone are used to generate the learned parameter 110 corresponding to the type of bone.

[0140] As described above, the estimation device 1 and the bone density estimation system 600 have been described in detail. However, the above descriptions are examples in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not contradict each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0141] Embodiment 2.

[0142] Figure 7 FIG. is an example showing the structure of the estimation device 1A according to the present embodiment. In the estimation device 1A, the approximator 280 further includes a second neural network 900. The second neural network 900 can detect a fracture based on the learned parameters 910. In addition, the estimation device 1A according to the present embodiment has the same structure as the estimation device 1 according to the first embodiment, and the description of the same structure is omitted. In addition, for convenience of explanation, the neural network 200 described in the above example is referred to as the first neural network 200. The second neural network 900 has, for example, the same structure as the first neural network 200.

[0143] The second neural network 900 can detect a fracture based on the same estimation image data as the estimation image data included in the estimation data 120 input to the first neural network 200. That is, based on one piece of estimation image data, the bone density can be estimated in the first neural network 200, and a fracture can be detected in the second neural network 900. In addition, the detection result 920 of the second neural network 900 may be output from the output layer 230 of the second neural network 900 in the same manner as in the above example.

[0144] In the learning of the second neural network 900, the learning image data of a non-fractured bone and the learning image data of a fractured bone are used to learn the parameters. In addition, in the training data, information indicating whether there is a current fracture and information indicating the fracture site for the bone in which the learning image data is captured are associated with each learning image data. In addition, the training data may include information indicating the past fracture history and information indicating the past fracture site. As a result, the second neural network 900 can detect the presence and location of a fracture for the bone in which the estimation image data is captured based on the estimation image data, and output the detection result 920.

[0145] In addition, the estimation device 1A according to the present embodiment may also be as Figure 8As shown, there is a determination unit 930 for determining whether the subject has osteoporosis. The determination unit 930 can compare and study the estimation result 300 of the first neural network 200 and the perception result 920 of the second neural network 900 to determine whether the subject has osteoporosis.

[0146] The determination unit 900 can also determine osteoporosis based on, for example, specific criteria or well-known guidelines. Specifically, the determination unit 900 can determine that it is osteoporosis when the perception result 920 indicates a fracture at the vertebral body or the proximal part of the femur. In addition, when the bone density estimation value 300 output by the first neural network 200 represents YAM, the determination unit 900 can also determine that it is osteoporosis when YAM is a value less than 80% and the perception result 920 indicates a fracture other than the vertebral body and the proximal part of the femur. In addition, the determination unit 900 can also determine that it is osteoporosis when YAM represented by the bone density estimation value 300 represents a value of 70% or less.

[0147] In addition, in the estimation device 1A according to the present embodiment, the approximator 280 can also be as Figure 9 shown and further has a third neural network 950. The third neural network 950 can divide the bones of the subject based on the learned parameters 960 and the estimation image data included in the estimation data 120.

[0148] For each pixel data of the input estimation image data, the third neural network 950 outputs part information indicating the part of the bone represented by the pixel data. Thus, the bones captured in the X-ray image represented by the estimation image data can be divided. Sometimes the part information is called segment data.

[0149] For example, when the lumbar vertebra is captured in the X-ray image represented by the input estimation image data, for each pixel data of the estimation image data, the third neural network 950 outputs part information indicating which part of L1-L5 of the lumbar vertebra the pixel data indicates. For example, when a certain pixel data of the estimation image data represents L1 of the lumbar vertebra, the third neural network 950 outputs part information representing L1 as the part information corresponding to the pixel data.

[0150] The third neural network 950 uses the learned parameters 960 corresponding to the types of bones captured in the X-ray image represented by the estimation image data. The third neural network 950 has multiple learned parameters 960 corresponding to multiple types of bones respectively. The third neural network 950 uses the learned parameters 960 corresponding to the type of bone captured in the X-ray image represented by the input estimation image data to classify the bones captured in the X-ray image represented by the estimation image data. For example, when the lumbar vertebra is captured in the X-ray image represented by the input estimation image data, the third neural network 950 uses the learned parameters 960 corresponding to the lumbar vertebra to classify the lumbar vertebra into L1-L5. The third neural network 950 uses, for example, the learned parameters 960 indicated by the user through the input unit 50 among the multiple learned parameters 960. In this case, the user indicates the learned parameters 960 used by the third neural network 950 according to the type of bone captured in the X-ray image represented by the estimation image data input to the third neural network 950.

[0151] The third neural network 950 can also classify the bones captured in the X-ray image represented by the input estimation image data into a first part with an implant, a second part with a tumor, and a third part with a fracture. In this case, for each pixel data of the estimation image data, the third neural network 950 outputs part information indicating which of the first part, the second part, and the third part the pixel data indicates. When the pixel data indicates a part other than the first part, the second part, and the third part, the third neural network 950 outputs part information indicating that the pixel data indicates a part other than the first part, the second part, and the third part. When the third neural network 950 classifies the bones captured in the X-ray image represented by the estimation image data into the first part, the second part, and the third part, it can also be said that the third neural network 950 senses the implant embedded in the bones captured in the X-ray image represented by the estimation image data, the fracture of the bones, and the tumor of the bones.

[0152] In the learning of the third neural network 950, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 960 corresponding to the type of the bone. When the third neural network 950 divides the bone captured in the X-ray image represented by the input estimation image data into a first part, a second part, and a third part, among the multiple learning image data, there are included learning image data representing X-ray images of cases where an implant is embedded, learning image data representing X-ray images of cases where there is a tumor in the bone, and learning image data representing X-ray images of cases where there is a fracture. In addition, in the training data, for each learning image data, there is included annotation information for dividing the bone represented by the learning image data. In the annotation information, for each pixel data of the learning image data corresponding thereto, there is included part information indicating the part of the bone represented by the pixel data.

[0153] In addition, the first neural network 200 may also estimate the bone density for each part divided in the third neural network 950. In this case, as Figure 10 shown, the estimation image data 121 and the part information 965 corresponding to each pixel data of the estimation image data 121 output by the third neural network 950 based on the estimation image data 121 are input to the first neural network 200. The first neural network 200 outputs the bone density estimation value 300 for each part divided in the third neural network based on the learned parameters 110 corresponding to the type of the bone captured in the X-ray image represented by the estimation image data 121. For example, when the third neural network 950 divides the cervical vertebra captured in the X-ray image represented by the estimation image data 121 into L1 to L5, the first neural network 200 separately outputs the bone density estimation value 300 for L1, the bone density estimation value 300 for L2, the bone density estimation value 300 for L3, the bone density estimation value 300 for L4, and the bone density estimation value 300 for L5.

[0154] In the learning of the first neural network 200, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 110 corresponding to the type of the bone. In addition, in the training data, for each learning image data, there is included the reference bone density of each part of the bone represented by the learning image data.

[0155] The first neural network 200 uses, for example, the learned parameters 110 indicated by the user through the input unit 50 among the multiple learned parameters 110. In this case, the user indicates the learned parameters 110 to be used by the first neural network 200 according to the type of bone captured in the X-ray image represented by the estimation target image data input to the first neural network 200.

[0156] In the case where the third neural network 950 divides the bones captured in the X-ray image represented by the estimation target image data into a first part where an implant is embedded, a second part with a tumor, and a third part with a fracture, it is also possible to adjust the brightness of the first partial image data representing the first part, the second partial image data representing the second part with a tumor, and the third partial image data representing the third part in the estimation target image data. Figure 11 It is a diagram showing a structural example in this case.

[0157] As Figure 11 shown, the estimation target image data 121 and the part information 965 corresponding to each pixel data of the estimation target image data 121 output by the third neural network 950 based on the estimation target image data 121 are input to the adjustment unit 968. The adjustment unit 968 determines the first partial image data, the second partial image data, and the third partial image data included in the estimation target image data 121 based on the part information 965. Then, the adjustment unit 968 adjusts the brightness of the determined first partial image data, second partial image data, and third partial image data.

[0158] The adjustment unit 968 stores, for example, the brightness of the first part captured in a general X-ray image as the first reference brightness. In addition, the adjustment unit 968 stores the brightness of the second part captured in a general X-ray image as the second reference brightness. Moreover, the adjustment unit 968 stores the brightness of the third part captured in a general X-ray image as the third reference brightness. The adjustment unit 968 subtracts the first reference brightness from the brightness of the first partial image data to adjust the brightness of the first partial image data. In addition, the adjustment unit 968 subtracts the second reference brightness from the brightness of the second partial image data to adjust the brightness of the second partial image data. Moreover, the adjustment unit 968 subtracts the third reference brightness from the brightness of the third partial image data to adjust the brightness of the third partial image data. The adjustment unit 968 inputs the estimation target image data in which the brightness of the first partial image data, the second partial image data, and the third partial image data in the estimation target image data has been adjusted as the estimation target image data with adjusted brightness to the first neural network 200. The first neural network 200 estimates the bone density of the bone captured in the X-ray image represented by the estimation target image data with adjusted brightness.

[0159] Here, it is not easy to accurately estimate the bone density based on the first part where the implant is embedded, the second part with a tumor, and the third part with a fracture. By adjusting the brightness of the first partial image data representing the first part, the second partial image data representing the second part, and the third partial image data representing the third part to be smaller as described above, it is possible to accurately estimate the bone density of the bone captured in the X-ray image represented by the estimation target image data.

[0160] In addition, the adjustment unit 968 may also input, as the estimation target image data whose brightness has been adjusted, the estimation target image data in which the brightness of the first partial image data, the second partial image data, and the third partial image data is forcibly set to zero, into the first neural network 200.

[0161] In addition, the third neural network 950 may also detect only one of the implant, the fracture, and the tumor. In addition, the third neural network 950 may also detect only two of the implant, the fracture, and the tumor. That is, the third neural network 950 may also detect at least one of the implant, the fracture, and the tumor.

[0162] In addition, the estimation device 1A may not include the second neural network 900, but may include the first neural network 200 and the third neural network 950. In addition, the estimation device 1A may not include the first neural network 200, but may include at least one of the second neural network 900 and the third neural network 950.

[0163] As described above in detail for the estimation device 1A, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples that are not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0164] Embodiment 3.

[0165] Figure 12 FIG. is an example showing the structure of an estimation device 1B according to the present embodiment. The estimation device 1B includes a fracture prediction unit 980. The fracture prediction unit 980 can, for example, predict the probability of a fracture based on the estimation result 300 of the neural network 200 of the estimation device 1 according to Embodiment 1. Specifically, for example, an arithmetic expression 990 representing the relationship between the estimation result related to the bone density (such as bone density, etc.) and the probability of a fracture is obtained based on past literature, etc. The fracture prediction unit 980 stores the arithmetic expression 990. The fracture prediction unit 980 can predict the probability of a fracture based on the input estimation result 300 and the stored arithmetic expression 990.

[0166] In addition, the arithmetic expression 990 may also be an arithmetic expression representing the relationship between the estimated result associated with bone density and the probability of fracture after bone nail implantation. As a result, it is possible to study whether to implant a bone nail and a treatment plan including drug administration.

[0167] In addition, the estimation device 1B may also include a second neural network 900. Furthermore, the estimation device 1B may also include a third neural network 950.

[0168] As described above in detail for the estimation device 1B, the above descriptions are all illustrative in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by imagining without departing from the scope of the present disclosure.

[0169] Embodiment 4.

[0170] In Figure 13 it shows the concept of the structure of the estimation system 801 of this embodiment.

[0171] The estimation system 801 of the present disclosure can, for example, estimate the future bone mass of a subject based on an image of the subject's bone such as an X-ray image. The estimation system 801 of the present disclosure includes a terminal device 802 and an estimation device 803. In addition, the bone mass is an index associated with bone density and includes the concept of bone density.

[0172] The terminal device 802 can acquire input information I for input to the estimation device 803. The input information I can be, for example, an X-ray image or the like. In this case, the terminal device 802 can be a device for a doctor or the like to take an X-ray image of the subject. For example, the terminal device 802 can be a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device).

[0173] In addition, the terminal device 802 is not limited to a simple X-ray imaging device. For example, the terminal device 802 may also be an X-ray fluoroscopic imaging device, a CT (Computed Tomography), an MRI (Magnetic Resonance Imaging), a SPECT (Single Photon Emission Computed Tomography)-CT, or tomosynthesis. In this case, the input information I may be, for example, an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image.

[0174] The estimation system 801 can be used, for example, for diagnosing osteoporosis and the like in patients who go to the hospital. The estimation system 801 of the present disclosure uses, for example, the terminal device 802 provided in the X-ray room to take an X-ray photograph of the patient. Then, the image data is transmitted from the terminal device 802 to the estimation device 803, and through the estimation device 803, it is possible to estimate not only the bone mass or bone density of the patient at the current time point but also the bone mass or bone density of the patient in the future that is further in the future than the time of the photograph.

[0175] In addition, the terminal device 802 may not directly transmit the input information I to the estimation device 803. In this case, for example, the input information I obtained by the terminal device 802 may be stored in a storage medium, and the input information I may be input to the estimation device 803 via the storage medium.

[0176] In Figure 14 shows the concept of the structure of the estimation device 803 according to the present embodiment.

[0177] The estimation device 803 can estimate the future bone mass or bone density of the subject based on the input information I obtained by the terminal device 802. In the estimation device 803, the future bone mass or bone density of the subject can be estimated based on the image data obtained by the terminal device 802, and the estimation result O is output.

[0178] The estimation device 803 includes an input unit 831, an approximator 832, and an output unit 833. The input unit 831 inputs the input information I from the terminal device 802. The approximator 832 can estimate the future bone mass or bone density based on the input information I. The output unit 833 can output the estimation result O predicted by the approximator 832.

[0179] The estimation device 803 includes various electronic components and circuits. As a result, the estimation device 803 can form each structural element. For example, the estimation device 803 can integrate multiple semiconductor elements to form at least one integrated circuit (such as an IC: Integrated Circuit or an LSI: Large Scale Integration), or further integrate multiple integrated circuits to form at least one unit, thereby constituting each functional part of the estimation device 803, etc.

[0180] The multiple electronic components can be, for example, active components such as transistors or diodes, or passive components such as capacitors. In addition, the multiple electronic components and the integrated circuits formed by integrating them can be formed by conventionally known methods.

[0181] The input unit 831 inputs the information used in the estimation device 803. For example, the input information I having an X-ray image obtained by the terminal device 802 is input to the input unit 831. The input unit 831 has a communication unit, and the input information I obtained by the terminal device 802 is directly input from the terminal device 802. In addition, the input unit 831 may include an input device capable of inputting the input information I or other information. The input device can be, for example, a keyboard, a touch screen, or a mouse.

[0182] Based on the information input to the input unit 831, the approximator 832 estimates the future bone mass or bone density of the subject. The approximator 832 has AI (Artificial Intelligence). The approximator 832 has a program that functions as AI, and various electronic components and circuits for executing the program. The approximator 832 has a neural network.

[0183] The approximator 832 performs a learning process in advance for the input-output relationship. That is, machine learning can be applied to the approximator 832 using learning data and training data, so that the approximator 832 can calculate the estimation result O based on the input information I. In addition, the learning data or training data can be the input information I input to the estimation device 803 and the data corresponding to the estimation result O output from the estimation device 803.

[0184] Figure 15 Concept of the structure of the approximator 832 of the present disclosure is shown.

[0185] The approximator 832 has a first neural network 8321 and a second neural network 8322. The first neural network 8321 may be any neural network suitable for processing time-series information. For example, the first neural network 8321 may be a ConvLSTM network that combines LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network), etc. The second neural network 8322 may be, for example, a convolutional neural network composed of CNN, etc.

[0186] The first neural network 8321 has an encoding unit E and a decoding unit D. The encoding unit E can extract the temporal changes and the feature amounts of the position information of the input information I. The decoding unit D can calculate new feature amounts based on the feature amounts extracted by the encoding unit E, the temporal changes of the input information I, and the initial values.

[0187] Figure 16 Concept of the structure of the first neural network 8321 of the present disclosure is shown.

[0188] The encoding unit E has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers E1. The decoding unit D has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers D1. Each of the encoding unit E and the decoding unit D may have three or more ConvLSTM layers E1, D1. In addition, the number of the plurality of ConvLSTM layers E1 and the number of the plurality of ConvLSTM layers D1 may be the same.

[0189] In addition, the content learned by each of the plurality of ConvLSTM layers E1 may be different. The content learned by each of the plurality of ConvLSTM layers D1 may be different. For example, for a certain ConvLTSM layer, it learns fine content such as the change of each pixel, and for another ConvLSTM layer, it learns rough content such as the change of the overall image.

[0190] Figure 17 Concept of the structure of the second neural network 8322 is shown.

[0191] The second neural network 8322 has a transformation unit C. The transformation unit C can transform the feature amount calculated by the decoding unit D of the first neural network 8321 into bone mass or bone density. The transformation unit C has a plurality of convolutional layers C1, a plurality of pooling layers C2, and a fully connected layer C3. The fully connected layer C3 is in the stage before the output unit 33. Moreover, in the transformation unit C, the convolutional layer C1 and the pooling layer C2 are alternately arranged between the first neural network 8321 and the fully connected layer C3.

[0192] In addition, during the learning of the approximator 832, the learning data is input to the encoding unit E of the approximator 832. The training data is compared with the output data output from the transformation unit C of the approximator 832 during the learning of the approximator 832. The training data is data representing values measured using a conventional bone density measuring device.

[0193] The output unit 833 can display the estimation result O. The output unit 833 is, for example, a liquid crystal display or an organic EL display. The output unit 833 can display various information such as characters, symbols, and graphics. The output unit 833 can, for example, display numbers or images.

[0194] The estimation device 803 of the present disclosure further has a control unit 834 and a storage unit 835. The control unit 834 can uniformly manage the operation of the estimation device 803 by controlling other structural elements of the estimation device 803.

[0195] The control unit 834, for example, has a processor. The processor can include, for example, one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, or combinations of these devices or any calibrated combinations, or other basic devices or calibrated combinations. The control unit 834, for example, includes a CPU.

[0196] The storage unit 835 includes, for example, a non-transitory storage medium such as a RAM (Random Access Memory) or a ROM (Read-Only Memory) that can be read by the CPU of the control unit 834. A control program for controlling the estimation device 803 such as firmware is stored in the storage unit 835. In addition, the input input information I, the learned learning data, and the training data can also be stored in the storage unit 835.

[0197] The processor of the control unit 834 can execute one or more data calculation programs or processes according to the control program of the storage unit 835. Various functions of the control unit 834 are realized by the CPU of the control unit 834 executing the control program in the storage unit 835.

[0198] In addition, the control unit 834 may also perform other processes as preprocessing of calculation processing as needed.

[0199] <An example of input information, learning data, and training data>

[0200] The input information (hereinafter, also referred to as the first input information I1) has image data of a bone of a subject whose bone mass or bone density is to be estimated. The image data can be, for example, a simple X-ray image. The subject of bone mass or bone density estimation is, for example, a human. In this case, it can be said that the first input information I1 is image data of a simple X-ray image of a human bone. A simple X-ray image is a two-dimensional image and is also referred to as a general X-ray image or a Roentgen image.

[0201] The first input information I1 is preferably a simple X-ray image that is relatively easy to obtain, but is not limited thereto. For example, there are cases where bone mass or bone density can be estimated more accurately by using an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image as input information.

[0202] In addition, the subject of bone mass or bone density estimation may also be other than a human. For example, the subject of bone mass or bone density estimation may be an animal such as a dog, a cat, or a horse. In addition, the bones to be targeted are mainly cortical bone and cancellous bone of biological origin, but the bones to be targeted may also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine or the like.

[0203] As the imaging site of the X-ray image, for example, it can be any of the neck, chest, waist, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, or jaw joint. In addition, parts other than bones may also be imaged in the X-ray image. For example, in the case of a chest simple X-ray image, it may also include an image of the lungs and an image of the thoracic vertebrae. The X-ray image can be a frontal image of the target site taken from the front or a lateral image of the target site taken from the side.

[0204] The learning data or training data may be data corresponding to the input information I input to the estimation device 803 and the estimation result O output from the estimation device 803.

[0205] The learning data has the same type of information as the first input information I1. For example, if the first input information I1 is a simple X-ray image, the learning data only needs to have a simple X-ray image. Further, when the first input information I1 is a chest simple X-ray image, the learning data only needs to have a chest simple X-ray image.

[0206] The learning image data including multiple simple X-ray images of bones is included in the learning data. Among the shooting parts of the multiple learning image data, for example, at least one of the neck, chest, waist, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, and jaw joint is included. In the learning data, it may include image data of some types among the 11 types of image data, or may include all types of image data. In addition, among the multiple learning image data, it may include a front image and a side image.

[0207] In the learning data, bones of multiple different individuals are respectively photographed. For each of the multiple learning image data, as training data, the measured values of the bone mass or bone density of the subject of each learning image data are established in correspondence. The measured values of the bone mass or bone density are the measured values measured at a time period almost the same as the time period when the learning image data was photographed.

[0208] In addition, the learning image data of the learning data may also be a series of data with different time axes for photographing the same person. That is, the learning image data is the first learning data with an X-ray image of a bone and an image of the same person as the first learning data, and may also include the second learning data with an X-ray image photographed after the first learning data.

[0209] In addition, the learning image data of the learning data may also be a data set with different ages, etc. for photographing the same part of others. In addition, the learning image data of the learning data may also be a series of data with different time axes for photographing the same person and the same part.

[0210] The learning data and the first input information I1 may also use input information obtained by compressing image data representing the gray scale of a simple X-ray image (in other words, a general X-ray imaging device or a Roentgen imaging device) and reducing the number of gray scales. For example, consider the case where the number of pixel data of the image data is more than (1024×640) and the number of bits of the pixel data is 16 bits. In this case, input information obtained by compressing the number of pixel data to, for example, (256×256), (1024×512), or (1024×640) and reducing the number of bits of the pixel data to 8 bits is used as the first input information I1 and the learning data.

[0211] The training data includes, for each of the plurality of learning image data included in the learning data, a measured value of the bone mass or bone density of the bone captured in the simple learning X-ray image represented by the learning image data. The bone mass or bone density may be measured, for example, by a DEXA (dual-energy X-ray absorptiometry) method or an ultrasonic method.

[0212] <Neural network learning example>

[0213] The control unit 834 performs machine learning using the learning data and the training data on the approximator 832, so that the approximator 832 can calculate the estimated result O related to the bone mass or bone density based on the input information I. The approximator 832 is optimized by the well-known machine learning using the training data. The approximator 832 performs calculations based on the learning data input to the encoding unit E, and adjusts the variable parameters in the approximator 832 so that the difference between the pseudo estimated result output from the conversion unit C and the training data is small.

[0214] Specifically, the control unit 834 inputs the learning data in the storage unit 835 to the encoding unit E. When the learning data is input to the encoding unit E, the control unit 834 inputs the plurality of pixel data constituting the learning image data to the plurality of artificial neurons constituting the encoding unit E. Then, when the learning image data is input to the encoding unit E, the control unit 834 adjusts the parameters so that the error of the estimated result O output from the conversion unit C relative to the bone mass or bone density actual measurement value corresponding to the learning image data becomes smaller. The adjusted parameters become the parameters after learning and are stored in the storage unit 835.

[0215] As a parameter adjustment method, for example, an error back propagation method may be used. The parameters include, for example, parameters used by the encoding unit E, the decoding unit D, and the transformation unit C. Specifically, the parameters include weighting coefficients used by the ConvLSTM layers of the encoding unit E and the decoding unit D, and the convolutional layer and the fully connected layer of the transformation unit C.

[0216] As a result, the approximator 832 performs operations based on the learned parameters for the input information I input to the encoder E, and outputs the estimated result O from the transform unit C. When the X-ray image data as the input information I is input to the encoder E, the plurality of pixel data constituting the image data are respectively input to the plurality of artificial neurons constituting the input unit 831. Then, the ConvLSTM layer, the convolution layer, and the fully connected layer can perform operations using the weighting coefficients included in the learned parameters, thereby outputting the estimated result O.

[0217] As described above, in the estimation system 801, learning of the approximator 832 and estimation of bone mass or bone density in the approximator 832 are performed using the image data of the plain X-ray image. Therefore, input information I can be input to the estimation system 801, and future bone mass or bone density can be output as the estimation result O.

[0218] The estimation result O of the estimation system 801 only needs to be the estimation result on a certain day in the future that is more future than the acquisition date of the input information I. For example, the estimation system 1 can estimate the bone mass or bone density from 3 months to 50 years after the shooting, and more preferably from 6 months to 10 years after the shooting.

[0219] The estimation result O only needs to be output as a value. For example, it can be represented by at least one of YAM (Young Adult Mean), T-score, and Z-score. For example, the output unit 833 can output the estimated value represented as YAM, or the estimated value represented as YAM, the estimated value represented as T-score, and the estimated value represented as Z-score.

[0220] In addition, the estimation result O can also be output as an image. When the estimation result O is an image, for example, it only needs to be an image that can display an X-ray-like image. In addition, an image of an X-ray-like image is an image that imitates an X-ray image. In addition, when using ConvLSTM to learn a series of data with different time axes for the same person and the same part, the time change of the image can be predicted. Thus, a future image can be generated based on the X-ray image at one time point of another patient.

[0221] In the learning data and the input information I, in addition to bones, internal organs, muscles, fat, or blood vessels can also be photographed. Even in this case, highly accurate estimation can be performed.

[0222] The first input information I1 can also have the individual data (first individual data) of the subject. The so-called first individual data can be, for example, age information, gender information, height information, weight information, or fracture history, etc. As a result, highly accurate estimation can be performed.

[0223] The first input information I1 can also have the second individual data of the subject. The so-called second individual data can also include, for example, information on blood pressure, lipids, cholesterol, neutral fat, and blood glucose level. As a result, highly accurate estimation can be performed.

[0224] The first input information I1 can also have the lifestyle information of the subject. The so-called lifestyle information can be information such as drinking habits, smoking habits, exercise habits, or eating habits. As a result, highly accurate estimation can be performed.

[0225] The first input information I1 may also include the bone metabolism information of the subject. The so-called bone metabolism information can be, for example, bone resorption ability or bone formation ability. For example, they can be measured by at least one of N-terminal telopeptide of type I collagen cross-linking (NTX), C-terminal telopeptide of type I collagen cross-linking (CTX), tartrate-resistant acid phosphatase (TRACP-5b), deoxypyridinoline (DPD) as bone resorption markers, bone-specific alkaline phosphatase (BAP), N-propeptide of type I collagen cross-linking (P1NP) as bone formation markers, and undercarboxylated osteocalcin (ucOC) as bone-related matrix markers. The bone resorption markers can be measured using serum or urine as the test body.

[0226] In the estimation system 801, as the input information I, the second input information I2 related to the future planned actions of the subject can be further input. The so-called second input information I2 can be, for example, information related to the individual data that is planned to be improved or has been improved, or the lifestyle and exercise and eating habits that are planned to be improved or have been improved. Specifically, the second input information I2 can be information such as weight data, drinking habits, smoking habits, sun exposure time, number of steps or walking distance per day, dairy product intake, or intake of foods rich in vitamin D such as improved fish and mushrooms. As a result, the estimation system 801 can present an estimated result O in which the future bone mass or bone density has been improved.

[0227] In addition, the second input information I2 can also be, for example, information related to the lifestyle habits that are expected to deteriorate. As a result, the estimation system can present an estimated result O in which the future bone mass or bone density has deteriorated.

[0228] In the estimation system 801, as the input information I, the third input information I3 related to the therapy for the subject can be further input. The so-called third input information I3 is, for example, information related to physical therapy or drug therapy. Specifically, the third input information I3 can be at least one of calcium tablets, female hormone drugs, vitamin drugs, bisphosphonate drugs, SERM (Selective Estrogen Receptor Modulator) drugs, calcitonin drugs, thyroid hormone drugs, and denosumab drugs.

[0229] In the estimation system 801, the estimation result O can also output the first result O1 based only on the first input information I1 and the second result O2 based on at least one of the first input information I1 and the second and third input information I2 and I3. As a result, it is possible to compare the effects of future planned actions.

[0230] In the estimation system 801, the estimation result O not only outputs the future bone mass or bone density, but can also output the current result. As a result, it is possible to compare the changes in bone mass or bone density over time.

[0231] Figure 18 Concept of the structure of the approximator 832 showing another embodiment of the estimation system 1.

[0232] The estimation device 803 of the estimation system 801 may also have a first approximator 832a and a second approximator 832b. That is, in addition to the above-described approximator 832 (first approximator 832a), a second approximator 832b may also be provided. The second approximator 832b may be, for example, a CNN.

[0233] In this case, in the estimation system 801, the first approximator 832a outputs the first image and the first value to the first output unit 833a as the first estimation result O1. Further, the second approximator 832b outputs the second value to the second output unit 833b as the second estimation result O2 based on the first image from the first output unit 833a. As a result, it is possible to compare the first value and the second value as the estimation result O of the future bone mass or bone density.

[0234] The estimation system 801 may also output a third value based on the first value and the second value as the estimation result O. As a result, for example, it is possible to set the result (third value) obtained by calibrating the first value based on the second value as the estimation result O.

[0235] As described above in detail for the estimation system 801, the above description is an example in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

Claims

1. A presumption device comprising: A presumption unit that, based on learned parameters, presumes the future bone mass or bone density of a person according to first input information that includes at least one of a simple X-ray image including the person's bones, the person's bone metabolism information, and information related to a therapy for the person.

2. The presumption device according to claim 1, wherein The information related to the therapy includes information related to physical therapy or drug therapy.

3. The presumption device according to claim 1, wherein The information related to the therapy includes information related to at least one of calcium tablets, female hormone drugs, vitamin drugs, bisphosphonate drugs, SERM drugs (selective estrogen receptor modulators), calcitonin drugs, thyroid hormone drugs, and denosumab drugs.

4. The presumption device according to claim 1 or 2, wherein The bone metabolism information includes information related to bone resorption ability or bone formation ability.

5. The presumption device according to claim 4, wherein The information related to the bone resorption ability includes the measurement result of a bone resorption marker.

6. The presumption device according to claim 4, wherein The information related to the bone formation ability includes the measurement result of a bone formation marker.

7. The presumption device according to claim 1 or 2, wherein The presumption unit has: an encoding unit that extracts feature amounts of the temporal change and position information of the first input information; a decoding unit that calculates new feature amounts based on the feature amounts, the temporal change of the first input information, and an initial value; And a transformation unit that transforms the new feature amounts into the future bone mass or bone density of the person.

8. The presumption device according to claim 1 or 2, wherein The presumption result of the presumption unit has a first image.

9. The presumption device according to claim 8, wherein The first image has an X-ray image.

10. The presumption device according to claim 8, wherein The presumption unit has: a first presumption unit that can presume the first image and a first value as the presumption result; and a second presumption unit that can presume a second value as the presumption result according to the first image.

11. The presumption device according to claim 10, wherein The presumption unit outputs a third value as the presumption result based on the first value and the second value.

12. The presumption device according to claim 1 or 2, wherein The simple X-ray image is a front image.

13. The presumption device according to claim 1 or 2, wherein The simple X-ray image is a side image.

14. The presumption device according to claim 1 or 2, wherein The bone mineral density estimated by the estimation unit is represented by at least one of the bone mineral density per unit area g / cm 2 , the bone mineral density per unit volume g / cm 3 , YAM, T-score, and Z-score.

15. The presumption device according to claim 1 or 2, wherein The first input information has the person's individual data.

16. The presumption device according to claim 15, wherein The individual data includes at least one of age information, gender information, height information, weight information, and fracture history.

17. The presumption device according to claim 15, wherein The individual data includes information on at least one of blood pressure, lipids, cholesterol, neutral fat, and blood glucose level.

18. The estimation device according to claim 1 or 2, wherein the estimation unit estimates the future bone mass or bone density of the person based on the first input information and second information related to a future predetermined action of the person.

19. The estimation device according to claim 18, wherein the estimation unit estimates the future bone mass or bone density of the person based on the first input information and the second information, and the estimation unit estimates the future bone mass or bone density of the person based only on the first input information among the first input information and the second information.

20. The estimation device according to claim 1 or 2, wherein the estimation unit estimates the future bone mass or bone density of the person and the current bone mass or bone density of the person.

21. The estimation device according to claim 20, wherein the estimation device displays the future bone mass or bone density estimated by the estimation unit and the current bone mass or bone density estimated by the estimation unit.

22. The estimation device according to claim 1 or 2, wherein the estimation device displays the estimation result of the estimation unit by an image.

23. An estimation method, which is an estimation method in an estimation device. In the estimation method, based on the first input information having at least one of a simple X-ray image of a person's skeleton, the person's bone metabolism information, and information related to a therapy for the person, and based on parameters with learning completed, the future bone mass or bone density of the person is estimated.

24. A system includes: an estimation unit that estimates the future bone mass or bone density of the person based on the first input information having at least one of a simple X-ray image of a person's skeleton, the person's bone metabolism information, and information related to a therapy for the person, and based on parameters with learning completed.

Images