Estimation device, device, learning model generation method, and prediction system

Abstract

The invention provides an estimation device, an estimation device, a learning model generation method, and a prediction system. The estimation device includes an input unit and an approximator. The input unit inputs input information having an image in which a bone has been captured. The approximator is capable of estimating an estimation result relating to the bone mineral density of the bone on the basis of the input information input to the input unit. The approximator has a parameter for obtaining a learning completion of an estimation result relating to the bone mineral density of the bone from input information.

Description

[0001] This application is a divisional application of the invention patent application with the application date of September 10, 2019, the application number of 201980058486.5, and the invention title of "Presumption Device, Presumption System, and Computer-Readable Non-Temporary Recording Medium Recording a Presumption Program".

[0002] Cross-reference to related applications

[0003] This application claims the priority of Japanese Patent Application No. 2018-168502 (filed on September 10, 2018) and Japanese National Application No. 2018-220401 (filed on November 26, 2018), and the entire disclosure of this application is incorporated herein by reference for reference purposes. Technical field

[0004] This disclosure relates to the presumption of bone density. Background art

[0005] In Patent Document 1, a technique for determining osteoporosis is described. In Patent Document 2, a technique for presuming the strength of bones is described.

[0006] Prior art documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2008-36068

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2002-523204 Summary of the invention

[0010] A presumption device, a presumption system, and a presumption program are disclosed. In one embodiment, the presumption device includes: an input unit that inputs input information having an image of a bone captured; an approximator that can presume a presumption result associated with the bone density of the bone based on the input information input to the input unit; and an output unit that outputs the presumption result presumed by the approximator, and the approximator has parameters that have completed learning for obtaining a presumption result associated with the bone density of the bone based on the input information.

[0011] In addition, in one embodiment, the presumption system includes: an input unit that inputs input information having an image of a bone captured; and an approximator that has parameters that have completed learning for obtaining a presumption result associated with the bone density of the bone based on the input information and can presume a presumption result associated with the bone density of the bone based on the input information input to the input unit, and when the input information is input to the input unit, the approximator performs arithmetic processing on the input information.

[0012] In addition, in one embodiment, the estimation program is an estimation program for causing the device to function as a neural network that performs operations based on parameters for which learning has been completed and outputs an estimated value of the bone density of the bone captured in the image, and the parameters for which learning has been completed are used to obtain an estimation result associated with the bone density of the bone from input information having an image in which the bone is captured. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. is an example of the configuration of a computer device (estimation device).

[0014] Figure 2 FIG. is a diagram for explaining the operation of the estimation device.

[0015] Figure 3 FIG. is an example of the configuration of a neural network.

[0016] Figure 4 FIG. is an example of the state in which learning image data and reference bone density are associated with each other.

[0017] Figure 5 FIG. is a diagram for explaining the operation of the estimation device.

[0018] Figure 6 FIG. is an example of an estimation system.

[0019] Figure 7 FIG. is an example of the configuration of the estimation device.

[0020] Figure 8 FIG. is an example of the configuration of the estimation device.

[0021] Figure 9 FIG. is an example of the configuration of the estimation device.

[0022] Figure 10 FIG. is an example of the configuration of the estimation device.

[0023] Figure 11 FIG. is an example of the configuration of the estimation device.

[0024] Figure 12 FIG. is an example of the configuration of the estimation device.

[0025] Figure 13 FIG. is a conceptual diagram schematically showing the configuration of the estimation system.

[0026] Figure 14 FIG. is a conceptual diagram schematically showing the configuration of a part of the estimation system.

[0027] Figure 15 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0028] Figure 16 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0029] Figure 17 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0030] Figure 18 It is a conceptual diagram schematically showing the structure of a part of another embodiment of a presumption system.

[0031] Symbol Explanation

[0032] 1: Computer device (presumption device);

[0033] 20: Storage unit;

[0034] 100: Control program;

[0035] 110, 910, 960: Learned parameters;

[0036] 120: Data for presumption;

[0037] 130: Data for learning;

[0038] 140: Training data;

[0039] 200: Neural network;

[0040] 210: Input layer;

[0041] 230: Output layer;

[0042] 280, 832: Approximator;

[0043] 500: Processing device;

[0044] 600: Bone density presumption system;

[0045] 801: Presumption system;

[0046] 802: Terminal device;

[0047] 803: Presumption device;

[0048] 831: Input unit;

[0049] 833: Output unit;

[0050] 834: Control unit;

[0051] 835: Storage unit;

[0052] 900, 8322: Second neural network;

[0053] 930: Determination unit;

[0054] 950: Third neural network;

[0055] 980: Fracture prediction unit;

[0056] 8321: First neural network;

[0057] O: Estimation result;

[0058] I: Input information;

[0059] E: Encoding unit;

[0060] D: Decoding unit;

[0061] C: Transformation unit. Detailed implementation

[0062] Embodiment 1.

[0063] Figure 1 is a block diagram showing an example of the structure of the computer device 1 according to Embodiment 1. The computer device 1 functions as an estimation device for estimating bone density. Hereinafter, the computer device 1 may sometimes be referred to as the "estimation device 1".

[0064] As Figure 1 shown, the estimation device 1 includes, for example, a control unit 10, a storage unit 20, a communication unit 30, a display unit 40, and an input unit 50. The control unit 10, the storage unit 20, the communication unit 30, the display unit 40, and the input unit 50 are electrically connected to each other through a bus 60, for example.

[0065] The control unit 10 can manage the operation of the estimation device 1 uniformly by controlling other structural elements of the estimation device 1. The control unit 10 is also referred to as a control device or a control circuit. As described in further detail below, the control unit 10 includes at least one processor in order to provide control and processing capabilities for executing various functions.

[0066] According to various embodiments, at least one processor may also be implemented as a single integrated circuit (IC) or as multiple communicably connected integrated circuits (ICs) and / or discrete circuits. At least one processor can be implemented according to various known techniques.

[0067] In one embodiment, a processor includes, for example, one or more circuits or units configured to execute one or more data calculation programs or processes by executing instructions stored in an associated memory. In another embodiment, the processor may also be firmware (e.g., discrete logic elements) configured to execute one or more data calculation programs or processes.

[0068] According to various embodiments, the processor may also include one or more processors, controllers, microprocessors, microcontrollers, application-specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, field-programmable logic devices, or any combination of these devices or structures, or a combination of other known devices and structures, and perform the functions described below. In this example, the control unit 10 includes, for example, a CPU (Central Processing Unit).

[0069] The storage unit 20 includes non-transitory storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory) that can be read by the CPU of the control unit 10. A control program 100 for controlling the estimation device 1 is stored in the storage unit 20. Various functions of the control unit 10 are implemented by the CPU of the control unit 10 executing the control program 100 in the storage unit 20. It can also be said that the control program 100 is a bone density estimation program for causing the computer device 1 to function as an estimation device. In this example, by the control unit 10 executing the control program 100 in the storage unit 20, as Figure 2 shown, an approximator 280 capable of outputting an estimated value 300 of bone density is formed in the control unit 10. The approximator 280 includes, for example, a neural network 200. The control program 100 can also be said to be a program for causing the computer device 1 to function as the neural network 200. Thereafter, the estimated value of bone density is sometimes referred to as the "bone density estimated value". In addition, a structural example of the neural network 200 will be described in detail later.

[0070] In addition to the control program 100, learned parameters 110 related to the neural network 200, estimation data 120 (hereinafter, also referred to as "input information"), learning data 130, and training data 140 are also stored in the storage unit 20. The learning data 130 and the training data 140 are data used when the neural network 200 is being learned. The learned parameters 110 and the estimation data 120 are data used by the learned neural network 200 when estimating bone density.

[0071] The learning data 130 is the data input to the input layer 210 of the neural network 200 during the learning of the neural network 200. The learning data 130 is also referred to as learning data. The training data 140 is the data representing the correct value of bone density. The training data 140 is compared with the output data output from the output layer 230 of the neural network 200 during the learning of the neural network 200. The learning data 130 and the training data 140 are sometimes collectively referred to as the learning data with training.

[0072] The data for estimation 120 is the data input to the input layer 210 of the neural network 200 when the bone density is estimated by the learned neural network 200. The learned parameters 110 are the learned parameters within the neural network 200. The learned parameters 110 can also be referred to as the parameters adjusted through the learning of the neural network 200. Among the learned parameters 110, there are included the weighting coefficients representing the weights of the connections between artificial neurons. As shown in Figure 2 the learned neural network 200 performs an operation based on the learned parameters 110 for the data for estimation 120 input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230.

[0073] In addition, the data input to the input layer 210 can be input to the input layer 210 either via the input unit 50 or directly to the input layer 210. In the case of directly inputting to the input layer 210, the input layer 210 can also be set as a part or all of the input unit 50. Thereafter, the bone density estimation value 300 is sometimes referred to as the estimation result 300.

[0074] The communication unit 30 is connected to a communication network including the Internet or the like by wire or wirelessly. The communication unit 30 can communicate with other devices such as a cloud server and a network server through the communication network. The communication unit 30 can input the information received from the communication network to the control unit 10. In addition, the communication unit 30 can output the information received from the control unit 10 to the communication network.

[0075] The display unit 40 is, for example, a liquid crystal display or an organic EL display. The display unit 40 can display various information such as characters, symbols, and graphics by being controlled by the control unit 10.

[0076] The input unit 50 can accept the input from the user to the estimation device 1. The input unit 50 includes, for example, a keyboard and a mouse. The input unit 50 can also include a touch screen capable of detecting the operation of the user on the display surface of the display unit 40.

[0077] In addition, the structure of the estimation device 1 is not limited to the above examples. For example, the control unit 10 may also include multiple CPUs. In addition, the control unit 10 may also include at least one DSP. In addition, all functions of the control unit 10 or some of the functions of the control unit 10 may also be implemented by a hardware circuit that does not require software for the implementation of its functions. In addition, the storage unit 20 may also include a non-transitory storage medium readable by a computer other than a ROM and a RAM. For example, the storage unit 20 may also include a small hard disk drive and an SSD (Solid State Drive), etc. In addition, the storage unit 20 may also include a memory such as a USB (Universal Serial Bus) memory that can be plugged into and unplugged from the estimation device 1. Hereinafter, a memory that can be plugged into and unplugged from the estimation device 1 is sometimes referred to as a "plug-and-play memory".

[0078] <Example of the structure of the neural network>

[0079] Figure 3 FIG. is an example showing the structure of the neural network 200. In this example, the neural network 200 is, for example, a convolutional neural network (CNN (Convolutional Neural Network)). As Figure 3 shown, the neural network 200 includes, for example, an input layer 210, a hidden layer 220, and an output layer 230. The hidden layer 220 is also referred to as an intermediate layer. The hidden layer 220 includes, for example, multiple convolutional layers 240, multiple pooling layers 250, and a fully connected layer 260. In the neural network 200, a fully connected layer 260 exists in front of the output layer 230. Moreover, in the neural network 200, the convolutional layers 240 and the pooling layers 250 are alternately arranged between the input layer 210 and the fully connected layer 260.

[0080] In addition, the structure of the neural network 200 is not limited to Figure 3 the example. For example, the neural network 200 may also include one convolutional layer 240 and one pooling layer 250 between the input layer 210 and the fully connected layer 260. In addition, the neural network 200 may also be a neural network other than a convolutional neural network.

[0081] <An example of the data for estimation, the data for learning, and the training data>

[0082] The data 120 for estimation includes image data of a simple X-ray image of a bone that is the object of bone density estimation. The object of bone density estimation is, for example, a human. Therefore, it can be said that the data 120 for estimation includes image data of a simple X-ray image of a human bone. In the data 130 for learning, it includes image data of a plurality of simple X-ray images of a human bone. A simple X-ray image is also referred to as a two-dimensional image, a general X-ray image, or a Roentgen image. In addition, the object of bone density estimation may be other than a human. For example, the object of bone density estimation may be an animal such as a dog, a cat, or a horse. Furthermore, the bone set as the object is mainly cortical bone and cancellous bone from a living body, but among the bones set as the object, it may also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine or the like.

[0083] After that, sometimes the image data included in the data 120 for estimation is referred to as "estimation image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 120 for estimation is referred to as "estimation simple X-ray image". In addition, sometimes the image data included in the data 130 for learning is referred to as "learning image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 130 for learning is referred to as "learning simple X-ray image". In the data 130 for learning, it includes a plurality of learning X-ray image data respectively representing a plurality of learning simple X-ray images.

[0084] As the imaging site for the simple X-ray image for estimation, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers or jaw joint can be used. In other words, as the estimation data 120, the following can be used: the image data of the simple X-ray image obtained by irradiating the head with X-rays, the image data of the simple X-ray image obtained by irradiating the neck with X-rays, the image data of the simple X-ray image obtained by irradiating the chest with X-rays, the image data of the simple X-ray image obtained by irradiating the waist with X-rays, the image data of the simple X-ray image obtained by irradiating the hip joint with X-rays, the image data of the simple X-ray image obtained by irradiating the knee joint with X-rays, the image data of the simple X-ray image obtained by irradiating the ankle joint with X-rays, the image data of the simple X-ray image obtained by irradiating the foot with X-rays, the image data of the simple X-ray image obtained by irradiating the toes with X-rays, the image data of the simple X-ray image obtained by irradiating the shoulder joint with X-rays, the image data of the simple X-ray image obtained by irradiating the elbow joint with X-rays, the image data of the simple X-ray image obtained by irradiating the wrist joint with X-rays, the image data of the simple X-ray image obtained by irradiating the hand with X-rays, the image data of the simple X-ray image obtained by irradiating the fingers with X-rays or the image data of the simple X-ray image obtained by irradiating the jaw joint with X-rays. Among the simple X-ray images obtained by irradiating the chest with X-rays, there are the simple X-ray images that capture the lungs and the simple X-ray images that capture the thoracic vertebrae. In addition, the types of the imaging sites for the simple X-ray image for estimation are not limited to this. Furthermore, the simple X-ray image for estimation can be either a front image of the target site captured from the front or a side image of the target site captured from the side.

[0085] Among the imaging regions of the multiple learning X-ray images respectively represented by the multiple pieces of learning data 130, there is included at least one of, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers, and jaw joint. In other words, in the learning data 130, there is included at least one of the 15 types of image data, namely, the image data of a simple X-ray image obtained by irradiating the head with X-rays, the image data of a simple X-ray image obtained by irradiating the neck with X-rays, the image data of a simple X-ray image obtained by irradiating the chest with X-rays, the image data of a simple X-ray image obtained by irradiating the waist with X-rays, the image data of a simple X-ray image obtained by irradiating the hip joint with X-rays, the image data of a simple X-ray image obtained by irradiating the knee joint with X-rays, the image data of a simple X-ray image obtained by irradiating the ankle joint with X-rays, the image data of a simple X-ray image obtained by irradiating the foot with X-rays, the image data of a simple X-ray image obtained by irradiating the toes with X-rays, the image data of a simple X-ray image obtained by irradiating the shoulder joint with X-rays, the image data of a simple X-ray image obtained by irradiating the elbow joint with X-rays, the image data of a simple X-ray image obtained by irradiating the wrist joint with X-rays, the image data of a simple X-ray image obtained by irradiating the hand with X-rays, the image data of a simple X-ray image obtained by irradiating the fingers with X-rays, and the image data of a simple X-ray image obtained by irradiating the jaw joint with X-rays. In the learning data 130, it may include some types of the 15 types of image data or all types of image data. In addition, the types of imaging regions of the learning simple X-ray images are not limited to this. Furthermore, among the multiple learning simple X-ray images, there may be included a front image or a side image. In addition, among the multiple learning simple X-ray images, there may also be included both a front image and a side image of the same imaging region.

[0086] In the training data 140, for each of the plurality of learning image data included in the learning data 130, it includes the measured value of the bone density of a person having a bone photographed in the learning simple X-ray image represented by the learning image data. Among the plurality of measured values of bone density included in the training data 140, for example, it includes at least one of the measured value of bone density measured by irradiating X-rays to the lumbar spine, the bone density measured by irradiating X-rays to the proximal part of the femur, the bone density measured by irradiating X-rays to the radius, the bone density measured by irradiating X-rays to the metacarpal bone, the bone density measured by applying ultrasonic waves to the wrist, and the bone density measured by applying ultrasonic waves to the heel. After that, sometimes the measured value of bone density included in the training data 140 is referred to as "reference bone density".

[0087] Here, as a method for measuring bone density, the DEXA (dual-energy X-ray absorptiometry) method is known. In a DEXA device that uses the DEXA method to measure bone density, when measuring the bone density of the lumbar spine, X-rays (specifically, two types of X-rays) are irradiated to the lumbar spine from its front. In addition, in the DEXA device, when measuring the bone density of the proximal part of the femur, X-rays are irradiated to the proximal part of the femur from its front.

[0088] In the training data 140, it may include the bone density of the lumbar spine measured by the DEXA device, or may include the bone density of the proximal part of the femur measured by the DEXA device. In addition, in the training data 140, it may also include the bone density measured by irradiating X-rays to the target part from its side. For example, in the training data 140, it may also include the bone density measured by irradiating X-rays to the lumbar spine from its side.

[0089] In addition, as another method for measuring bone density, the ultrasonic wave method is known. In a device that uses the ultrasonic wave method to measure bone density, for example, ultrasonic waves are applied to the wrist to measure the bone density of the wrist, and ultrasonic waves are applied to the heel to measure the bone density of the heel. In the training data 140, it may also include the bone density measured by the ultrasonic wave method.

[0090] In the plurality of learning simple X-ray images represented by the plurality of learning image data included in the learning data 130, bones of different people are photographed. And, as Figure 4As shown, in the storage unit 20, for each of the multiple learning image data included in the learning data 130, a correspondence is established with the reference bone density of a person who has bones photographed in the learning simple X-ray image represented by the learning image data. It can also be said that for each of the multiple learning simple X-ray images used in the learning of the neural network 200, a correspondence is established with the reference bone density of a person who has bones photographed in the learning simple X-ray image. The reference bone density corresponding to the learning image data is the bone density measured for the same person as the person who has bones photographed in the learning simple X-ray image represented by the learning image data at a time almost the same as the time when the learning simple X-ray image represented by the learning image data was photographed.

[0091] In the part photographed in the learning simple X-ray image represented by the learning image data (in other words, the photographing part of the learning simple X-ray image), the part (i.e., the bone) for which the reference bone density corresponding to the learning image data is measured may or may not be included. In other words, in the part photographed in the learning simple X-ray image, the part for which the reference bone density corresponding to the learning simple X-ray image is measured may or may not be included. As an example of the former, consider the case where a correspondence is established between the learning image data representing the learning simple X-ray image of the waist being photographed and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the hip joint being photographed and the reference bone density of the proximal part of the femur. On the other hand, as an example of the latter, consider the case where a correspondence is established between the learning image data of the chest being photographed and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the knee joint being photographed and the reference bone density of the heel.

[0092] In addition, the orientation of the part captured in the learning single X-ray image represented by the image data for learning and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning image data may be the same or different. In other words, the orientation of the part captured in the learning single X-ray image and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning single X-ray image may be the same or different. As an example of the former, consider the case of establishing a correspondence between the learning image data representing a single X-ray image of the chest captured from the front (hereinafter, sometimes referred to as a "front chest single X-ray image") and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing a correspondence between the learning image data representing a single X-ray image of the waist captured from the front (hereinafter, sometimes referred to as a "front waist single X-ray image") and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front. On the other hand, as an example of the latter, consider the case of establishing a correspondence between the learning image data representing a single X-ray image of the waist captured from the side (hereinafter, sometimes referred to as a "side waist single X-ray image") and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing a correspondence between the learning image data representing a single X-ray image of the knee joint captured from the side (hereinafter, sometimes referred to as a "side knee single X-ray image") and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front.

[0093] In addition, among the multiple learning single X-ray images respectively represented by the multiple learning image data included in the learning data 130, there may be included a single X-ray image capturing the same type of part as the estimation single X-ray image, or a single X-ray image capturing a different type of part from the estimation single X-ray image. As an example of the former, consider the case where the estimation single X-ray image is a front chest single X-ray image and a front chest single X-ray image is included among the multiple learning single X-ray images. As another example, consider the case where the estimation single X-ray image is a single X-ray image of the knee joint captured from the front (hereinafter, sometimes referred to as a "front knee single X-ray image") and a side knee single X-ray image is included among the multiple learning single X-ray images. On the other hand, as an example of the latter, consider the case where the estimation single X-ray image is a front waist single X-ray image and a front chest single X-ray image is included among the multiple learning single X-ray images. As another example, consider the case where the estimation single X-ray image is a side waist single X-ray image and a front knee single X-ray image is included among the multiple learning single X-ray images.

[0094] In addition, among the multiple learning plain X-ray images, there may be included plain X-ray images of parts taken in the same orientation as the estimation plain X-ray image, or plain X-ray images of parts taken in an orientation different from that of the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar anterior plain X-ray image, and a lumbar anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the estimation plain X-ray image is a knee anterior plain X-ray image, and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a knee lateral plain X-ray image, and a knee anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the estimation plain X-ray image is a lumbar lateral plain X-ray image, and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images.

[0095] In addition, in the training data 140, there may be included reference bone densities measured from parts (bones) included in the part imaged in the estimation plain X-ray image, or reference bone densities measured from parts (bones) not included in the part imaged in the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar anterior plain X-ray image, and the reference bone density of the lumbar vertebra is included in the training data 140. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a chest anterior plain X-ray image, and the reference bone density of the metacarpal head is included in the training data 140.

[0096] In addition, in the training data 140, there may be included reference bone densities measured by irradiating the target part with X-rays from the same orientation as the orientation of the part imaged in the estimation plain X-ray image, or reference bone densities measured by irradiating the target part with X-rays from an orientation different from that of the part imaged in the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar anterior plain X-ray image, and the reference bone density measured by irradiating the lumbar vertebra with X-rays from its anterior side is included in the training data 140. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a lumbar lateral plain X-ray image, and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its anterior side is included in the training data 140.

[0097] In this example, the image data representing the gray scale of a simple X-ray image obtained by a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device) is compressed, and the image data obtained by reducing the number of gray scales is used as learning image data and estimation image data. For example, when the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is more than (1024×640) and the number of bits of the pixel data is 16 bits. In this case, the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is compressed to, for example, (256×256), (1024×512), or (1024×640), and the number of bits of the pixel data is reduced to 8 bits, and then used as learning image data and estimation image data. In this case, each of the learning simple X-ray image and the estimation simple X-ray image includes (256×256), (1024×512), or (1024×640) pixels, and the value of the pixel is expressed in 8 bits.

[0098] For the learning image data and the estimation image data, they can be generated by the control unit 10 of the estimation device 1 based on the image data obtained by the simple X-ray imaging device, or can be generated by a device other than the estimation device 1 based on the image data obtained by the simple X-ray imaging device. In the former case, for the image data obtained by the simple X-ray imaging device, it can be received by the communication unit 30 through the communication network, or can be stored in the plug-in memory included in the storage unit 20. In the latter case, it can also be that the communication unit 30 receives the learning image data and the estimation image data from other devices through the communication network, and the control unit 10 stores the learning image data and the estimation image data received by the communication unit 30 in the storage unit 20. Or, the learning image data and the estimation image data generated by other devices can be stored in the plug-in memory included in the storage unit 20. In addition, for the training data 140, it can also be that the communication unit 30 receives it through the communication network, and the control unit 10 stores the training data 140 received by the communication unit 30 in the storage unit 20. Or, the training data 140 can be stored in the plug-in memory included in the storage unit 20. In addition, the number and number of bits of the pixel data of the learning image data and the estimation image data are not limited to the above.

[0099] <Example of neural network learning>

[0100] Figure 5 It is a diagram for explaining an example of the learning of the neural network 200. When the control unit 10 performs the learning of the neural network 200, as Figure 5As shown, learning data 130 is input to the input layer 210 of the neural network 200. Then, the control unit 10 adjusts the variable parameters 110a within the neural network 200 so that the error of the output data 400 output from the output layer 230 of the neural network 200 with respect to the training data 140 is small. More specifically, the control unit 10 inputs each learning image data within the storage unit 20 to the input layer 210. When inputting the learning image data to the input layer 210, the control unit 10 inputs the multiple pixel data constituting the learning image data to the multiple artificial neurons constituting the input layer 210 respectively. Then, when the learning image data is input to the input layer 210, the control unit 10 adjusts the parameters 110a so that the error of the output data 400 output from the output layer 230 with respect to the reference bone density corresponding to the learning image data is small. As a method for adjusting the parameters 110a, for example, the error backpropagation method can be adopted. The adjusted parameters 110a become the learned parameters 110 and are stored in the storage unit 20. Among the parameters 110a, for example, the parameters used in the hidden layer 220 are included. Specifically, the filter coefficients used in the convolutional layer 240 and the weighting coefficients used in the fully connected layer 260 are included in the parameters 110a. In addition, the method for adjusting the parameters 110a, in other words, the learning method of the parameters 110a is not limited to this.

[0101] In this way, in the storage unit 20, the learned parameters 110 are stored, which have learned the relationship between the learning data 130 of the image data including multiple learning simple X-ray images and the measured values of the bone density as the training data 140 using the neural network 200.

[0102] In the above example, the estimation device 1 performs the learning of the neural network 200, but other devices can also perform the learning of the neural network 200. In this case, the learned parameters 110 generated by other devices are stored in the storage unit 20 of the estimation device 1. In addition, it becomes unnecessary to store the learning data 130 and the training data 140 in the storage unit 20. For the learned parameters 110 generated by other devices, they can also be received by the communication unit 30 through the communication network, and the control unit 10 stores the learned parameters 110 received by the communication unit 30 in the storage unit 20. Alternatively, the learned parameters 110 generated by other devices can be stored in the pluggable memory included in the storage unit 20.

[0103] In the neural network 200 learned as above, the image data of multiple learning simple X-ray images is input to the input layer 210 as the learning data 130, and the learned parameters 110a learned using the reference bone density as the training data 140 are included. As described above Figure 2As shown, the neural network 200 performs operations based on the learned parameters 110a on the estimation data 120 input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230. When the estimation image data as the estimation data 120 is input to the input layer 210, the multiple pixel data constituting the estimation image data are respectively input to multiple artificial neurons constituting the input layer 210. Then, the convolutional layer 240 performs operations using the filter coefficients included in the learned parameters 110a, and the fully connected layer 260 performs operations using the weighting coefficients included in the learned parameters 110a.

[0104] For example, when the estimation image data representing a frontal chest plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the bones of the chest photographed in the frontal chest plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a frontal lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the lumbar vertebrae included in the lumbar region photographed in the frontal lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a lateral lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the lumbar vertebrae included in the lumbar region photographed in the lateral lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a frontal knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the bones of the knee joint photographed in the frontal knee plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a lateral knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the bones of the knee joint photographed in the lateral knee plain X-ray image represented by the estimation image data is output from the output layer 230.

[0105] The estimation value 300 output from the output layer 230 can be represented by at least one of the bone mineral density per unit area (g / cm 2 ), the bone mineral density per unit volume (g / cm 3 ), YAM, T-score, and Z-score. YAM is an abbreviation for "Young Adult Mean" and is sometimes referred to as the young adult average percentage. For example, from the output layer 230, it is possible to output the bone mineral density per unit area (g / cm 2The estimated value 300 expressed as ) and the estimated value 300 expressed as YAM can also output the estimated value 300 expressed as YAM, the estimated value 300 expressed as T-score, and the estimated value 300 expressed as Z-score.

[0106] In addition, the storage unit 20 can also store a plurality of estimation data 120. In this case, among the plurality of estimation-only X-ray images respectively represented by the plurality of estimation data 120 in the storage unit 20, there may be included a plurality of X-ray images of the same type of part, or a plurality of X-ray images of mutually different types of parts. Further, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images of a part taken from the same direction, or a plurality of X-ray images of a part taken from different directions. In other words, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images in which the parts captured therein have the same orientation, or a plurality of X-ray images in which the parts captured therein have different orientations. The control unit 10 inputs each of the plurality of estimation data 120 in the storage unit 20 to the input layer 210 of the neural network 200, and outputs from the output layer 230 of the neural network 200 the bone density estimated value 300 corresponding to each estimation data 120.

[0107] As described above, in this example, the image data of the plain X-ray image is used for the learning of the neural network 200 and the estimation of the bone density in the neural network 200. For the image data of the plain X-ray image, that is, the image data of the Roentgen image, since it is used in various examinations and the like in many hospitals, it can be easily obtained. Therefore, the bone density can be simply estimated without using an expensive device such as a DEXA device.

[0108] In addition, by using the image data of the plain X-ray image taken for examinations and the like as the estimation image data, the bone density can be simply estimated by taking advantage of the opportunity of such examinations. Thus, by using the estimation device 1, the service for hospital users can be improved.

[0109] In addition, in a frontal simple X-ray image of the chest, etc., it may be difficult to capture bones due to the influence of internal organs. On the other hand, in many hospitals, the possibility of capturing a frontal simple X-ray image is high. In this example, even when a frontal simple X-ray image in which bones may be difficult to capture is used as an estimation simple X-ray image or a learning simple X-ray image, bone density can be estimated. Therefore, bone density can be simply estimated using image data of an easily available frontal simple X-ray image. In addition, a chest frontal simple X-ray image is often captured in health diagnosis, etc., and can be said to be a simple X-ray image that is particularly easy to obtain. By using a chest frontal simple X-ray image as an estimation simple X-ray image or a learning simple X-ray image, bone density can be further simply estimated.

[0110] Furthermore, in this example, even when the plurality of learning simple X-ray images include a simple X-ray image that captures a part of a different type from that of the estimation simple X-ray image, the bone density can be estimated based on the image data of the estimation simple X-ray image. This can improve the convenience of the estimation device 1 (in other words, the computer device 1).

[0111] Furthermore, in this example, even when the plurality of learning simple X-ray images include a simple X-ray image that captures a part in a different orientation from that of the estimation simple X-ray image, the bone density can be estimated based on the image data of the estimation simple X-ray image. This can improve the convenience of the estimation device 1.

[0112] Furthermore, in this example, even when the parts captured in the simple X-ray image for learning do not include the parts (bone) for which the reference bone density corresponding to the simple X-ray image for learning has been measured, the neural network 200 can estimate the bone density based on the learned parameters 110. This can improve the convenience of the estimation device 1.

[0113] Furthermore, in this example, even when the orientation of the part captured in the simple X-ray image for learning and the orientation of the X-ray irradiation of the target part in the measurement of the reference bone density corresponding to the simple X-ray image for learning are different from each other, the neural network 200 can estimate the bone density based on the learned parameter 110. Thus, the convenience of the estimation device 1 can be improved.

[0114] Furthermore, in this example, even when the training data 140 includes a reference bone density measured from a site not included in the site captured by the estimation simple X-ray image, the bone density can be estimated based on the image data of the estimation simple X-ray image. This can improve the convenience of the estimation device 1.

[0115] In addition, the bone density estimated value 300 obtained by the estimation device 1 can also be displayed by the display unit 40. Further, the bone density estimated value 300 obtained by the estimation device 1 can also be used by other devices.

[0116] Figure 6 FIG. is an example of a bone density estimation system 600 including an estimation device 1 and a processing device 500 that processes using the bone density estimated value 300 obtained by the estimation device 1. In Figure 6 this example, the estimation device 1 and the processing device 500 can communicate with each other via a communication network 700. The communication network 700 includes at least one of a wireless network and a wired network, for example. In the communication network 700, for example, a wireless LAN (Local Area Network) and the Internet are included.

[0117] In the estimation device 1, the communication network 700 is connected to the communication unit 30. The control unit 10 causes the communication unit 30 to send the bone density estimated value 300 to the processing device 500. The processing device 500 processes using the bone density estimated value 300 received from the estimation device 1 via the communication network 700. For example, the processing device 500 is a display device such as a liquid crystal display device, and displays the bone density estimated value 300. At this time, the processing device 500 may display the bone density estimated value 300 in a tabular form or in a graph form. Further, when a plurality of estimation devices 1 are connected to the communication network 700, the processing device 500 may also display the bone density estimated values 300 obtained by the plurality of estimation devices 1. The structure of the processing device 500 may be the same as the structure of the Figure 1 estimation device 1 shown, or may be different from the structure of the estimation device 1.

[0118] In addition, the processing using the bone density estimated value 300 performed by the processing device 500 is not limited to the above example. Further, the processing device 500 may communicate directly with the estimation device 1 wirelessly or by wire without going through the communication network 700.

[0119] <Other examples of estimation data and learning data>

[0120] <First other example>

[0121] In this example, in the learning data 130, for each piece of learning image data, information related to the health status of the person with the bone captured in the learning simple X-ray image represented by the learning image data is included. In other words, in the learning data 130, for each piece of learning image data, information related to the health status of the subject (examined subject) of the learning simple X-ray image represented by the learning image data is included. Thereafter, the information related to the health status of the subject of the learning simple X-ray image is sometimes referred to as "learning health-related information". In addition, the information related to the health status of the subject of the learning simple X-ray image represented by the learning image data is sometimes referred to as the learning health-related information corresponding to the learning image data.

[0122] The learning health-related information includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and information on the presence or absence of a fracture history. The learning health-related information is databaseized for each person and generated as a file in CSV (Comma-Separated Value) format or text format. Each of the age information, height information, and weight information is represented as numerical data of multiple bits, for example. In addition, regarding the gender information, for example, "male" or "female" is represented as 1-bit data, regarding the drinking habit information, "has a drinking habit" or "has no drinking habit" is represented as 1-bit data. In addition, regarding the smoking habit information, "has a smoking habit" or "has no smoking habit" is represented as 1-bit data, and regarding the information on the presence or absence of a fracture history, "has a fracture" or "has no fracture" is represented as 1-bit data. In addition, the learning health-related information may also include the body fat percentage or subcutaneous fat percentage of the examinee.

[0123] In the case where the learning data 130 includes the learning image data and the learning health-related information corresponding thereto, the reference bone density corresponding to the learning image data (refer to Figure 4is associated with the learning health-related information corresponding to the learning image data. That is, the measured value of the bone density (reference bone density) of a certain person is associated with the learning image data representing the learning simple X-ray image of the bone of the certain person and the information related to the health status of the certain person (learning health-related information). Then, in the learning of the neural network 200, the learning image data and the corresponding learning health-related information are input to the input layer 210 simultaneously. Specifically, the learning image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the learning health-related information is input to the other part of the multiple artificial neurons. Then, when the learning image data and the corresponding learning health-related information are input to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data and the learning health-related information.

[0124] In addition, in this example, the estimation data 120 includes the estimation image data and the information related to the health status of the person with the bone photographed in the estimation simple X-ray image represented by the estimation image data. In other words, the estimation data 120 includes the estimation image data and the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data. Hereinafter, the information related to the health status of the subject of the estimation simple X-ray image may sometimes be referred to as "estimation health-related information (hereinafter, in another embodiment, sometimes also referred to as "individual data")." In addition, the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data may sometimes be referred to as the estimation health-related information corresponding to the estimation image data.

[0125] Similar to the learning health-related information, the estimation health-related information includes at least one of, for example, age information, gender information, height information, weight information, drinking habit information, smoking habit information, and information on the presence or absence of a fracture history. The estimation health-related information includes the same types of information as the learning health-related information. In addition, the estimation health-related information may also include the body fat percentage or subcutaneous fat percentage of the subject, similar to the learning health-related information.

[0126] In this example, when estimating bone density, the image data for estimation and the health-related information for estimation corresponding thereto are input to the input layer 210 simultaneously. Specifically, the image data for estimation is input to a part of the multiple artificial neurons constituting the input layer 210, and the health-related information for estimation is input to other parts of the multiple artificial neurons. If the image data for estimation and the health-related information for estimation for a certain person are input to the input layer 210, the estimated value of the bone density of the certain person is output from the output layer 230.

[0127] In this way, by using not only the image data of the simple X-ray image but also the information related to the health state of the subject of the simple X-ray image, the accuracy of bone density estimation can be improved.

[0128] <Other Example 2>

[0129] In this example, in the learning data 130, the image data of N (N≥2) learning simple X-ray images that capture the parts of the same person and have different orientations of the captured parts are included. After that, the N learning simple X-ray images are sometimes collectively referred to as the "learning simple X-ray image group".

[0130] In the learning simple X-ray image group, for example, a front image and a side image of the same person are included. In the learning simple X-ray image group, for example, a chest front simple X-ray image and a waist side simple X-ray image of a certain person are included. The image sizes of the front image and the side image included in the learning simple X-ray image group may also be different from each other. For example, the horizontal width of the image size of the side image may be smaller than the horizontal width of the image size of the front image. After that, the image data of each learning simple X-ray image in the learning simple X-ray image group is sometimes collectively referred to as the "learning image data group".

[0131] In the learning data 130, the learning image data groups for multiple different people are included. Thus, multiple learning image data groups are included in the learning data 130. Moreover, one reference bone density is corresponded to one learning image data group. That is, the measured value (reference bone density) of the bone density of a certain person is corresponded to the learning image data group for the certain person.

[0132] In the learning of the neural network 200 in this example, each learning image data set is input with respect to the input layer 210. When one learning image data set is input with respect to the input layer 210, the N learning image data constituting the one learning image data set are simultaneously input to the input layer 210. For example, it is assumed that the learning image data set includes the first learning image data and the second learning image data. In this case, the first learning image data (for example, the image data of a frontal plain X-ray image of the chest) is input to a part of the multiple artificial neurons constituting the input layer 210, and the second learning image data (for example, the image data of a lateral plain X-ray image of the waist) is input to another part of the multiple artificial neurons. Then, when the learning image data set is input with respect to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data set.

[0133] In addition, in this example, the estimation data 120 includes the image data of N estimation plain X-ray images that capture parts of the same person and have different orientations of the captured parts. After that, the N estimation plain X-ray images are sometimes collectively referred to as the "estimation plain X-ray image group".

[0134] In the estimation plain X-ray image group, for example, it includes a frontal image and a lateral image of the same person. In the estimation plain X-ray image group, for example, it includes a frontal plain X-ray image of a person's waist and a lateral plain X-ray image of the knee. The image sizes of the frontal image and the lateral image included in the estimation plain X-ray image group may also be different from each other. For example, the horizontal width of the image size of the lateral image may be smaller than the horizontal width of the image size of the frontal image. After that, the image data of each estimation plain X-ray image in the estimation plain X-ray image group are sometimes collectively referred to as the "estimation image data set".

[0135] In this example, when using the estimation data 120 to estimate the bone density, the N estimation image data constituting the estimation image data set are simultaneously input with respect to the input layer 210. For example, it is assumed that the estimation image data set includes the first estimation image data and the second estimation image data. In this case, the first estimation image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the second estimation image data is input to another part of the multiple artificial neurons. If the estimation image data set of a certain person is input to the input layer 210, the estimated value of the bone density of the certain person is estimated from the output layer 230.

[0136] In this way, by using the image data of a plurality of simple X-ray images that capture the parts of the same subject and in which the orientations of the captured parts are different from each other, the accuracy of bone density estimation can be improved.

[0137] In addition, the learning data 130 may also include a learning image data group and learning health-related information. In this case, in the learning of the neural network 200, the learning image data group and the learning health-related information for the same person are input to the input layer 210 simultaneously. Similarly, the estimation data 120 may also include an estimation image data group and estimation health-related information. In this case, the estimation image data group and the estimation health-related information are input to the input layer 210 simultaneously.

[0138] In each of the above examples, the same learned parameter 110 is used regardless of the type of bone captured in the X-ray image represented by the estimation image data. However, learned parameters 110 corresponding to the type of bone captured in the X-ray image represented by the estimation image data may also be used. In this case, the neural network 200 has a plurality of learned parameters 110 corresponding to multiple types of bones respectively. The neural network 200 uses the learned parameter 110 corresponding to the type of bone captured in the X-ray image represented by the input estimation image data to estimate the bone density. For example, when the X-ray image represented by the input estimation image data captures the lumbar vertebrae, the neural network 200 uses the learned parameter 110 for bone density estimation of the lumbar vertebrae to estimate the bone density. In addition, when the X-ray image represented by the input estimation image data captures the proximal part of the femur, the neural network 200 uses the learned parameter 110 for bone density estimation of the proximal part of the femur to estimate the bone density. The neural network 200 uses, for example, the learned parameter 110 indicated by the user through the input unit 50 among the plurality of learned parameters 110. In this case, the user indicates the learned parameter 110 used by the neural network 200 according to the type of bone captured in the X-ray image represented by the estimation image data input to the neural network 200.

[0139] In the learning of the neural network 200, a plurality of learning image data respectively representing a plurality of X-ray images capturing the same type of bone are used to generate the learned parameter 110 corresponding to the type of bone.

[0140] As described above, the estimation device 1 and the bone density estimation system 600 have been described in detail. However, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not contradict each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0141] Embodiment 2.

[0142] Figure 7 FIG. is an example showing the structure of the estimation device 1A according to the present embodiment. In the estimation device 1A, the approximator 280 further includes a second neural network 900. The second neural network 900 can detect a fracture based on the learned parameters 910. In addition, the estimation device 1A according to the present embodiment has the same structure as the estimation device 1 according to the first embodiment, and the description of the same structure is omitted. In addition, for convenience of explanation, the neural network 200 described in the above example is referred to as the first neural network 200. The second neural network 900 has, for example, the same structure as the first neural network 200.

[0143] The second neural network 900 can detect a fracture based on the same estimation image data as the estimation image data included in the estimation data 120 input to the first neural network 200. That is, based on one estimation image data, the bone density can be estimated in the first neural network 200, and a fracture can be detected in the second neural network 900. In addition, the detection result 920 of the second neural network 900 may be output from the output layer 230 of the second neural network 900 in the same manner as in the above example.

[0144] In the learning of the second neural network 900, learning image data of a non-fractured bone and learning image data of a fractured bone are used to learn the parameters. In addition, in the training data, information indicating whether there is a current fracture and information indicating the fracture site for the bone in which the learning image data is captured are associated with each learning image data. In addition, the training data may include information indicating a past fracture history and information indicating a past fracture site. As a result, the second neural network 900 can detect the presence or absence and location of a fracture in the bone in which the estimation image data is captured based on the estimation image data, and output the detection result 920.

[0145] In addition, the estimation device 1A according to the present embodiment may also be as Figure 8As shown, there is a determination unit 930 that determines whether the subject has osteoporosis. The determination unit 930 can compare and study the estimation result 300 of the first neural network 200 and the perception result 920 of the second neural network 900 to determine whether the subject has osteoporosis.

[0146] The determination unit 900 can also determine osteoporosis based on, for example, specific criteria or well-known guidelines. Specifically, the determination unit 900 can determine that it is osteoporosis when the perception result 920 indicates a fracture at the vertebral body or the proximal part of the femur. In addition, when the bone density estimation value 300 output by the first neural network 200 represents YAM, the determination unit 900 can also determine that it is osteoporosis when YAM is a value less than 80% and the perception result 920 indicates a fracture other than the vertebral body and the proximal part of the femur. In addition, the determination unit 900 can also determine that it is osteoporosis when YAM represented by the bone density estimation value 300 indicates a value of 70% or less.

[0147] In addition, in the estimation device 1A according to the present embodiment, the approximator 280 can also be as Figure 9 shown, and further has a third neural network 950. The third neural network 950 can divide the bones of the subject based on the learned parameters 960 and the estimation image data included in the estimation data 120.

[0148] For each pixel data of the input estimation image data, the third neural network 950 outputs part information indicating the part of the bone represented by the pixel data. Thus, the bone captured in the X-ray image represented by the estimation image data can be divided. Sometimes the part information is called segment data.

[0149] For example, when the X-ray image represented by the input estimation image data captures the lumbar vertebrae, for each pixel data of the estimation image data, the third neural network 950 outputs part information indicating which part of L1-L5 of the lumbar vertebrae the pixel data indicates. For example, when a certain pixel data of the estimation image data represents L1 of the lumbar vertebrae, the third neural network 950 outputs part information indicating L1 as the part information corresponding to the pixel data.

[0150] The third neural network 950 uses the learned parameters 960 corresponding to the types of bones captured in the X-ray image represented by the estimation image data. The third neural network 950 has multiple learned parameters 960 corresponding to multiple types of bones respectively. The third neural network 950 uses the learned parameters 960 corresponding to the type of bone captured in the X-ray image represented by the input estimation image data to divide the bones captured in the X-ray image represented by the estimation image data. For example, when the lumbar vertebra is captured in the X-ray image represented by the input estimation image data, the third neural network 950 uses the learned parameters 960 corresponding to the lumbar vertebra to divide the lumbar vertebra into L1-L5. The third neural network 950 uses, for example, the learned parameters 960 indicated by the user through the input unit 50 among the multiple learned parameters 960. In this case, the user indicates the learned parameters 960 used by the third neural network 950 according to the type of bone captured in the X-ray image represented by the estimation image data input to the third neural network 950.

[0151] The third neural network 950 may also divide the bones captured in the X-ray image represented by the input estimation image data into a first part implanted with an implant, a second part with a tumor, and a third part with a fracture. In this case, for each pixel data of the estimation image data, the third neural network 950 outputs part information indicating which of the first part, the second part, and the third part the pixel data indicates. When the pixel data indicates a part other than the first part, the second part, and the third part, the third neural network 950 outputs part information indicating that the pixel data indicates a part other than the first part, the second part, and the third part. When the third neural network 950 divides the bones captured in the X-ray image represented by the estimation image data into the first part, the second part, and the third part, it can also be said that the third neural network 950 senses the implant embedded in the bones captured in the X-ray image represented by the estimation image data, the fracture of the bone, and the tumor of the bone.

[0152] In the learning of the third neural network 950, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 960 corresponding to the type of the bone. When the third neural network 950 divides the bone captured in the X-ray image represented by the input estimation image data into a first part, a second part, and a third part, among the multiple learning image data, there are included learning image data representing X-ray images of cases where implants are embedded, learning image data representing X-ray images of cases where there are tumors in the bone, and learning image data representing X-ray images of cases where there are fractures. In addition, in the training data, for each learning image data, there is included annotation information for dividing the bone represented by the learning image data. In the annotation information, for each pixel data of the corresponding learning image data, there is included part information indicating the part of the bone represented by the pixel data.

[0153] In addition, the first neural network 200 may also estimate the bone density for each part divided in the third neural network 950. In this case, as Figure 10 shown, the estimation image data 121 and the part information 965 corresponding to each pixel data of the estimation image data 121 output by the third neural network 950 based on the estimation image data 121 are input to the first neural network 200. The first neural network 200 outputs the bone density estimation value 300 for each part divided in the third neural network based on the learned parameters 110 corresponding to the type of the bone captured in the X-ray image represented by the estimation image data 121. For example, when the third neural network 950 divides the cervical vertebra captured in the X-ray image represented by the estimation image data 121 into L1 to L5, the first neural network 200 separately outputs the bone density estimation value 300 for L1, the bone density estimation value 300 for L2, the bone density estimation value 300 for L3, the bone density estimation value 300 for L4, and the bone density estimation value 300 for L5.

[0154] In the learning of the first neural network 200, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 110 corresponding to the type of the bone. In addition, in the training data, for each learning image data, there is included the reference bone density of each part of the bone represented by the learning image data.

[0155] The first neural network 200 uses, for example, the learned parameters 110 indicated by the user through the input unit 50 among the multiple learned parameters 110. In this case, the user indicates the learned parameters 110 to be used by the first neural network 200 according to the type of bone captured in the X-ray image represented by the estimation target image data input to the first neural network 200.

[0156] When the third neural network 950 divides the bones captured in the X-ray image represented by the estimation target image data into a first part where an implant is embedded, a second part with a tumor, and a third part with a fracture, it is also possible to adjust the brightness of the first partial image data representing the first part, the second partial image data representing the second part with a tumor, and the third partial image data representing the third part in the estimation target image data. Figure 11 It is a diagram showing a structural example in this case.

[0157] As Figure 11 shown, the estimation target image data 121 and the part information 965 corresponding to each pixel data of the estimation target image data 121 output by the third neural network 950 based on the estimation target image data 121 are input to the adjustment unit 968. The adjustment unit 968 determines the first partial image data, the second partial image data, and the third partial image data included in the estimation target image data 121 based on the part information 965. Then, the adjustment unit 968 adjusts the brightness of the determined first partial image data, second partial image data, and third partial image data.

[0158] The adjustment unit 968 stores, for example, the brightness of the first part captured in a general X-ray image as the first reference brightness. In addition, the adjustment unit 968 stores the brightness of the second part captured in a general X-ray image as the second reference brightness. Moreover, the adjustment unit 968 stores the brightness of the third part captured in a general X-ray image as the third reference brightness. The adjustment unit 968 subtracts the first reference brightness from the brightness of the first partial image data to adjust the brightness of the first partial image data. In addition, the adjustment unit 968 subtracts the second reference brightness from the brightness of the second partial image data to adjust the brightness of the second partial image data. Moreover, the adjustment unit 968 subtracts the third reference brightness from the brightness of the third partial image data to adjust the brightness of the third partial image data. The adjustment unit 968 inputs the estimation target image data with the brightness of the first partial image data, the second partial image data, and the third partial image data adjusted in the estimation target image data to the first neural network 200 as the estimation target image data with adjusted brightness. The first neural network 200 estimates the bone density of the bone captured in the X-ray image represented by the estimation target image data with adjusted brightness.

[0159] Here, it is not easy to accurately estimate the bone density based on the first part where the implant is embedded, the second part with a tumor, and the third part with a fracture. By adjusting the brightness of the first partial image data representing the first part, the second partial image data representing the second part, and the third partial image data representing the third part to be smaller as described above, it is possible to accurately estimate the bone density of the bone captured in the X-ray image represented by the estimation target image data.

[0160] In addition, the adjustment unit 968 may also input the estimation target image data in which the brightness of the first partial image data, the second partial image data, and the third partial image data is forcibly set to zero as the estimation target image data after brightness adjustment into the first neural network 200.

[0161] In addition, the third neural network 950 may also detect only one of the implant, the fracture, and the tumor. In addition, the third neural network 950 may also detect only two of the implant, the fracture, and the tumor. That is, the third neural network 950 may also detect at least one of the implant, the fracture, and the tumor.

[0162] In addition, the estimation device 1A may not include the second neural network 900, but may include the first neural network 200 and the third neural network 950. In addition, the estimation device 1A may not include the first neural network 200, but may include at least one of the second neural network 900 and the third neural network 950.

[0163] As described above in detail for the estimation device 1A, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not contradict each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0164] Embodiment 3.

[0165] Figure 12 FIG. is an example showing the structure of the estimation device 1B according to the present embodiment. The estimation device 1B includes a fracture prediction unit 980. The fracture prediction unit 980 can, for example, predict the probability of a fracture based on the estimation result 300 of the neural network 200 of the estimation device 1 according to Embodiment 1. Specifically, for example, an arithmetic expression 990 representing the relationship between the estimation result (such as bone density, etc.) related to the bone density and the probability of a fracture is obtained based on past literature, etc. The fracture prediction unit 980 stores the arithmetic expression 990. The fracture prediction unit 980 can predict the probability of a fracture based on the input estimation result 300 and the stored arithmetic expression 990.

[0166] In addition, the arithmetic expression 990 may also be an arithmetic expression representing the relationship between the estimated result associated with bone density and the probability of fracture after bone screw implantation. As a result, it is possible to conduct research on whether to implant a bone screw and a treatment plan including drug administration.

[0167] In addition, the estimation device 1B may also include a second neural network 900. Further, the estimation device 1B may also include a third neural network 950.

[0168] As described above in detail for the estimation device 1B, the above description is illustrative in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by imagining without departing from the scope of the present disclosure.

[0169] Embodiment 4.

[0170] In Figure 13 it shows the concept of the structure of the estimation system 801 of this embodiment.

[0171] The estimation system 801 of the present disclosure can, for example, estimate the future bone mass of a subject based on an image of the subject's bone such as an X-ray image. The estimation system 801 of the present disclosure includes a terminal device 802 and an estimation device 803. In addition, the bone mass is an index associated with bone density and includes the concept of bone density.

[0172] The terminal device 802 can acquire input information I for input to the estimation device 803. The input information I can be, for example, an X-ray image or the like. In this case, the terminal device 802 can be a device for a doctor or the like to take an X-ray image of the subject. For example, the terminal device 802 can be a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device).

[0173] In addition, the terminal device 802 is not limited to a simple X-ray imaging device. For example, the terminal device 802 may also be an X-ray fluoroscopic imaging device, a CT (Computed Tomography), an MRI (Magnetic Resonance Imaging), an SPECT (Single Photon Emission Computed Tomography)-CT, or tomosynthesis. In this case, the input information I may be, for example, an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image.

[0174] The estimation system 801 can be used, for example, for diagnosing osteoporosis and the like in patients who go to the hospital. The estimation system 801 of the present disclosure uses, for example, the terminal device 802 provided in the X-ray room to take an X-ray photograph of the patient. Then, the image data is transmitted from the terminal device 802 to the estimation device 803, and through the estimation device 803, it is possible to estimate not only the bone mass or bone density of the patient at the current time point but also the bone mass or bone density of the patient in the future that is further in the future than the time of the photograph.

[0175] In addition, the terminal device 802 may not directly transmit the input information I to the estimation device 803. In this case, for example, the input information I obtained by the terminal device 802 may be stored in a storage medium, and the input information I may be input to the estimation device 803 via the storage medium.

[0176] In Figure 14 shows the concept of the structure of the estimation device 803 according to the present embodiment.

[0177] The estimation device 803 can estimate the future bone mass or bone density of the subject based on the input information I obtained by the terminal device 802. In the estimation device 803, it is possible to estimate the future bone mass or bone density of the subject based on the image data obtained by the terminal device 802 and output the estimation result O.

[0178] The estimation device 803 includes an input unit 831, an approximator 832, and an output unit 833. The input unit 831 inputs the input information I from the terminal device 802. The approximator 832 can estimate the future bone mass or bone density based on the input information I. The output unit 833 can output the estimation result O predicted by the approximator 832.

[0179] The estimation device 803 has various electronic components and circuits. As a result, the estimation device 803 can form each structural element. For example, the estimation device 803 can integrate multiple semiconductor elements to form at least one integrated circuit (e.g., IC: Integrated Circuit or LSI: Large Scale Integration), or further integrate multiple integrated circuits to form at least one unit, etc., thereby constituting each functional part of the estimation device 803, etc.

[0180] The multiple electronic components can be, for example, active elements such as transistors or diodes, or passive elements such as capacitors. In addition, the multiple electronic components and the integrated circuits formed by integrating them can be formed by conventionally known methods.

[0181] The input unit 831 inputs the information used in the estimation device 803. For example, the input information I with an X-ray image obtained by using the terminal device 802 is input to the input unit 831. The input unit 831 has a communication unit, and the input information I obtained by using the terminal device 802 is directly input from the terminal device 802. In addition, the input unit 831 may include an input device capable of inputting the input information I or other information. The input device can be, for example, a keyboard, a touch screen, or a mouse, etc.

[0182] Based on the information input to the input unit 831, the approximator 832 estimates the future bone mass or bone density of the subject. The approximator 832 has AI (Artificial Intelligence). The approximator 832 has a program that functions as AI, and various electronic components and circuits for executing this program. The approximator 832 has a neural network.

[0183] The approximator 832 performs a learning process in advance for the input-output relationship. That is, machine learning can be applied to the approximator 832 using learning data and training data, so that the approximator 832 can calculate the estimation result O based on the input information I. In addition, the learning data or training data can be the input information I input to the estimation device 803 and the data corresponding to the estimation result O output from the estimation device 803.

[0184] Figure 15 Conceptually shows the structure of the approximator 832 of the present disclosure.

[0185] The approximator 832 has a first neural network 8321 and a second neural network 8322. The first neural network 8321 may be any neural network suitable for processing temporal information. For example, the first neural network 8321 may be a ConvLSTM network that combines LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network), etc. The second neural network 8322 may be, for example, a convolutional neural network constituted by using CNN, etc.

[0186] The first neural network 8321 has an encoding unit E and a decoding unit D. The encoding unit E can extract the temporal changes and the feature amounts of the position information of the input information I. The decoding unit D can calculate new feature amounts based on the feature amounts extracted by the encoding unit E, the temporal changes of the input information I, and the initial values.

[0187] Figure 16 Concept of the structure of the first neural network 8321 of the present disclosure is shown.

[0188] The encoding unit E has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers E1. The decoding unit D has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers D1. Each of the encoding unit E and the decoding unit D may have three or more ConvLSTM layers E1, D1. In addition, the number of the plurality of ConvLSTM layers E1 and the number of the plurality of ConvLSTM layers D1 may be the same number.

[0189] In addition, the content learned by each of the plurality of ConvLSTM layers E1 may be different. The content learned by each of the plurality of ConvLSTM layers D1 may be different. For example, for a certain ConvLTSM layer, fine content such as the change of each pixel is learned, and for another ConvLSTM layer, rough content such as the change of the entire image is learned.

[0190] Figure 17 Concept of the structure of the second neural network 8322 is shown.

[0191] The second neural network 8322 has a transformation unit C. The transformation unit C can transform the feature amount calculated by the decoding unit D of the first neural network 8321 into bone mass or bone density. The transformation unit C has a plurality of convolutional layers C1, a plurality of pooling layers C2, and a fully connected layer C3. The fully connected layer C3 is at the stage preceding the output unit 33. Moreover, in the transformation unit C, the convolutional layer C1 and the pooling layer C2 are alternately arranged between the first neural network 8321 and the fully connected layer C3.

[0192] In addition, during the learning of the approximator 832, the learning data is input to the encoding unit E of the approximator 832. The training data is compared with the output data output from the transformation unit C of the approximator 832 during the learning of the approximator 832. The training data is data representing values measured using a conventional bone density measurement device.

[0193] The output unit 833 can display the estimation result O. The output unit 833 is, for example, a liquid crystal display or an organic EL display. The output unit 833 can display various information such as characters, symbols, and graphics. The output unit 833 can, for example, display numbers or images.

[0194] The estimation device 803 of the present disclosure further has a control unit 834 and a storage unit 835. The control unit 834 can uniformly manage the operation of the estimation device 803 by controlling other structural elements of the estimation device 803.

[0195] The control unit 834 has, for example, a processor. The processor can include, for example, one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, or combinations of these devices or any calibrated combinations, or other basic devices or calibrated combinations. The control unit 834 includes, for example, a CPU.

[0196] The storage unit 835 includes, for example, a non - transient storage medium such as a RAM (Random Access Memory) or a ROM (Read - Only Memory) that can be read by the CPU of the control unit 834. A control program for controlling the estimation device 803 such as firmware is stored in the storage unit 835. In addition, the input information I input, the learning data for learning, and the training data can also be stored in the storage unit 835.

[0197] The processor of the control unit 834 can execute one or more data calculation programs or processes according to the control program in the storage unit 835. Various functions of the control unit 834 are realized by the CPU of the control unit 834 executing the control program in the storage unit 835.

[0198] In addition, the control unit 834 can also perform other processes as preprocessing of calculation processing as needed.

[0199] <Example of input information, learning data, and training data>

[0200] The input information (hereinafter, also referred to as the first input information I1) has image data of a bone of an object whose bone mass or bone density is to be estimated. The image data can be, for example, a simple X-ray image. The object whose bone mass or bone density is to be estimated is, for example, a human. In this case, it can be said that the first input information I1 is image data of a simple X-ray image of a human bone. A simple X-ray image is a two-dimensional image and is also referred to as a general X-ray image or a Roentgen image.

[0201] The first input information I1 is preferably a simple X-ray image that is relatively easy to obtain, but is not limited thereto. For example, there are cases where bone mass or bone density can be estimated more accurately by using an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image as input information.

[0202] In addition, the object whose bone mass or bone density is to be estimated can also be other than a human. For example, the object whose bone mass or bone density is to be estimated can also be an animal such as a dog, a cat, or a horse. In addition, the bones to be targeted are mainly cortical bone and cancellous bone of biological origin, but the bones to be targeted can also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine, etc.

[0203] As the shooting part of the X-ray image, for example, it can be any of the neck, chest, waist, proximal part of the femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, or jaw joint. In addition, parts other than bones can also be photographed in the X-ray image. For example, in the case of a chest simple X-ray image, it can also include an image of the lungs and an image of the thoracic vertebrae. The X-ray image can be a front image of the target part taken from the front or a side image of the target part taken from the side.

[0204] The learning data or the training data can be any data corresponding to the input information I input to the estimation device 803 and the estimation result O output from the estimation device 803.

[0205] The learning data has the same type of information as the first input information I1. For example, if the first input information I1 is a simple X-ray image, the learning data only needs to have a simple X-ray image. Further, when the first input information I1 is a chest simple X-ray image, the learning data only needs to have a chest simple X-ray image.

[0206] The learning image data including multiple simple X-ray images of bones is included in the learning data. Among the shooting parts of the multiple learning image data, for example, at least one of the neck, chest, waist, proximal part of the femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, and jaw joint is included. In the learning data, it can include image data of some types among the 11 types of image data, or it can include all types of image data. In addition, among the multiple learning image data, it can include front images and side images.

[0207] In the learning data, bones of multiple different individuals are respectively photographed. For each of the multiple learning image data, as training data, the measured values of the bone mass or bone density of the subject of each learning image data are established corresponding. The measured values of the bone mass or bone density are the measured values measured at a time period almost the same as the time period when the learning image data was photographed.

[0208] In addition, the learning image data of the learning data can also be a series of data with different time axes of photographing the same person. That is, the learning image data is the first learning data with an X-ray image of a bone and an image of the same person as the first learning data, and can also include the second learning data with an X-ray image photographed after the first learning data.

[0209] In addition, the learning image data of the learning data can also be a data set of photographing the same part of others with different ages, etc. In addition, the learning image data of the learning data can also be a series of data with different time axes of photographing the same person and the same part.

[0210] The learning data and the first input information I1 can also use the input information obtained by compressing the image data representing the gray scale of the simple X-ray image (in other words, the general X-ray imaging device or the Roentgen imaging device) and reducing the number of gray scales. For example, consider the case where the number of pixel data of the image data is more than (1024×640), and the number of bits of the pixel data is 16 bits. In this case, the input information obtained by compressing the number of pixel data to, for example, (256×256), (1024×512), or (1024×640) and reducing the number of bits of the pixel data to 8 bits is used as the first input information I1 and the learning data.

[0211] In the training data, for each of the plurality of learning image data included in the learning data, it includes a measured value of bone mass or bone density of bones photographed in the learning simple X-ray image represented by the learning image data. The bone mass or bone density can be measured, for example, by the DEXA (dual-energy X-ray absorptiometry) method or the ultrasonic method.

[0212] <Example of neural network learning>

[0213] The control unit 834 performs machine learning using the learning data and the training data on the approximator 832 so that the approximator 832 can calculate an estimation result O related to bone mass or bone density according to the input information I. The approximator 832 is optimized by known machine learning using the training data. The approximator 832 performs operations based on the learning data input to the encoding unit E, and adjusts the variable parameters in the approximator 832 so that the difference between the pseudo-estimation result output from the transformation unit C and the training data is small.

[0214] Specifically, the control unit 834 inputs the learning data in the storage unit 835 into the encoding unit E. When the control unit 834 inputs the learning data into the encoding unit E, it inputs the plurality of pixel data constituting the learning image data into the plurality of artificial neurons constituting the encoding unit E respectively. Then, when the control unit 834 inputs the learning image data into the encoding unit E, it adjusts the parameters so that the error of the estimation result O output from the transformation unit C with respect to the measured value of bone mass or bone density corresponding to the learning image data becomes small. The adjusted parameters become the learned parameters and are stored in the storage unit 835.

[0215] As a method for adjusting the parameters, for example, the error backpropagation method can be adopted. Among the parameters, for example, the parameters used by the encoding unit E, the decoding unit D, and the transformation unit C are included. Specifically, among the parameters, the weighting coefficients used by the ConvLSTM layers of the encoding unit E and the decoding unit D, and the convolutional layer and the fully connected layer of the transformation unit C are included.

[0216] As a result, the approximator 832 performs operations based on the learned parameters on the input information I input to the encoding unit E, and outputs an estimation result O from the transformation unit C. When X-ray image data as the input information I is input to the encoding unit E, the plurality of pixel data constituting the image data are respectively input to the plurality of artificial neurons constituting the input unit 831. Then, the ConvLSTM layer, the convolutional layer, and the fully connected layer can perform operations using the weighting coefficients included in the learned parameters, and thus output the estimation result O.

[0217] As described above, in the estimation system 801, using the image data of the simple X-ray image, learning of the approximator 832 and estimation of bone mass or bone density in the approximator 832 are performed. Therefore, input information I can be input to the estimation system 801, and future bone mass or bone density can be output as an estimation result O.

[0218] The estimation result O of the estimation system 801 only needs to be the estimation result on a certain day in the future that is more future than the acquisition date of the input information I. For example, the estimation system 1 can estimate the bone mass or bone density from 3 months to 50 years after the shooting, and more preferably from 6 months to 10 years after the shooting.

[0219] The estimation result O only needs to be output as a value. For example, it can be represented by at least one of YAM (Young Adult Mean), T-score, and Z-score. For example, from the output unit 833, the estimated value represented as YAM can be output, or the estimated value represented as YAM, the estimated value represented as T-score, and the estimated value represented as Z-score can be output.

[0220] In addition, the estimation result O can also be output as an image. When the estimation result O is an image, for example, an image imitating an X-ray image can be displayed. In addition, an image imitating an X-ray image is an image that imitates an X-ray image. In addition, when using ConvLSTM to learn a series of data with different time axes for the same person and the same part, the time change of the image can be predicted. Thereby, a future image can be generated based on the X-ray image at one time point of another patient.

[0221] In the learning data and the input information I, in addition to bones, internal organs, muscles, fat, or blood vessels can also be photographed. Even in this case, highly accurate estimation can be performed.

[0222] The first input information I1 can also have the individual data (first individual data) of the subject. The so-called first individual data can be, for example, age information, gender information, height information, weight information, or fracture history, etc. As a result, highly accurate estimation can be performed.

[0223] The first input information I1 can also have the second individual data of the subject. The so-called second individual data can also include information such as blood pressure, lipids, cholesterol, neutral fat, and blood glucose level. As a result, highly accurate estimation can be performed.

[0224] The first input information I1 can also have the lifestyle information of the subject. The so-called lifestyle information can be information such as drinking habits, smoking habits, exercise habits, or eating habits, etc. As a result, highly accurate estimation can be performed.

[0225] The first input information I1 may also include the bone metabolism information of the subject. The so-called bone metabolism information can be, for example, the bone resorption ability or the bone formation ability. They can be measured, for example, by at least one of N-terminal telopeptide of type I collagen cross-linking (NTX), C-terminal telopeptide of type I collagen cross-linking (CTX), tartrate-resistant acid phosphatase (TRACP-5b), deoxypyridinoline (DPD) as bone resorption markers, bone-specific alkaline phosphatase (BAP), N-propeptide of type I collagen cross-linking (P1NP) as bone formation markers, and undercarboxylated osteocalcin (ucOC) as bone-related matrix markers. The bone resorption markers can be measured using serum or urine as the test body.

[0226] In the estimation system 801, as the input information I, the second input information I2 related to the future planned actions of the subject can also be further input. The so-called second input information I2 can be, for example, information related to the individual data that is planned to be improved or has been improved, or the lifestyle and exercise and eating habits that are planned to be improved or have been improved. Specifically, the second input information I2 can be information such as body weight data, drinking habits, smoking habits, sun exposure time, number of steps or walking distance per day, dairy product intake, or intake of foods rich in vitamin D such as improved fish and mushrooms. As a result, the estimation system 801 can present an estimated result O in which the future bone mass or bone density has been improved.

[0227] In addition, the second input information I2 can also be, for example, information related to the lifestyle that is expected to deteriorate. As a result, the estimation system can present an estimated result O in which the future bone mass or bone density deteriorates.

[0228] In the estimation system 801, as the input information I, the third input information I3 related to the therapy for the subject can also be further input. The so-called third input information I3 is, for example, information related to physical therapy or drug therapy. Specifically, the third input information I3 can be at least one of calcium tablets, female hormone drugs, vitamin drugs, bisphosphonate drugs, SERM (Selective Estrogen Receptor Modulator) drugs, calcitonin drugs, thyroid hormone drugs, and denosumab drugs.

[0229] In the estimation system 801, the estimation result O can also output the first result O1 based only on the first input information I1, and the second result O2 based on at least one of the first input information I1 and the second and third input information I2 and I3. As a result, it is possible to compare the effects of future planned actions.

[0230] In the estimation system 801, the estimation result O not only outputs the future bone mass or bone density, but can also output the current result. As a result, it is possible to compare the changes in bone mass or bone density over time.

[0231] Figure 18 Concept of the structure of the approximator 832 showing another embodiment of the estimation system 1.

[0232] The estimation device 803 of the estimation system 801 may also have a first approximator 832a and a second approximator 832b. That is, in addition to the above-described approximator 832 (first approximator 832a), a second approximator 832b may also be provided. The second approximator 832b may be a CNN, for example.

[0233] In this case, in the estimation system 801, the first approximator 832a outputs the first image and the first value as the first estimation result O1 to the first output unit 833a. Further, the second approximator 832b outputs the second value as the second estimation result O2 to the second output unit 833b based on the first image from the first output unit 833a. As a result, it is possible to compare the first value and the second value as the estimation result O of the future bone mass or bone density.

[0234] The estimation system 801 may also output a third value based on the first value and the second value as the estimation result O. As a result, for example, it is possible to set the result (third value) obtained by calibrating the first value based on the second value as the estimation result O.

[0235] As described in detail above for the estimation system 801, the above description is an example in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not contradict each other. Moreover, countless examples that are not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

Claims

1. A presumption device that can presume a presumption result related to the bone density of a subject based on a first image that is a frontal image of a simple X-ray image of the bones of an animal including a person, the presumption device comprising: A sensing unit that senses fractures of the bones photographed in the first image.

2. A presumption device comprising: A presumption unit that presumes the bone density of bones based on input information having a lateral image of a simple X-ray image including a person's skeleton, based on parameters that have completed learning.

3. A device comprising: An acquisition unit that acquires, via a communication network, input information including a lumbar simple X-ray image obtained by irradiating the lumbar region of a first person with X-rays; and An output unit that outputs the presumed bone density of the first person based on the input information.

4. A device comprising: An acquisition unit that acquires, via a communication network, input information including a chest simple X-ray image obtained by irradiating the chest of a first person with X-rays; and An output unit that outputs the presumed bone density of the first person based on the input information.

5. A device comprising: A processing unit that segments the first simple X-ray image based on input information including the first simple X-ray image of the first skeleton of the first person and parameters that have completed learning.

6. A method for generating a learning model that presumes, using a neural network, at least one presumed value of the bone density and bone mass of an object person based on input information including a presumed simple X-ray image of the object person, by setting the neural network based on parameters that have completed learning including learning data and training data, the learning data including learning simple X-ray images of at least one of the thigh, lumbar region, and chest of a person, and the training data including measured values of at least one of the bone density and bone mass measured in the same person as the person.

7. A presumption device comprising: A presumption unit that presumes at least one presumed value of the bone density and bone mass of bones based on input information including a first chest simple X-ray image of the chest of a first person, based on a neural network, the neural network including parameters that have completed learning based on learning data and training data, the learning data including a second chest simple X-ray image of a second person different from the first person, and the training data including measured values of at least one of the bone density and bone mass of the second person.

8. A presumption device comprising: A presumption unit that presumes at least one presumed value of the bone density and bone mass of bones based on input information including a first lumbar simple X-ray image of the lumbar region of a first person, based on a neural network, the neural network including parameters that have completed learning based on learning data and training data, the learning data including a second lumbar simple X-ray image of a second person different from the first person, and the training data including measured values of at least one of the bone density and bone mass of the second person.

9. A prediction system comprising: a receiving unit that receives at least one estimated value of a bone density and a bone mass estimated based on a first simple X-ray image for estimation that captures a target part of a first person and based on a first learned parameter; and The bone fracture prediction unit predicts a bone fracture of the first person based on the estimated value.

Images