Estimation device, estimation system, estimation method, program product, and recording medium

Abstract

The invention provides an estimation device, an estimation system, an estimation method, a program product, and a recording medium. The estimation device includes an input unit and an approximator. The input unit inputs input information having an image in which a bone has been captured. The approximator is capable of estimating an estimation result relating to the bone mineral density of the bone on the basis of the input information input to the input unit. The approximator has a parameter for obtaining a learning completion of an estimation result relating to the bone mineral density of the bone from input information.

Description

[0001] This application is a divisional application of a patent application for invention titled "Estimation Device, Estimation System, and Computer-Readable Non-Temporary Recording Medium Recording an Estimation Program", with an application date of September 10, 2019, an application number of 201980058486.5.

[0002] Cross-reference to related applications

[0003] This application claims the priority of Japanese Patent Application No. 2018-168502 (filed on September 10, 2018) and Japanese National Application No. 2018-220401 (filed on November 26, 2018), and the entire disclosure of that application is incorporated herein by reference for that purpose. Technical field

[0004] This disclosure relates to the estimation of bone density. Background art

[0005] In Patent Document 1, a technique for diagnosing osteoporosis is described. In Patent Document 2, a technique for estimating the strength of bones is described.

[0006] Prior art documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2008-36068

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2002-523204 Summary of the invention

[0010] An estimation device, an estimation system, and an estimation program are disclosed. In one embodiment, the estimation device includes: an input unit that inputs input information having an image of a bone captured; an approximator that can estimate an estimation result associated with the bone density of the bone based on the input information input to the input unit; and an output unit that outputs the estimation result estimated by the approximator, and the approximator has parameters that have been learned for obtaining an estimation result associated with the bone density of the bone based on the input information.

[0011] Furthermore, in one embodiment, the estimation system includes: an input unit that inputs input information having an image of a bone captured; and an approximator that has parameters that have been learned for obtaining an estimation result associated with the bone density of the bone based on the input information and can estimate an estimation result associated with the bone density of the bone based on the input information input to the input unit, and when the input information is input to the input unit, the approximator performs arithmetic processing on the input information.

[0012] In addition, in one embodiment, the estimation program is an estimation program for causing the device to function as a neural network that performs operations based on parameters for which learning has been completed and outputs an estimated value of the bone density of the bone photographed in the image, and the parameters for which learning has been completed are used to obtain an estimation result associated with the bone density of the bone based on input information having an image in which the bone is photographed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. is an example showing the configuration of a computer device (estimation device).

[0014] Figure 2 FIG. is a diagram for explaining the operation of the estimation device.

[0015] Figure 3 FIG. is an example showing the configuration of a neural network.

[0016] Figure 4 FIG. is an example showing a state in which learning image data and reference bone density are mutually established.

[0017] Figure 5 FIG. is a diagram for explaining the operation of the estimation device.

[0018] Figure 6 FIG. is an example showing an estimation system.

[0019] Figure 7 FIG. is an example showing the configuration of the estimation device.

[0020] Figure 8 FIG. is an example showing the configuration of the estimation device.

[0021] Figure 9 FIG. is an example showing the configuration of the estimation device.

[0022] Figure 10 FIG. is an example showing the configuration of the estimation device.

[0023] Figure 11 FIG. is an example showing the configuration of the estimation device.

[0024] Figure 12 FIG. is an example showing the configuration of the estimation device.

[0025] Figure 13 FIG. is a conceptual diagram schematically showing the configuration of the estimation system.

[0026] Figure 14 FIG. is a conceptual diagram schematically showing the configuration of a part of the estimation system.

[0027] Figure 15 It is a conceptual diagram schematically showing the structure of a part of the estimation system.

[0028] Figure 16 It is a conceptual diagram schematically showing the structure of a part of the estimation system.

[0029] Figure 17 It is a conceptual diagram schematically showing the structure of a part of the estimation system.

[0030] Figure 18 It is a conceptual diagram schematically showing the structure of a part of another embodiment of the estimation system.

[0031] Symbol Explanation

[0032] 1: Computer device (estimation device);

[0033] 20: Storage unit;

[0034] 100: Control program;

[0035] 110, 910, 960: Learned parameters;

[0036] 120: Estimation data;

[0037] 130: Learning data;

[0038] 140: Training data;

[0039] 200: Neural network;

[0040] 210: Input layer;

[0041] 230: Output layer;

[0042] 280, 832: Approximator;

[0043] 500: Processing device;

[0044] 600: Bone density estimation system;

[0045] 801: Estimation system;

[0046] 802: Terminal device;

[0047] 803: Estimation device;

[0048] 831: Input unit;

[0049] 833: Output unit;

[0050] 834: Control unit;

[0051] 835: Storage unit;

[0052] 900, 8322: Second neural network;

[0053] 930: Determination unit;

[0054] 950: Third neural network;

[0055] 980: Fracture prediction unit;

[0056] 8321: First neural network;

[0057] O: Estimation result;

[0058] I: Input information;

[0059] E: Encoding unit;

[0060] D: Decoding unit;

[0061] C: Transformation unit. Detailed implementation

[0062] Embodiment 1.

[0063] Figure 1 is a block diagram showing an example of the structure of the computer device 1 according to Embodiment 1. The computer device 1 functions as an estimation device for estimating bone density. Hereinafter, the computer device 1 may sometimes be referred to as the "estimation device 1".

[0064] As Figure 1 shown, the estimation device 1 includes, for example, a control unit 10, a storage unit 20, a communication unit 30, a display unit 40, and an input unit 50. The control unit 10, the storage unit 20, the communication unit 30, the display unit 40, and the input unit 50 are electrically connected to each other through a bus 60, for example.

[0065] The control unit 10 can manage the operation of the estimation device 1 uniformly by controlling other structural elements of the estimation device 1. The control unit 10 is also referred to as a control device or a control circuit. As will be described in further detail below, the control unit 10 includes at least one processor in order to provide control and processing capabilities for executing various functions.

[0066] According to various embodiments, at least one processor may also be implemented as a single integrated circuit (IC) or as multiple communicably connected integrated circuits (ICs) and / or discrete circuits. At least one processor can be implemented according to various known techniques.

[0067] In one embodiment, a processor includes, for example, one or more circuits or units configured to execute one or more data calculation programs or processes by executing instructions stored in an associated memory. In another embodiment, the processor may also be firmware (e.g., discrete logic elements) configured to execute one or more data calculation programs or processes.

[0068] According to various embodiments, the processor may also include one or more processors, controllers, microprocessors, microcontrollers, application-specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, field-programmable logic devices, or any combination of these devices or structures, or a combination of other known devices and structures, and perform the functions described below. In this example, the control unit 10 includes, for example, a CPU (Central Processing Unit).

[0069] The storage unit 20 includes non-transitory storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory) that can be read by the CPU of the control unit 10. A control program 100 for controlling the estimation device 1 is stored in the storage unit 20. Various functions of the control unit 10 are implemented by the CPU of the control unit 10 executing the control program 100 in the storage unit 20. It can also be said that the control program 100 is a bone density estimation program for causing the computer device 1 to function as an estimation device. In this example, by the control unit 10 executing the control program 100 in the storage unit 20, as Figure 2 shown, an approximator 280 capable of outputting an estimated value 300 of bone density is formed in the control unit 10. The approximator 280 includes, for example, a neural network 200. The control program 100 can also be said to be a program for causing the computer device 1 to function as the neural network 200. Thereafter, the estimated value of bone density is sometimes referred to as the "bone density estimated value". In addition, a structural example of the neural network 200 will be described in detail later.

[0070] In addition to the control program 100, learned parameters 110 related to the neural network 200, estimation data 120 (hereinafter, also referred to as "input information"), learning data 130, and training data 140 are also stored in the storage unit 20. The learning data 130 and the training data 140 are data used when the neural network 200 is being learned. The learned parameters 110 and the estimation data 120 are data used by the learned neural network 200 when estimating bone density.

[0071] The learning data 130 is the data input to the input layer 210 of the neural network 200 during the learning of the neural network 200. The learning data 130 is also referred to as learning data. The training data 140 is the data representing the correct value of bone density. The training data 140 is compared with the output data output from the output layer 230 of the neural network 200 during the learning of the neural network 200. The learning data 130 and the training data 140 are sometimes collectively referred to as the learning data with training.

[0072] The data 120 for estimation is the data input to the input layer 210 of the neural network 200 when the bone density is estimated by the learned neural network 200. The learned parameter 110 is the learned parameter within the neural network 200. The learned parameter 110 can also be referred to as the parameter adjusted by the learning of the neural network 200. Among the learned parameters 110, it includes the weighting coefficients representing the weights of the connections between artificial neurons. As shown in Figure 2 the learned neural network 200 performs operations based on the learned parameter 110 for the data 120 for estimation input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230.

[0073] In addition, the data input to the input layer 210 can be input to the input layer 210 either via the input unit 50 or directly to the input layer 210. In the case of directly inputting to the input layer 210, the input layer 210 can also be set as part or all of the input unit 50. After that, the bone density estimation value 300 is sometimes referred to as the estimation result 300.

[0074] The communication unit 30 is connected to a communication network including the Internet or the like by wire or wirelessly. The communication unit 30 can communicate with other devices such as a cloud server and a network server through the communication network. The communication unit 30 can input the information received from the communication network to the control unit 10. In addition, the communication unit 30 can output the information received from the control unit 10 to the communication network.

[0075] The display unit 40 is, for example, a liquid crystal display or an organic EL display. The display unit 40 can display various information such as characters, symbols, and graphics under the control of the control unit 10.

[0076] The input unit 50 can accept the input from the user to the estimation device 1. The input unit 50 includes, for example, a keyboard and a mouse. The input unit 50 can also include a touch screen capable of detecting the operation of the user on the display surface of the display unit 40.

[0077] In addition, the structure of the estimation device 1 is not limited to the above examples. For example, the control unit 10 may also include multiple CPUs. In addition, the control unit 10 may also include at least one DSP. In addition, all functions of the control unit 10 or some of the functions of the control unit 10 may also be implemented by a hardware circuit that does not require software for the implementation of its functions. In addition, the storage unit 20 may also include a non-transitory storage medium readable by a computer other than a ROM and a RAM. For example, the storage unit 20 may also include a small hard disk drive and an SSD (Solid State Drive), etc. In addition, the storage unit 20 may also include a memory such as a USB (Universal Serial Bus) memory that can be plugged into and unplugged from the estimation device 1. Hereinafter, a memory that can be plugged into and unplugged from the estimation device 1 is sometimes referred to as a "plug-in memory".

[0078] <Example of the structure of the neural network>

[0079] Figure 3 FIG. is an example showing the structure of the neural network 200. In this example, the neural network 200 is, for example, a convolutional neural network (CNN (Convolutional Neural Network)). As Figure 3 shown, the neural network 200 includes, for example, an input layer 210, a hidden layer 220, and an output layer 230. The hidden layer 220 is also referred to as an intermediate layer. The hidden layer 220 includes, for example, multiple convolutional layers 240, multiple pooling layers 250, and a fully connected layer 260. In the neural network 200, a fully connected layer 260 exists in front of the output layer 230. Moreover, in the neural network 200, the convolutional layer 240 and the pooling layer 250 are alternately arranged between the input layer 210 and the fully connected layer 260.

[0080] In addition, the structure of the neural network 200 is not limited to Figure 3 the example. For example, the neural network 200 may also include one convolutional layer 240 and one pooling layer 250 between the input layer 210 and the fully connected layer 260. In addition, the neural network 200 may also be a neural network other than a convolutional neural network.

[0081] <An example of the data for estimation, the data for learning, and the training data>

[0082] The data 120 for estimation includes image data of a simple X-ray image of a bone that is the object of bone density estimation. The object of bone density estimation is, for example, a human. Therefore, it can be said that the data 120 for estimation includes image data of a simple X-ray image of a human bone. In the data 130 for learning, it includes image data of a plurality of simple X-ray images of human bones. A simple X-ray image is also referred to as a two-dimensional image, a general X-ray image, or a Roentgen image. In addition, the object of bone density estimation may be other than a human. For example, the object of bone density estimation may be an animal such as a dog, a cat, or a horse. Furthermore, the bone set as the object is mainly cortical bone and cancellous bone from a living body, but among the bones set as the object, it may also include an artificial bone mainly composed of calcium phosphate or a regenerated bone artificially manufactured through regenerative medicine or the like.

[0083] After that, sometimes the image data included in the data 120 for estimation is referred to as "estimation image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 120 for estimation is referred to as "estimation simple X-ray image". In addition, sometimes the image data included in the data 130 for learning is referred to as "learning image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 130 for learning is referred to as "learning simple X-ray image". In the data 130 for learning, it includes a plurality of learning X-ray image data respectively representing a plurality of learning simple X-ray images.

[0084] As the imaging region for the simple X-ray image for estimation, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers or jaw joint can be used. In other words, as the estimation data 120, the following can be used: the image data of the simple X-ray image obtained by irradiating the head with X-rays, the image data of the simple X-ray image obtained by irradiating the neck with X-rays, the image data of the simple X-ray image obtained by irradiating the chest with X-rays, the image data of the simple X-ray image obtained by irradiating the waist with X-rays, the image data of the simple X-ray image obtained by irradiating the hip joint with X-rays, the image data of the simple X-ray image obtained by irradiating the knee joint with X-rays, the image data of the simple X-ray image obtained by irradiating the ankle joint with X-rays, the image data of the simple X-ray image obtained by irradiating the foot with X-rays, the image data of the simple X-ray image obtained by irradiating the toes with X-rays, the image data of the simple X-ray image obtained by irradiating the shoulder joint with X-rays, the image data of the simple X-ray image obtained by irradiating the elbow joint with X-rays, the image data of the simple X-ray image obtained by irradiating the wrist joint with X-rays, the image data of the simple X-ray image obtained by irradiating the hand with X-rays, the image data of the simple X-ray image obtained by irradiating the fingers with X-rays or the image data of the simple X-ray image obtained by irradiating the jaw joint with X-rays. Among the simple X-ray images obtained by irradiating the chest with X-rays, there are the simple X-ray images that capture the lungs and the simple X-ray images that capture the thoracic vertebrae. In addition, the types of the imaging regions for the simple X-ray image for estimation are not limited to this. Furthermore, the simple X-ray image for estimation can be either a front image of the target region captured from the front or a side image of the target region captured from the side.

[0085] Among the imaging regions of the multiple learning X-ray images respectively represented by the multiple pieces of learning data 130, there is included at least one of, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers, and jaw joint. In other words, in the learning data 130, there is included at least one of the 15 types of image data, namely, the image data of a simple X-ray image obtained by irradiating the head with X-rays, the image data of a simple X-ray image obtained by irradiating the neck with X-rays, the image data of a simple X-ray image obtained by irradiating the chest with X-rays, the image data of a simple X-ray image obtained by irradiating the waist with X-rays, the image data of a simple X-ray image obtained by irradiating the hip joint with X-rays, the image data of a simple X-ray image obtained by irradiating the knee joint with X-rays, the image data of a simple X-ray image obtained by irradiating the ankle joint with X-rays, the image data of a simple X-ray image obtained by irradiating the foot with X-rays, the image data of a simple X-ray image obtained by irradiating the toes with X-rays, the image data of a simple X-ray image obtained by irradiating the shoulder joint with X-rays, the image data of a simple X-ray image obtained by irradiating the elbow joint with X-rays, the image data of a simple X-ray image obtained by irradiating the wrist joint with X-rays, the image data of a simple X-ray image obtained by irradiating the hand with X-rays, the image data of a simple X-ray image obtained by irradiating the fingers with X-rays, and the image data of a simple X-ray image obtained by irradiating the jaw joint with X-rays. In the learning data 130, there may be included image data of some of the 15 types of image data or all types of image data. In addition, the types of imaging regions of the learning simple X-ray images are not limited to this. Further, among the multiple learning simple X-ray images, there may be included a frontal image or a lateral image. Further, among the multiple learning simple X-ray images, there may also be included both a frontal image and a lateral image of the same imaging region.

[0086] In the training data 140, for each of the multiple learning image data included in the learning data 130, it includes the measured value of the bone density of a person having bones photographed in the learning simple X-ray image represented by the learning image data. Among the multiple measured values of bone density included in the training data 140, for example, it includes at least one of the measured value of bone density measured by irradiating X-rays to the lumbar vertebra, the bone density measured by irradiating X-rays to the proximal part of the femur, the bone density measured by irradiating X-rays to the radius, the bone density measured by irradiating X-rays to the metacarpal bone, the bone density measured by applying ultrasonic waves to the wrist, and the bone density measured by applying ultrasonic waves to the heel. Thereafter, the measured value of bone density included in the training data 140 is sometimes referred to as the "reference bone density".

[0087] Here, as a method for measuring bone density, the DEXA (dual-energy X-ray absorptiometry) method is known. In a DEXA device that uses the DEXA method to measure bone density, when measuring the bone density of the lumbar vertebra, X-rays (specifically, two types of X-rays) are irradiated to the lumbar vertebra from its front. In addition, in the DEXA device, when measuring the bone density of the proximal part of the femur, X-rays are irradiated to the proximal part of the femur from its front.

[0088] In the training data 140, it may include the bone density of the lumbar vertebra measured by the DEXA device and may also include the bone density of the proximal part of the femur measured by the DEXA device. In addition, in the training data 140, it may also include the bone density measured by irradiating X-rays to the target part from its side. For example, in the training data 140, it may also include the bone density measured by irradiating X-rays to the lumbar vertebra from its side.

[0089] In addition, as another method for measuring bone density, the ultrasonic wave method is known. In a device that uses the ultrasonic wave method to measure bone density, for example, ultrasonic waves are applied to the wrist to measure the bone density of the wrist, and ultrasonic waves are applied to the heel to measure the bone density of the heel. In the training data 140, it may also include the bone density measured by the ultrasonic wave method.

[0090] In the multiple learning simple X-ray images represented by the multiple learning image data included in the learning data 130, bones of multiple different people are photographed. And, as Figure 4As shown, in the storage unit 20, for each of the multiple pieces of learning image data included in the learning data 130, a correspondence is established with the reference bone density of a person who has bones photographed in the learning simple X-ray image represented by the learning image data. In other words, for each of the multiple learning simple X-ray images used in the learning of the neural network 200, a correspondence is established with the reference bone density of a person who has bones photographed in the learning simple X-ray image. The reference bone density corresponding to the learning image data is the bone density measured for the same person as the person who has bones photographed in the learning simple X-ray image represented by the learning image data at a time almost the same as the time when the learning simple X-ray image was photographed.

[0091] In the part photographed in the learning simple X-ray image represented by the learning image data (in other words, the photographing part of the learning simple X-ray image), the part (i.e., the bone) for which the reference bone density corresponding to the learning image data is measured may or may not be included. In other words, in the part photographed in the learning simple X-ray image, the part for which the reference bone density corresponding to the learning simple X-ray image is measured may or may not be included. As an example of the former, consider the case where a correspondence is established between the learning image data representing the learning simple X-ray image of the photographed waist and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the photographed hip joint and the reference bone density of the proximal part of the femur. On the other hand, as an example of the latter, consider the case where a correspondence is established between the learning image data of the photographed chest and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the photographed knee joint and the reference bone density of the heel.

[0092] In addition, the orientation of the part imaged in the learning-only X-ray image represented by the image data for learning and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning image data may be the same or different. In other words, the orientation of the part imaged in the learning-only X-ray image and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning-only X-ray image may be the same or different. As an example of the former, consider the case of establishing correspondence between the learning image data representing a learning-only X-ray image of the chest imaged from the front (hereinafter sometimes referred to as a "front chest learning-only X-ray image") and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing correspondence between the learning image data representing a learning-only X-ray image of the waist imaged from the front (hereinafter sometimes referred to as a "front waist learning-only X-ray image") and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front. On the other hand, as an example of the latter, consider the case of establishing correspondence between the learning image data representing a learning-only X-ray image of the waist imaged from the side (hereinafter sometimes referred to as a "side waist learning-only X-ray image") and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing correspondence between the learning image data representing a learning-only X-ray image of the knee joint imaged from the side (hereinafter sometimes referred to as a "side knee learning-only X-ray image") and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front.

[0093] In addition, among the multiple learning-only X-ray images respectively represented by the multiple pieces of learning image data included in the learning data 130, there may be included a learning-only X-ray image that images the same type of part as the estimation-only X-ray image, or a learning-only X-ray image that images a different type of part from the estimation-only X-ray image. As an example of the former, consider the case where the estimation-only X-ray image is a front chest learning-only X-ray image and a front chest learning-only X-ray image is included among the multiple learning-only X-ray images. As another example, consider the case where the estimation-only X-ray image is a learning-only X-ray image of the knee joint imaged from the front (hereinafter sometimes referred to as a "front knee learning-only X-ray image") and a side knee learning-only X-ray image is included among the multiple learning-only X-ray images. On the other hand, as an example of the latter, consider the case where the estimation-only X-ray image is a front waist learning-only X-ray image and a front chest learning-only X-ray image is included among the multiple learning-only X-ray images. As another example, consider the case where the estimation-only X-ray image is a side waist learning-only X-ray image and a front knee learning-only X-ray image is included among the multiple learning-only X-ray images.

[0094] In addition, among the multiple learning plain X-ray images, there may be included plain X-ray images of parts taken in the same orientation as the estimation plain X-ray image, or plain X-ray images of parts taken in an orientation different from that of the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar anterior plain X-ray image, and a lumbar anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the estimation plain X-ray image is a knee anterior plain X-ray image, and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a knee lateral plain X-ray image, and a knee anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the estimation plain X-ray image is a lumbar lateral plain X-ray image, and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images.

[0095] In addition, in the training data 140, there may be included reference bone densities measured from parts (bones) included in the part imaged in the estimation plain X-ray image, or reference bone densities measured from parts (bones) not included in the part imaged in the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar anterior plain X-ray image, and the reference bone density of the lumbar vertebra is included in the training data 140. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a chest anterior plain X-ray image, and the reference bone density of the metacarpal head is included in the training data 140.

[0096] In addition, in the training data 140, there may be included reference bone densities measured by irradiating the target part with X-rays from the same orientation as the orientation of the part imaged in the estimation plain X-ray image, or reference bone densities measured by irradiating the target part with X-rays from an orientation different from that of the part imaged in the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar anterior plain X-ray image, and the reference bone density measured by irradiating the lumbar vertebra with X-rays from its anterior side is included in the training data 140. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a lumbar lateral plain X-ray image, and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its anterior side is included in the training data 140.

[0097] In this example, image data representing the gray scale of a simple X-ray image obtained by a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device) is compressed, and the image data obtained by reducing the number of gray scales is used as learning image data and estimation image data. For example, when the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is more than (1024×640) and the number of bits of the pixel data is 16 bits. In this case, the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is compressed to, for example, (256×256), (1024×512), or (1024×640), and the number of bits of the pixel data is reduced to 8 bits, and then used as learning image data and estimation image data. In this case, each of the learning simple X-ray image and the estimation simple X-ray image includes (256×256), (1024×512), or (1024×640) pixels, and the value of the pixel is expressed in 8 bits.

[0098] For the learning image data and the estimation image data, they can be generated by the control unit 10 of the estimation device 1 based on the image data obtained by the simple X-ray imaging device, or can be generated by a device other than the estimation device 1 based on the image data obtained by the simple X-ray imaging device. In the former case, for the image data obtained by the simple X-ray imaging device, it can be received by the communication unit 30 through the communication network, or can be stored in the plug-in memory included in the storage unit 20. In the latter case, it can also be that the communication unit 30 receives the learning image data and the estimation image data from other devices through the communication network, and the control unit 10 stores the learning image data and the estimation image data received by the communication unit 30 in the storage unit 20. Or, the learning image data and the estimation image data generated by other devices can be stored in the plug-in memory included in the storage unit 20. In addition, for the training data 140, it can also be that the communication unit 30 receives it through the communication network, and the control unit 10 stores the training data 140 received by the communication unit 30 in the storage unit 20. Or, the training data 140 can be stored in the plug-in memory included in the storage unit 20. In addition, the number and number of bits of the pixel data of the learning image data and the estimation image data are not limited to the above.

[0099] <Example of learning of neural network>

[0100] Figure 5 is a diagram for explaining an example of the learning of the neural network 200. When the control unit 10 performs the learning of the neural network 200, as Figure 5As shown, learning data 130 is input to the input layer 210 of the neural network 200. Then, the control unit 10 adjusts the variable parameter 110a within the neural network 200 so that the error of the output data 400 output from the output layer 230 of the neural network 200 with respect to the training data 140 is small. More specifically, the control unit 10 inputs each piece of learning image data within the storage unit 20 to the input layer 210. When inputting the learning image data to the input layer 210, the control unit 10 inputs the multiple pixel data constituting the learning image data to the multiple artificial neurons constituting the input layer 210 respectively. Then, when the learning image data is input to the input layer 210, the control unit 10 adjusts the parameter 110a so that the error of the output data 400 output from the output layer 230 with respect to the reference bone density corresponding to the learning image data is small. As a method for adjusting the parameter 110a, for example, the error backpropagation method can be adopted. The adjusted parameter 110a becomes the learned parameter 110 and is stored in the storage unit 20. In the parameter 110a, for example, it includes the parameters used in the hidden layer 220. Specifically, the parameter 110a includes the filter coefficients used in the convolutional layer 240 and the weighting coefficients used in the fully connected layer 260. In addition, the method for adjusting the parameter 110a, in other words, the learning method of the parameter 110a is not limited to this.

[0101] In this way, in the storage unit 20, the learned parameter 110 that has learned the relationship between the learning data 130 of the image data including multiple learning simple X-ray images and the measured values of the bone density as the training data 140 using the neural network 200 is stored.

[0102] In the above example, the estimation device 1 performs the learning of the neural network 200, but other devices can also perform the learning of the neural network 200. In this case, the learned parameter 110 generated by other devices is stored in the storage unit 20 of the estimation device 1. In addition, it becomes unnecessary to store the learning data 130 and the training data 140 in the storage unit 20. For the learned parameter 110 generated by other devices, it can also be received by the communication unit 30 through the communication network, and the control unit 10 stores the learned parameter 110 received by the communication unit 30 in the storage unit 20. Alternatively, the learned parameter 110 generated by other devices can be stored in the pluggable memory included in the storage unit 20.

[0103] In the neural network 200 learned as above, the image data of multiple learning simple X-ray images is input to the input layer 210 as the learning data 130, and it includes the learned parameter 110a learned using the reference bone density as the training data 140. As described above Figure 2As shown, the neural network 200 performs operations based on the learned parameters 110a on the estimation data 120 input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230. When the estimation image data as the estimation data 120 is input to the input layer 210, the multiple pixel data constituting the estimation image data are respectively input to the multiple artificial neurons constituting the input layer 210. Then, the convolutional layer 240 performs operations using the filter coefficients included in the learned parameters 110a, and the fully connected layer 260 performs operations using the weighting coefficients included in the learned parameters 110a.

[0104] For example, when the estimation image data representing a frontal chest plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the bones of the chest photographed in the frontal chest plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a frontal lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the lumbar vertebrae included in the lumbar region photographed in the frontal lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a lateral lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the lumbar vertebrae included in the lumbar region photographed in the lateral lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a frontal knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the bones of the knee joint photographed in the frontal knee plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing a lateral knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of a person with the bones of the knee joint photographed in the lateral knee plain X-ray image represented by the estimation image data is output from the output layer 230.

[0105] The estimation value 300 output from the output layer 230 can be represented by at least one of the bone mineral density per unit area (g / cm 2 ), the bone mineral density per unit volume (g / cm 3 ), YAM, T-score, and Z-score. YAM is an abbreviation for "Young Adult Mean" and is sometimes referred to as the young adult average percentage. For example, from the output layer 230, it is possible to output the representation as the bone mineral density per unit area (g / cm 2The estimated value 300 expressed as YAM and the estimated value 300 expressed as YAM can also output the estimated value 300 expressed as YAM, the estimated value 300 expressed as T-score, and the estimated value 300 expressed as Z-score.

[0106] In addition, the storage unit 20 may also store a plurality of estimation data 120. In this case, among the plurality of estimation-only X-ray images respectively represented by the plurality of estimation data 120 in the storage unit 20, there may be included a plurality of X-ray images of the same type of part, or a plurality of X-ray images of parts of different types. In addition, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images of a part taken from the same direction, or a plurality of X-ray images of a part taken from different directions. In other words, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images in which the orientation of the part captured therein is the same, or a plurality of X-ray images in which the orientation of the part captured therein is different. The control unit 10 inputs each of the plurality of estimation data 120 in the storage unit 20 to the input layer 210 of the neural network 200, and outputs a bone density estimated value 300 corresponding to each estimation data 120 from the output layer 230 of the neural network 200.

[0107] As described above, in this example, the image data of the plain X-ray image is used for the learning of the neural network 200 and the estimation of the bone density in the neural network 200. For the image data of the plain X-ray image, in other words, the image data of the Roentgen image, since it is used in various examinations and the like in many hospitals, it can be easily obtained. Therefore, it is possible to simply estimate the bone density without using an expensive device such as a DEXA device.

[0108] In addition, by using the image data of the plain X-ray image taken for examinations and the like as the estimation image data, it is possible to simply estimate the bone density by taking advantage of the opportunity of such examinations. Thus, by using the estimation device 1, the service for hospital users can be improved.

[0109] In addition, in a frontal plain X-ray image of the chest or the like, it may be affected by internal organs, making it difficult to capture bones. On the other hand, the possibility of taking a frontal plain X-ray image is relatively high in many hospitals. In this example, even when using a frontal plain X-ray image in which it may be difficult to capture bones as a presumptive plain X-ray image or a learning plain X-ray image, bone density can still be estimated. Therefore, it is possible to simply estimate bone density using the image data of the easily obtainable frontal plain X-ray image. In addition, for a frontal chest plain X-ray image, it is often taken in health examinations and the like, and can be said to be a particularly easily obtainable plain X-ray image. By using the frontal chest plain X-ray image as a presumptive plain X-ray image or a learning plain X-ray image, bone density can be estimated even more simply.

[0110] In addition, in this example, even when the learning plain X-ray images include a plain X-ray image that captures a part of a different type from the presumptive plain X-ray image, bone density can be estimated based on the image data of the presumptive plain X-ray image. Thereby, the convenience of the estimation device 1 (in other words, the computer device 1) can be improved.

[0111] In addition, in this example, even when the learning plain X-ray images include a plain X-ray image that captures a part in a different orientation from the presumptive plain X-ray image, bone density can be estimated based on the image data of the presumptive plain X-ray image. Thereby, the convenience of the estimation device 1 can be improved.

[0112] In addition, in this example, even when the part captured in the learning plain X-ray image does not include the part (bone) for which the reference bone density corresponding to the learning plain X-ray image has been measured, the neural network 200 can estimate the bone density based on the learned parameters 110. Thereby, the convenience of the estimation device 1 can be improved.

[0113] In addition, in this example, even when the orientation of the part captured in the learning plain X-ray image and the orientation of the X-ray irradiation of the object part in the measurement of the reference bone density corresponding to the learning plain X-ray image are different from each other, the neural network 200 can estimate the bone density based on the learned parameters 110. Thereby, the convenience of the estimation device 1 can be improved.

[0114] In addition, in this example, even when the training data 140 includes the reference bone density measured from a part that is not included in the part captured in the presumptive plain X-ray image, bone density can be estimated based on the image data of the presumptive plain X-ray image. Thereby, the convenience of the estimation device 1 can be improved.

[0115] In addition, the bone density estimation value 300 obtained by the estimation device 1 can also be displayed by the display unit 40. Furthermore, the bone density estimation value 300 obtained by the estimation device 1 can also be used by other devices.

[0116] Figure 6 FIG. is an example of a bone density estimation system 600 including an estimation device 1 and a processing device 500 that processes using the bone density estimation value 300 obtained by the estimation device 1. In Figure 6 this example, the estimation device 1 and the processing device 500 can communicate with each other via a communication network 700. The communication network 700 includes at least one of a wireless network and a wired network, for example. In the communication network 700, for example, a wireless LAN (Local Area Network) and the Internet are included.

[0117] In the estimation device 1, the communication network 700 is connected to the communication unit 30. The control unit 10 causes the communication unit 30 to send the bone density estimation value 300 to the processing device 500. The processing device 500 processes using the bone density estimation value 300 received from the estimation device 1 via the communication network 700. For example, the processing device 500 is a display device such as a liquid crystal display device, and displays the bone density estimation value 300. At this time, the processing device 500 can also display the bone density estimation value 300 in a table or as a graph. Furthermore, when a plurality of estimation devices 1 are connected to the communication network 700, the processing device 500 can also display the bone density estimation values 300 obtained by the plurality of estimation devices 1. The structure of the processing device 500 can be the same as the Figure 1 structure of the estimation device 1 shown, or can be different from the structure of the estimation device 1.

[0118] In addition, the processing using the bone density estimation value 300 performed by the processing device 500 is not limited to the above example. Furthermore, the processing device 500 can also communicate directly with the estimation device 1 wirelessly or by wire without going through the communication network 700.

[0119] <Other Examples of Estimation Data and Learning Data>

[0120] <First Other Example>

[0121] In this example, in the learning data 130, for each piece of learning image data, information related to the health status of the person with the bone photographed in the learning simple X-ray image represented by the learning image data is included. In other words, in the learning data 130, for each piece of learning image data, information related to the health status of the subject (examined person) of the learning simple X-ray image represented by the learning image data is included. After that, sometimes the information related to the health status of the subject of the learning simple X-ray image is referred to as "learning health-related information". In addition, sometimes the information related to the health status of the subject of the learning simple X-ray image represented by the learning image data is referred to as the learning health-related information corresponding to the learning image data.

[0122] The learning health-related information includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and information on the presence or absence of a fracture history. The learning health-related information is databaseized for each person and generated into a file in CSV (Comma-Separated Value) format or text format. Each of the age information, height information, and weight information is represented as numerical data of multiple bits, for example. In addition, regarding the gender information, for example, "male" or "female" is represented as 1-bit data, regarding the drinking habit information, "has a drinking habit" or "has no drinking habit" is represented as 1-bit data. In addition, regarding the smoking habit information, "has a smoking habit" or "has no smoking habit" is represented as 1-bit data, and regarding the information on the presence or absence of a fracture history, "has a fracture" or "has no fracture" is represented as 1-bit data. In addition, the learning health-related information may also include the body fat percentage or subcutaneous fat percentage of the examinee.

[0123] When the learning data 130 includes the learning image data and the learning health-related information corresponding thereto, the reference bone density corresponding to the learning image data (refer to Figure 4)(It) is associated with the learning health-related information corresponding to the learning image data. That is, with respect to the learning image data representing a learning simple X-ray image of a person's bone and the information related to the health status of this person (learning health-related information), the measured value of the bone density of this person (reference bone density) is associated. Then, in the learning of the neural network 200, with respect to the input layer 210, the learning image data and the learning health-related information corresponding thereto are input simultaneously. Specifically, the learning image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the learning health-related information is input to the other part of the multiple artificial neurons. Then, when the learning image data and the learning health-related information corresponding thereto are input with respect to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data and the learning health-related information.

[0124] In addition, in this example, in the estimation data 120, it includes estimation image data and information related to the health status of the person having the bone photographed in the estimation simple X-ray image represented by the estimation image data. In other words, in the estimation data 120, it includes estimation image data and information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data. Hereinafter, the information related to the health status of the subject of the estimation simple X-ray image is sometimes referred to as "estimation health-related information (hereinafter, in another embodiment, sometimes also referred to as 'individual data')." In addition, the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data is sometimes referred to as the estimation health-related information corresponding to the estimation image data.

[0125] The estimation health-related information, similar to the learning health-related information, includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and information on the presence or absence of a fracture history. In the estimation health-related information, it includes the same types of information as the learning health-related information. In addition, the estimation health-related information may also include the body fat percentage or subcutaneous fat percentage of the subject, similar to the learning health-related information.

[0126] In this example, when estimating bone density, the image data for estimation and the health-related information for estimation corresponding thereto are input to the input layer 210 simultaneously. Specifically, the image data for estimation is input to a part of the multiple artificial neurons constituting the input layer 210, and the health-related information for estimation is input to other parts of the multiple artificial neurons. If the image data for estimation and the health-related information for estimation for a certain person are input to the input layer 210, the estimated value of the bone density of the certain person is output from the output layer 230.

[0127] In this way, by using not only the image data of the simple X-ray image but also the information related to the health state of the subject of the simple X-ray image, the estimation accuracy of bone density can be improved.

[0128] <Other Example 2>

[0129] In this example, the learning data 130 includes the image data of N (N≥2) learning simple X-ray images that are taken of the parts of the same person and have different orientations of the parts taken therein. After that, the N learning simple X-ray images are sometimes collectively referred to as the "learning simple X-ray image group".

[0130] In the learning simple X-ray image group, for example, a front image and a side image of the same person are included. In the learning simple X-ray image group, for example, a chest front simple X-ray image and a waist side simple X-ray image of a certain person are included. The image sizes of the front image and the side image included in the learning simple X-ray image group may also be different from each other. For example, the horizontal width of the image size of the side image may be smaller than the horizontal width of the image size of the front image. After that, the image data of each learning simple X-ray image in the learning simple X-ray image group is sometimes collectively referred to as the "learning image data group".

[0131] In the learning data 130, the learning image data groups for multiple different people are included. Thus, multiple learning image data groups are included in the learning data 130. Moreover, one learning image data group corresponds to one reference bone density. That is, one learning image data group for a certain person corresponds to the measured value (reference bone density) of the bone density of the certain person.

[0132] In the learning of the neural network 200 in this example, with respect to the input layer 210, each learning image data set is input. When one learning image data set is input with respect to the input layer 210, the N learning image data constituting the one learning image data set are simultaneously input to the input layer 210. For example, assume that the learning image data set includes the first learning image data and the second learning image data. In this case, the first learning image data (for example, the image data of a frontal plain X-ray image of the chest) is input to a part of the multiple artificial neurons constituting the input layer 210, and the second learning image data (for example, the image data of a lateral plain X-ray image of the waist) is input to another part of the multiple artificial neurons. Then, when the learning image data set is input with respect to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data set.

[0133] In addition, in this example, in the estimation data 120, it includes the image data of N estimation plain X-ray images that are of the parts of the same person and in which the orientations of the parts captured are different from each other. After that, the N estimation plain X-ray images are sometimes collectively referred to as the "estimation plain X-ray image group".

[0134] In the estimation plain X-ray image group, for example, it includes a frontal image and a lateral image of the same person. In the estimation plain X-ray image group, for example, it includes a frontal plain X-ray image of a person's waist and a lateral plain X-ray image of the knee. The image sizes of the frontal image and the lateral image included in the estimation plain X-ray image group may also be different from each other. For example, the horizontal width of the image size of the lateral image may also be smaller than the horizontal width of the image size of the frontal image. After that, the image data of each estimation plain X-ray image in the estimation plain X-ray image group are sometimes collectively referred to as the "estimation image data set".

[0135] In this example, when using the estimation data 120 to estimate the bone density, with respect to the input layer 210, the N estimation image data constituting the estimation image data set are simultaneously input. For example, assume that the estimation image data set includes the first estimation image data and the second estimation image data. In this case, the first estimation image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the second estimation image data is input to another part of the multiple artificial neurons. If the estimation image data set of a certain person is input to the input layer 210, the estimated value of the bone density of the certain person is estimated from the output layer 230.

[0136] In this way, by using the image data of a plurality of simple X-ray images that capture parts of the same subject and have different orientations of the captured parts, the accuracy of bone density estimation can be improved.

[0137] In addition, the learning data 130 may also include a learning image data group and learning health-related information. In this case, in the learning of the neural network 200, the learning image data group and the learning health-related information for the same person are input to the input layer 210 simultaneously. Similarly, the estimation data 120 may also include an estimation image data group and estimation health-related information. In this case, the estimation image data group and the estimation health-related information are input to the input layer 210 simultaneously.

[0138] In each of the above examples, the same learned parameter 110 is used regardless of the type of bone captured in the X-ray image represented by the estimation image data. However, learned parameters 110 corresponding to the type of bone captured in the X-ray image represented by the estimation image data may also be used. In this case, the neural network 200 has a plurality of learned parameters 110 corresponding to multiple types of bones respectively. The neural network 200 uses the learned parameter 110 corresponding to the type of bone captured in the input estimation image data to estimate the bone density. For example, when the X-ray image represented by the input estimation image data captures the lumbar vertebra, the neural network 200 uses the learned parameter 110 for estimating the bone density of the lumbar vertebra to estimate the bone density. In addition, when the X-ray image represented by the input estimation image data captures the proximal part of the femur, the neural network 200 uses the learned parameter 110 for estimating the bone density of the proximal part of the femur to estimate the bone density. The neural network 200 uses, for example, the learned parameter 110 indicated by the user through the input unit 50 among the plurality of learned parameters 110. In this case, the user indicates the learned parameter 110 used by the neural network 200 according to the type of bone captured in the X-ray image represented by the estimation image data input to the neural network 200.

[0139] In the learning of the neural network 200, a plurality of learning image data respectively representing a plurality of X-ray images that capture the same type of bone are used to generate the learned parameter 110 corresponding to the type of bone.

[0140] As described above, the estimation device 1 and the bone density estimation system 600 have been described in detail. However, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0141] Embodiment 2.

[0142] Figure 7 FIG. is an example showing the structure of the estimation device 1A according to the present embodiment. In the estimation device 1A, the approximator 280 further includes a second neural network 900. The second neural network 900 can detect a fracture based on the learned parameters 910. In addition, the estimation device 1A according to the present embodiment has the same structure as the estimation device 1 according to the first embodiment, and the description of the same structure is omitted. In addition, for convenience of explanation, the neural network 200 described in the above example is referred to as the first neural network 200. The second neural network 900 has, for example, the same structure as the first neural network 200.

[0143] The second neural network 900 can detect a fracture based on the same estimation image data as the estimation image data included in the estimation data 120 input to the first neural network 200. That is, based on one estimation image data, the bone density can be estimated in the first neural network 200, and a fracture can be detected in the second neural network 900. In addition, the detection result 920 of the second neural network 900 may be output from the output layer 230 of the second neural network 900 in the same manner as in the above example.

[0144] In the learning of the second neural network 900, the learning image data of the non-fractured bone and the learning image data of the fractured bone are used to learn the parameters. In addition, in the training data, information indicating whether there is a current fracture and information indicating the fracture site for the bone in which the learning image data is captured are associated with each learning image data. In addition, the training data may include information indicating the past fracture history and information indicating the past fracture site. As a result, the second neural network 900 can detect the presence and location of a fracture for the bone in which the estimation image data is captured based on the estimation image data, and output the detection result 920.

[0145] In addition, the estimation device 1A according to the present embodiment may also be as Figure 8As shown, there is a determination unit 930 for determining whether the subject has osteoporosis. The determination unit 930 can compare and study the estimation result 300 of the first neural network 200 and the perception result 920 of the second neural network 900 to determine whether the subject has osteoporosis.

[0146] The determination unit 900 can also determine osteoporosis based on, for example, specific criteria or well-known guidelines, etc. Specifically, the determination unit 900 can determine that it is osteoporosis when the perception result 920 indicates a fracture at the vertebral body or the proximal part of the thigh bone. In addition, when the bone density estimation value 300 output by the first neural network 200 represents YAM, the determination unit 900 can also determine that it is osteoporosis when YAM is a value less than 80% and the perception result 920 indicates a fracture other than the vertebral body and the proximal part of the thigh bone. In addition, the determination unit 900 can also determine that it is osteoporosis when YAM represented by the bone density estimation value 300 indicates a value of 70% or less.

[0147] In addition, in the estimation device 1A according to the present embodiment, the approximator 280 can also be as Figure 9 shown, and further has a third neural network 950. The third neural network 950 can divide the bones of the subject based on the learned parameters 960 according to the estimation image data included in the estimation data 120.

[0148] The third neural network 950 outputs part information indicating the part of the bone represented by the pixel data for each pixel data of the input estimation image data. Thus, the bones captured in the X-ray image represented by the estimation image data can be divided. Sometimes the part information is called segment data.

[0149] For example, when the X-ray image represented by the input estimation image data captures the lumbar vertebrae, the third neural network 950 outputs part information indicating which part of L1 - L5 of the lumbar vertebrae the pixel data indicates for each pixel data of the estimation image data. For example, when a certain pixel data of the estimation image data represents L1 of the lumbar vertebrae, the third neural network 950 outputs part information indicating L1 as the part information corresponding to the pixel data.

[0150] The third neural network 950 uses the learned parameters 960 corresponding to the types of bones captured in the X-ray image represented by the estimation image data. The third neural network 950 has multiple learned parameters 960 corresponding to multiple types of bones respectively. The third neural network 950 uses the learned parameters 960 corresponding to the type of bone captured in the X-ray image represented by the input estimation image data to divide the bones captured in the X-ray image represented by the estimation image data. For example, when the lumbar vertebra is captured in the X-ray image represented by the input estimation image data, the third neural network 950 uses the learned parameters 960 corresponding to the lumbar vertebra to divide the lumbar vertebra into L1 to L5. The third neural network 950 uses, for example, the learned parameters 960 indicated by the user through the input unit 50 among the multiple learned parameters 960. In this case, the user indicates the learned parameters 960 used by the third neural network 950 according to the type of bone captured in the X-ray image represented by the estimation image data input to the third neural network 950.

[0151] The third neural network 950 may also divide the bones captured in the X-ray image represented by the input estimation image data into a first part implanted with an implant, a second part with a tumor, and a third part with a fracture. In this case, for each pixel data of the estimation image data, the third neural network 950 outputs part information indicating which of the first part, the second part, and the third part the pixel data indicates. When the pixel data indicates a part other than the first part, the second part, and the third part, the third neural network 950 outputs part information indicating that the pixel data indicates a part other than the first part, the second part, and the third part. When the third neural network 950 divides the bones captured in the X-ray image represented by the estimation image data into the first part, the second part, and the third part, it can also be said that the third neural network 950 senses the implant embedded in the bones captured in the X-ray image represented by the estimation image data, the fracture of the bone, and the tumor of the bone.

[0152] In the learning of the third neural network 950, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 960 corresponding to the type of the bone. When the third neural network 950 divides the bone captured in the X-ray image represented by the input estimation image data into a first part, a second part, and a third part, among the multiple learning image data, there are included learning image data representing an X-ray image of a case where an implant is embedded, learning image data representing an X-ray image of a case where there is a tumor in the bone, and learning image data representing an X-ray image of a case where there is a fracture. In addition, in the training data, for each learning image data, there is included annotation information for dividing the bone represented by the learning image data. In the annotation information, for each pixel data of the learning image data corresponding thereto, there is included part information indicating the part of the bone represented by the pixel data.

[0153] In addition, the first neural network 200 may also estimate the bone density for each part divided in the third neural network 950. In this case, as Figure 10 shown, the estimation image data 121 and the part information 965 corresponding to each pixel data of the estimation image data 121 output by the third neural network 950 based on the estimation image data 121 are input to the first neural network 200. The first neural network 200 outputs the bone density estimation value 300 for each part divided in the third neural network based on the learned parameters 110 corresponding to the type of the bone captured in the X-ray image represented by the estimation image data 121. For example, when the third neural network 950 divides the cervical vertebra captured in the X-ray image represented by the estimation image data 121 into L1 to L5, the first neural network 200 separately outputs the bone density estimation value 300 for L1, the bone density estimation value 300 for L2, the bone density estimation value 300 for L3, the bone density estimation value 300 for L4, and the bone density estimation value 300 for L5.

[0154] In the learning of the first neural network 200, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 110 corresponding to the type of the bone. In addition, in the training data, for each learning image data, there is included the reference bone density of each part of the bone represented by the learning image data.

[0155] The first neural network 200 uses, for example, the learned parameters 110 input from the user via the input unit 50 among the multiple learned parameters 110. In this case, the user indicates the learned parameters 110 to be used by the first neural network 200 according to the type of bone captured in the X-ray image represented by the estimation target image data input to the first neural network 200.

[0156] When the third neural network 950 divides the bone captured in the X-ray image represented by the estimation target image data into a first part where an implant is embedded, a second part with a tumor, and a third part with a fracture, it is also possible to adjust the brightness of the first partial image data representing the first part, the second partial image data representing the second part with a tumor, and the third partial image data representing the third part in the estimation target image data. Figure 11 It is a diagram showing a structural example in this case.

[0157] As Figure 11 shown, the estimation target image data 121 and the part information 965 corresponding to each pixel data of the estimation target image data 121 output by the third neural network 950 based on the estimation target image data 121 are input to the adjustment unit 968. The adjustment unit 968 determines the first partial image data, the second partial image data, and the third partial image data included in the estimation target image data 121 based on the part information 965. Then, the adjustment unit 968 adjusts the brightness of the determined first partial image data, second partial image data, and third partial image data.

[0158] The adjustment unit 968 stores, for example, the brightness of the first part captured in a general X-ray image as the first reference brightness. In addition, the adjustment unit 968 stores the brightness of the second part captured in a general X-ray image as the second reference brightness. Moreover, the adjustment unit 968 stores the brightness of the third part captured in a general X-ray image as the third reference brightness. The adjustment unit 968 subtracts the first reference brightness from the brightness of the first partial image data to adjust the brightness of the first partial image data. In addition, the adjustment unit 968 subtracts the second reference brightness from the brightness of the second partial image data to adjust the brightness of the second partial image data. Moreover, the adjustment unit 968 subtracts the third reference brightness from the brightness of the third partial image data to adjust the brightness of the third partial image data. The adjustment unit 968 inputs the estimation target image data with the brightness of the first partial image data, the second partial image data, and the third partial image data adjusted in the estimation target image data to the first neural network 200 as the estimation target image data with adjusted brightness. The first neural network 200 estimates the bone density of the bone captured in the X-ray image represented by the estimation target image data with adjusted brightness.

[0159] Here, it is not easy to accurately estimate the bone density based on the first part where the implant is embedded, the second part with a tumor, and the third part with a fracture. By adjusting the brightness of the first partial image data representing the first part, the second partial image data representing the second part, and the third partial image data representing the third part to be smaller as described above, it is possible to accurately estimate the bone density of the bone captured in the X-ray image represented by the estimation target image data.

[0160] In addition, the adjustment unit 968 may also input, as the estimation target image data with adjusted brightness, the estimation target image data in which the brightness of the first partial image data, the second partial image data, and the third partial image data is forcibly set to zero, into the first neural network 200.

[0161] In addition, the third neural network 950 may also detect only one of the implant, the fracture, and the tumor. In addition, the third neural network 950 may also detect only two of the implant, the fracture, and the tumor. That is, the third neural network 950 may also detect at least one of the implant, the fracture, and the tumor.

[0162] In addition, the estimation device 1A may not include the second neural network 900, but may include the first neural network 200 and the third neural network 950. In addition, the estimation device 1A may not include the first neural network 200, but may include at least one of the second neural network 900 and the third neural network 950.

[0163] As described above in detail for the estimation device 1A, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0164] Embodiment 3.

[0165] Figure 12 FIG. is an example showing the structure of an estimation device 1B according to the present embodiment. The estimation device 1B includes a fracture prediction unit 980. The fracture prediction unit 980 can, for example, predict the probability of a fracture based on the estimation result 300 of the neural network 200 of the estimation device 1 according to Embodiment 1. Specifically, for example, an arithmetic expression 990 representing the relationship between the estimation result related to the bone density (such as bone density, etc.) and the probability of a fracture is obtained based on past literature, etc. The fracture prediction unit 980 stores the arithmetic expression 990. The fracture prediction unit 980 can predict the probability of a fracture based on the input estimation result 300 and the stored arithmetic expression 990.

[0166] In addition, the arithmetic expression 990 may also be an arithmetic expression representing the relationship between the estimated result associated with bone density and the probability of fracture after bone nail implantation. As a result, it is possible to conduct research on whether to implant a bone nail and a treatment plan including drug administration.

[0167] In addition, the estimation device 1B may also include a second neural network 900. Furthermore, the estimation device 1B may also include a third neural network 950.

[0168] As described above, the estimation device 1B has been described in detail, but the above description is illustrative in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0169] Embodiment 4.

[0170] In Figure 13 it shows the concept of the structure of the estimation system 801 of this embodiment.

[0171] The estimation system 801 of the present disclosure can, for example, estimate the future bone mass of a subject based on an image of the subject's bone such as an X-ray image. The estimation system 801 of the present disclosure includes a terminal device 802 and an estimation device 803. In addition, the bone mass is an index associated with bone density and includes the concept of bone density.

[0172] The terminal device 802 can acquire input information I for input to the estimation device 803. The input information I can be, for example, an X-ray image or the like. In this case, the terminal device 802 can be any device for a doctor or the like to take an X-ray image of the subject. For example, the terminal device 802 can be a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device).

[0173] In addition, the terminal device 802 is not limited to a simple X-ray imaging device. For example, the terminal device 802 may also be an X-ray fluoroscopic imaging device, a CT (Computed Tomography), an MRI (Magnetic Resonance Imaging), an SPECT (Single Photon Emission Computed Tomography)-CT, or tomosynthesis. In this case, the input information I may be, for example, an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image.

[0174] The estimation system 801 can be used, for example, for diagnosing osteoporosis and the like in patients who go to the hospital. The estimation system 801 of the present disclosure uses, for example, the terminal device 802 provided in the X-ray room to take an X-ray photograph of the patient. Then, the image data is transmitted from the terminal device 802 to the estimation device 803, and through the estimation device 803, it is possible to not only estimate the bone mass or bone density of the patient at the current time point, but also estimate the bone mass or bone density of the patient at a time point further in the future than when the photograph was taken.

[0175] In addition, the terminal device 802 may not directly transmit the input information I to the estimation device 803. In this case, for example, the input information I obtained by the terminal device 802 may be stored in a storage medium, and the input information I may be input to the estimation device 803 via the storage medium.

[0176] In Figure 14 shows the concept of the structure of the estimation device 803 according to the present embodiment.

[0177] The estimation device 803 can estimate the future bone mass or bone density of the subject based on the input information I obtained by the terminal device 802. In the estimation device 803, the future bone mass or bone density of the subject can be estimated based on the image data obtained by the terminal device 802, and the estimation result O is output.

[0178] The estimation device 803 includes an input unit 831, an approximator 832, and an output unit 833. The input unit 831 inputs the input information I from the terminal device 802. The approximator 832 can estimate the future bone mass or bone density based on the input information I. The output unit 833 can output the estimation result O predicted by the approximator 832.

[0179] The estimation device 803 includes various electronic components and circuits. As a result, the estimation device 803 can form each structural element. For example, the estimation device 803 can integrate multiple semiconductor elements to form at least one integrated circuit (such as an IC: Integrated Circuit or an LSI: Large Scale Integration), or further integrate multiple integrated circuits to form at least one unit, etc., thereby constituting each functional part of the estimation device 803.

[0180] The multiple electronic components can be, for example, active elements such as transistors or diodes, or passive elements such as capacitors. In addition, the multiple electronic components and the integrated circuits formed by integrating them can be formed by conventionally known methods.

[0181] The input unit 831 inputs the information used in the estimation device 803. For example, the input information I with an X-ray image obtained by using the terminal device 802 is input to the input unit 831. The input unit 831 has a communication unit, and the input information I obtained by using the terminal device 802 is directly input from the terminal device 802. In addition, the input unit 831 may include an input device capable of inputting the input information I or other information. The input device can be, for example, a keyboard, a touch screen, or a mouse.

[0182] Based on the information input to the input unit 831, the approximator 832 estimates the future bone mass or bone density of the subject. The approximator 832 has AI (Artificial Intelligence). The approximator 832 has a program that functions as AI, and various electronic components and circuits for executing this program. The approximator 832 has a neural network.

[0183] The approximator 832 performs a learning process in advance for the input-output relationship. That is, machine learning can be applied to the approximator 832 using learning data and training data, so that the approximator 832 can calculate the estimation result O based on the input information I. In addition, the learning data or training data can be the input information I input to the estimation device 803 and the data corresponding to the estimation result O output from the estimation device 803.

[0184] Figure 15 Conceptually shows the structure of the approximator 832 of the present disclosure.

[0185] The approximator 832 includes a first neural network 8321 and a second neural network 8322. The first neural network 8321 may be any neural network suitable for processing temporal information. For example, the first neural network 8321 may be a ConvLSTM network that combines LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network), etc. The second neural network 8322 may be, for example, a convolutional neural network composed of CNN, etc.

[0186] The first neural network 8321 has an encoding part E and a decoding part D. The encoding part E can extract the temporal changes and the feature quantities of the position information of the input information I. The decoding part D can calculate new feature quantities based on the feature quantities extracted by the encoding part E, the temporal changes of the input information I, and the initial values.

[0187] Figure 16 Concept of the structure of the first neural network 8321 of the present disclosure is shown.

[0188] The encoding part E has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers E1. The decoding part D has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers D1. Each of the encoding part E and the decoding part D may have three or more ConvLSTM layers E1, D1. In addition, the number of the plurality of ConvLSTM layers E1 and the number of the plurality of ConvLSTM layers D1 may be the same.

[0189] In addition, the content learned by each of the plurality of ConvLSTM layers E1 may be different. The content learned by each of the plurality of ConvLSTM layers D1 may be different. For example, for a certain ConvLTSM layer, it learns about fine details such as the changes of each pixel, and for another ConvLSTM layer, it learns about rough contents such as the changes of the overall image.

[0190] Figure 17 Concept of the structure of the second neural network 8322 is shown.

[0191] The second neural network 8322 has a transformation unit C. The transformation unit C can transform the feature amount calculated by the decoding unit D of the first neural network 8321 into bone mass or bone density. The transformation unit C has a plurality of convolutional layers C1, a plurality of pooling layers C2, and a fully connected layer C3. The fully connected layer C3 is at the front stage of the output unit 33. Moreover, in the transformation unit C, the convolutional layer C1 and the pooling layer C2 are alternately arranged between the first neural network 8321 and the fully connected layer C3.

[0192] In addition, during the learning of the approximator 832, the learning data is input to the encoding unit E of the approximator 832. During the learning of the approximator 832, the training data is compared with the output data output from the transformation unit C of the approximator 832. The training data is data representing values measured using a conventional bone density measurement device.

[0193] The output unit 833 can display the estimation result O. The output unit 833 is, for example, a liquid crystal display or an organic EL display. The output unit 833 can display various information such as characters, symbols, and graphics. The output unit 833 can, for example, display numbers or images.

[0194] The estimation device 803 of the present disclosure further has a control unit 834 and a storage unit 835. The control unit 834 can uniformly manage the operation of the estimation device 803 by controlling other structural elements of the estimation device 803.

[0195] The control unit 834, for example, has a processor. The processor can include, for example, one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, or any combination of these devices or any calibrated combination, or other basic devices or calibrated combinations. The control unit 834, for example, includes a CPU.

[0196] The storage unit 835 includes, for example, a non-temporary storage medium such as a RAM (Random Access Memory) or a ROM (Read-Only Memory) that can be read by the CPU of the control unit 834. A control program for controlling the estimation device 803 such as firmware is stored in the storage unit 835. In addition, the input information I input, the learning data for learning, and the training data can also be stored in the storage unit 835.

[0197] The processor of the control unit 834 can execute one or more data calculation programs or processes according to the control program in the storage unit 835. Various functions of the control unit 834 are realized by the CPU of the control unit 834 executing the control program in the storage unit 835.

[0198] In addition, the control unit 834 can also perform other processes as preprocessing of computational processing as needed.

[0199] <An example of input information, learning data, and training data>

[0200] The input information (hereinafter, also referred to as the first input information I1) has image data of a bone of an object whose bone mass or bone density is to be estimated. The image data can be, for example, a simple X-ray image. The object whose bone mass or bone density is to be estimated is, for example, a human. In this case, it can be said that the first input information I1 is image data of a simple X-ray image of a human bone. A simple X-ray image is a two-dimensional image and is also referred to as a general X-ray image or a Roentgen image.

[0201] The first input information I1 is preferably a simple X-ray image that is relatively easy to obtain, but is not limited thereto. For example, there are cases where bone mass or bone density can be estimated more accurately by using an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image as input information.

[0202] In addition, the object whose bone mass or bone density is to be estimated can also be other than a human. For example, the object whose bone mass or bone density is to be estimated can also be an animal such as a dog, a cat, or a horse. In addition, the bones that are the objects are mainly cortical bone and cancellous bone of biological origin, but the bones that are the objects can also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine or the like.

[0203] As the shooting site of the X-ray image, for example, it can be any of the neck, chest, waist, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, or jaw joint. In addition, parts other than bones can also be photographed in the X-ray image. For example, in the case of a chest simple X-ray image, it can also include an image of the lungs and an image of the thoracic vertebrae. The X-ray image can be a front image of the object site taken from the front or a side image of the object site taken from the side.

[0204] The learning data or the training data can be any data corresponding to the input information I input to the estimation device 803 and the estimation result O output from the estimation device 803.

[0205] The learning data has the same type of information as the first input information I1. For example, if the first input information I1 is a simple X-ray image, the learning data only needs to have a simple X-ray image. Further, in the case where the first input information I1 is a chest simple X-ray image, the learning data only needs to have a chest simple X-ray image.

[0206] The learning image data including multiple simple X-ray images of bones is included in the learning data. Among the shooting parts of the multiple learning image data, for example, at least one of the neck, chest, waist, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, and jaw joint is included. In the learning data, it may include image data of some types among the 11 types of image data, or may include all types of image data. In addition, among the multiple learning image data, it may include a front image and a side image.

[0207] In the learning data, bones of multiple different individuals are respectively photographed. For each of the multiple learning image data, as training data, the measured values of the bone mass or bone density of the subject of each learning image data are established corresponding. The measured value of the bone mass or bone density is the measured value measured at a time period almost the same as the time period when the learning image data is photographed.

[0208] In addition, the learning image data of the learning data may also be a series of data with different time axes for photographing the same person. That is, the learning image data is the first learning data with an X-ray image of a bone and an image of the same person as the first learning data, and may also include the second learning data with an X-ray image photographed after the first learning data.

[0209] In addition, the learning image data of the learning data may also be a data set with different ages, etc. for photographing the same part of others. In addition, the learning image data of the learning data may also be a series of data with different time axes for photographing the same person and the same part.

[0210] The learning data and the first input information I1 may also use the input information obtained by compressing the image data representing the gray scale of the simple X-ray image (in other words, the general X-ray imaging device or the Roentgen imaging device) and reducing the number of gray scales. For example, consider the case where the number of pixel data of the image data is more than (1024×640), and the number of bits of the pixel data is 16 bits. In this case, the input information obtained by compressing the number of pixel data to, for example, (256×256), (1024×512), or (1024×640) and reducing the number of bits of the pixel data to 8 bits is used as the first input information I1 and the learning data.

[0211] The training data includes, for each of the plurality of learning image data included in the learning data, a measured value of the bone mass or bone density of the bone captured in the simple learning X-ray image represented by the learning image data. The bone mass or bone density may be measured, for example, by a DEXA (dual-energy X-ray absorptiometry) method or an ultrasonic method.

[0212] <Neural network learning example>

[0213] The control unit 834 performs machine learning using the learning data and the training data on the approximator 832, so that the approximator 832 can calculate the estimated result O related to the bone mass or bone density based on the input information I. The approximator 832 is optimized by the well-known machine learning using the training data. The approximator 832 performs calculations based on the learning data input to the encoding unit E, and adjusts the variable parameters in the approximator 832 so that the difference between the pseudo estimated result output from the conversion unit C and the training data is small.

[0214] Specifically, the control unit 834 inputs the learning data in the storage unit 835 to the encoding unit E. When the learning data is input to the encoding unit E, the control unit 834 inputs the plurality of pixel data constituting the learning image data to the plurality of artificial neurons constituting the encoding unit E. Then, when the learning image data is input to the encoding unit E, the control unit 834 adjusts the parameters so that the error of the estimated result O output from the conversion unit C relative to the bone mass or bone density actual measurement value corresponding to the learning image data becomes smaller. The adjusted parameters become the parameters after learning and are stored in the storage unit 835.

[0215] As a parameter adjustment method, for example, an error back propagation method may be used. The parameters include, for example, parameters used by the encoding unit E, the decoding unit D, and the transformation unit C. Specifically, the parameters include weighting coefficients used by the ConvLSTM layers of the encoding unit E and the decoding unit D, and the convolutional layer and the fully connected layer of the transformation unit C.

[0216] As a result, the approximator 832 performs operations based on the learned parameters for the input information I input to the encoder E, and outputs the estimated result O from the transform unit C. When the X-ray image data as the input information I is input to the encoder E, the plurality of pixel data constituting the image data are respectively input to the plurality of artificial neurons constituting the input unit 831. Then, the ConvLSTM layer, the convolution layer, and the fully connected layer can perform operations using the weighting coefficients included in the learned parameters, thereby outputting the estimated result O.

[0217] As described above, in the estimation system 801, using the image data of the plain X-ray image, learning of the approximator 832 and estimation of bone mass or bone density in the approximator 832 are performed. Therefore, input information I can be input to the estimation system 801, and future bone mass or bone density can be output as an estimation result O.

[0218] The estimation result O of the estimation system 801 only needs to be an estimation result on a certain day in the future that is more future than the acquisition date of the input information I. For example, the estimation system 1 can estimate the bone mass or bone density from 3 months to 50 years after the shooting, more preferably from 6 months to 10 years after the shooting.

[0219] The estimation result O only needs to be output as a value. For example, it can be represented by at least one of YAM (Young Adult Mean), T-score, and Z-score. For example, from the output unit 833, an estimated value represented as YAM can be output, or an estimated value represented as YAM, an estimated value represented as T-score, and an estimated value represented as Z-score can be output.

[0220] In addition, the estimation result O can also be output as an image. When the estimation result O is an image, for example, an image imitating an X-ray image only needs to be displayable. In addition, an image imitating an X-ray image is an image that imitates an X-ray image. In addition, when learning a series of data with different time axes of the same person and the same part taken using ConvLSTM, the time change of the image can be predicted. Thereby, a future image can be generated based on the X-ray image at one time point of another patient.

[0221] In the learning data and the input information I, in addition to bones, internal organs, muscles, fat, or blood vessels can also be photographed. Even in this case, highly accurate estimation can be performed.

[0222] The first input information I1 can also have the individual data (first individual data) of the subject. The so-called first individual data, for example, can be age information, gender information, height information, weight information, or fracture history, etc. As a result, highly accurate estimation can be performed.

[0223] The first input information I1 can also have the second individual data of the subject. The so-called second individual data can also include information such as blood pressure, lipids, cholesterol, neutral fat, and blood glucose level. As a result, highly accurate estimation can be performed.

[0224] The first input information I1 can also have the lifestyle information of the subject. The so-called lifestyle information can be information such as drinking habits, smoking habits, exercise habits, or eating habits. As a result, highly accurate estimation can be performed.

[0225] The first input information I1 may also include the bone metabolism information of the subject. The so-called bone metabolism information can be, for example, the bone resorption ability or the bone formation ability. For example, they can be measured by at least one of N-terminal telopeptide of type I collagen cross-links (NTX), C-terminal telopeptide of type I collagen cross-links (CTX), tartrate-resistant acid phosphatase (TRACP-5b), deoxypyridinoline (DPD) as bone resorption markers, bone-specific alkaline phosphatase (BAP), N-propeptide of type I collagen cross-links (P1NP) as bone formation markers, and undercarboxylated osteocalcin (ucOC) as bone-related matrix markers. The bone resorption markers can be measured using serum or urine as the test body.

[0226] In the estimation system 801, as the input information I, the second input information I2 related to the future planned actions of the subject can also be further input. The so-called second input information I2 can be, for example, information related to the individual data that is planned to be improved or has been improved, or the lifestyle and exercise and eating habits that are planned to be improved or have been improved. Specifically, the second input information I2 can be information such as weight data, drinking habits, smoking habits, sun exposure time, the number of steps or walking distance per day, dairy product intake, or the intake of foods rich in vitamin D such as improved fish and mushrooms. As a result, the estimation system 801 can display an estimation result O in which the future bone mass or bone density has been improved.

[0227] In addition, the second input information I2 can also be, for example, information related to the lifestyle habits that are expected to deteriorate. As a result, the estimation system can display an estimation result O in which the future bone mass or bone density deteriorates.

[0228] In the estimation system 801, as the input information I, the third input information I3 related to the therapy for the subject can also be further input. The so-called third input information I3 is, for example, information related to physical therapy or drug therapy. Specifically, the third input information I3 can be at least one of calcium tablets, female hormone drugs, vitamin drugs, bisphosphonate drugs, SERM (Selective Estrogen Receptor Modulator) drugs, calcitonin drugs, thyroid hormone drugs, and denosumab drugs.

[0229] In the estimation system 801, the estimation result O can also output the first result O1 based only on the first input information I1 and the second result O2 based on at least one of the first input information I1 and the second and third input information I2 and I3. As a result, it is possible to compare the effects of future planned actions.

[0230] In the estimation system 801, the estimation result O can output not only the future bone mass or bone density but also the current result. As a result, it is possible to compare the changes in bone mass or bone density over time.

[0231] Figure 18 Conceptually shows the structure of the approximator 832 of another embodiment of the estimation system 1.

[0232] The estimation device 803 of the estimation system 801 may also have a first approximator 832a and a second approximator 832b. That is, in addition to the above-described approximator 832 (first approximator 832a), a second approximator 832b may also be provided. The second approximator 832b may be, for example, a CNN.

[0233] In this case, in the estimation system 801, the first approximator 832a outputs the first image and the first value as the first estimation result O1 to the first output unit 833a. Further, the second approximator 832b outputs the second value as the second estimation result O2 to the second output unit 833b based on the first image from the first output unit 833a. As a result, it is possible to compare the first value and the second value as the estimation result O of the future bone mass or bone density.

[0234] The estimation system 801 may also output a third value based on the first value and the second value as the estimation result O. As a result, for example, it is possible to set the result (third value) obtained by calibrating the first value based on the second value as the estimation result O.

[0235] As described in detail above for the estimation system 801, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

Claims

1. A presumption device, Based on a simple X-ray image for presumption obtained by irradiating a first person with X-rays, and based on parameters learned in a first learning, at least one of a bone density and a bone mass is presumed. The simple X-ray image for presumption captures at least one of the cortical bone and the cancellous bone of the first person.

2. The presumption device according to claim 1, wherein Based on the simple X-ray image for presumption and information related to the health status of the first person, and based on the parameters learned in the first learning, a first presumption value, which is at least one of the current bone density and the current bone mass, is presumed as the presumption value.

3. The presumption device according to claim 2, wherein The parameters learned in the first learning are learned based on first learning data including a simple X-ray image for first learning of a second person and information related to the health status of the second person.

4. The presumption device according to claim 1 or 2, wherein Based on input information including the simple X-ray image for presumption, and based on parameters learned in a second learning, at least one of a future bone density and a future bone mass, a second presumption value, is presumed.

5. The presumption device according to claim 4, wherein The parameters learned in the second learning are parameters learned based on a plurality of second learning simple X-ray images respectively capturing the bones of a plurality of different persons.

6. The presumption device according to claim 4, wherein The parameters learned in the second learning are parameters learned based on a plurality of second learning simple X-ray images capturing the same person at different times.

7. The presumption device according to claim 6, wherein The parameters learned in the second learning are parameters learned based on a plurality of second learning simple X-ray images capturing the same part of the bone of the same person at different times.

8. The presumption device according to claim 4, wherein The parameters learned in the second learning are parameters learned based on a plurality of second learning simple X-ray images capturing the same parts of the bones of a plurality of different persons.

9. The presumption device according to claim 4, wherein The parameters learned in the second learning are parameters learned based on second learning simple X-ray images obtained by reducing the size of a simple X-ray image captured by a simple X-ray imaging device and / or reducing the gray scale.

10. The presumption device according to claim 4, wherein Feature amounts of the temporal change and the position information of the input information are extracted, new feature amounts are calculated based on the feature amounts, the temporal change of the input information, and an initial value, and the new feature amounts are transformed into the second presumption value.

11. The presumption device according to claim 1 or 2, wherein The simple X-ray image for presumption is a front image irradiating the first person from the front or a side image irradiating the first person from the side.

12. The presumption device according to claim 1 or 2, wherein The parameter completed in the first learning is a parameter learned based on first learning data including a learning simple X-ray image of a second person, and first training data including at least one measured value of the bone density and bone mass of the second person.

13. The estimation device according to claim 12, wherein the simple X-ray image for estimation includes a front image of the first person taken from the front, the simple X-ray image for learning includes a front image of the second person taken from the front.

14. The estimation device according to claim 12, wherein the measured value is measured by irradiating the second person with X-rays from the front.

15. The estimation device according to claim 12, wherein the measured value is a value measured by dual-energy X-ray absorptiometry.

16. The estimation device according to claim 15, wherein the measured value is a value measured at least one of the lumbar spine, proximal femur, radius, and metacarpal bone of the second person.

17. The estimation device according to claim 12, wherein the measured value of the first training data is measured based on at least one of the wrist and heel of the second person based on the ultrasonic method.

18. The estimation device according to any one of claims 1 to 3, wherein the simple X-ray image for estimation includes a simple X-ray image obtained by irradiating at least one of the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, hand joint, hand, fingers, and jaw joint of the first person with X-rays.

19. The estimation device according to any one of claims 1 to 3, wherein the simple X-ray image for estimation includes an image taken for purposes other than the purpose of estimating the estimated value.

20. The estimation device according to any one of claims 1 to 3, wherein the simple X-ray image for estimation includes a chest simple X-ray image obtained by irradiating the chest of the first person with X-rays.

21. The estimation device according to claim 20, wherein at least one of the lungs and thoracic vertebrae of the first person is photographed in the chest simple X-ray image.

22. The estimation device according to claim 20, wherein the chest simple X-ray image includes an image taken during a medical examination or health check.

23. The estimation device according to any one of claims 1 to 3, wherein the parameter completed in the first learning has a plurality of parameters completed in learning corresponding to the types of bones.

24. The estimation device according to any one of claims 1 to 3, wherein the parameter completed in the first learning is a parameter adjusted by the error backpropagation method.

25. The estimation device according to any one of claims 1 to 3, wherein The estimated values ​​include bone mineral density per unit area (g / cm 2 , bone mineral density per unit volume g / cm 3 , young adult mean percentile (YAM), T score, and at least one of the Z score.

26. The estimation device according to any one of claims 1 to 3, wherein the simple X-ray image for estimation is obtained from a simple X-ray examination device that has photographed the first person, or a terminal connected to the simple X-ray examination device.

27. The estimation device according to claim 26, wherein The simple X-ray image for estimation is at least one of an image obtained by reducing image data obtained by the simple X-ray inspection device and an image obtained by reducing the gray scale of an image obtained by the simple X-ray inspection device.

28. The estimation device according to any one of claims 1 to 3, wherein Based on the parameters completed by the second learning according to the simple X-ray image for estimation, segmentation for dividing a specific part of the simple X-ray image for estimation is performed.

29. The estimation device according to claim 28, wherein The parameters completed by the second learning are based on second learning data including a second learning simple X-ray image of a third person different from the first person and second training data including annotation information indicating a specific part of the third person.

30. The estimation device according to claim 28, wherein The specific part specified by the segmentation includes at least one of a part implanted with an implant, a part having a tumor, and a part having a fracture.

31. The estimation device according to claim 28, wherein The simple X-ray image for estimation includes a lumbar simple X-ray image obtained by irradiating X-rays to the waist of the first person, The estimation device performs segmentation so that at least one of L1 to L5 of the lumbar vertebrae is included as the specific part among the lumbar simple X-ray images included in the simple X-ray image for estimation.

32. The estimation device according to claim 31, wherein The specific part includes at least two or more parts of L1 to L5 of the lumbar vertebrae, The estimated value is estimated for each of the parts of the specific part.

33. The estimation device according to any one of claims 1 to 3, wherein The estimation device detects a fracture of the first person based on the parameters completed by the third learning according to the simple X-ray image for estimation.

34. The estimation device according to claim 33, wherein The parameters completed by the third learning are based on third learning data of bones with or without fractures of a fourth person different from the first person and third training data indicating the presence or absence of fractures of the fourth person.

35. The estimation device according to claim 2 or 3, wherein The estimation device determines whether the first person has osteoporosis based on the first estimated value and a specified standard for osteoporosis.

36. The estimation device according to claim 35, wherein When the first estimated value represents YAM of 70% or less, or when the first estimated value represents YAM of less than 80%, it is determined that the first person has osteoporosis and output.

37. The estimation device according to claim 36, wherein When a fracture of the vertebra or a fracture of the proximal part of the femur of the first person is indicated according to the simple X-ray image for estimation, it is determined that the first person has osteoporosis.

38. An estimation system, comprising: The estimation unit estimates at least one estimated value of bone density and bone mass based on the parameters completed by the first learning, using the simple X-ray image for estimation obtained by irradiating the first person with X-rays. The simple X-ray image for estimation captures at least one of the cortical bone and cancellous bone of the first person.

39. A method of estimation Based on the parameters completed by the first learning, an estimated value of at least one of bone density and bone mass is estimated using the simple X-ray image for estimation obtained by irradiating the first person with X-rays. The simple X-ray image for estimation captures at least one of the cortical bone and cancellous bone of the first person.

40. A program product includes a program for causing a computer device to function as the estimation device according to any one of claims 1 to 3.

41. A recording medium stores the program included in the program product according to claim 40.

Images