Estimation device, estimation method, and estimation system

Abstract

The invention provides an estimation device, an estimation method and an estimation system. The estimation device includes an input unit and an approximator. The input unit inputs input information having an image in which a bone has been captured. The approximator is capable of estimating an estimation result relating to the bone mineral density of the bone on the basis of the input information input to the input unit. The approximator has a parameter for obtaining a learning completion of an estimation result relating to the bone mineral density of the bone from input information.

Description

[0001] This application is a divisional application of the invention patent application with the application date of September 10, 2019, the application number of 201980058486.5, and the invention title of "Presumption Device, Presumption System, and Computer-readable Non-transitory Recording Medium Recording a Presumption Program".

[0002] Cross-reference to related applications

[0003] This application claims the priority of Japanese Patent Application No. 2018-168502 (filed on September 10, 2018) and Japanese National Application No. 2018-220401 (filed on November 26, 2018), and the entire disclosure of that application is incorporated herein by reference for reference. Technical field

[0004] This disclosure relates to the presumption of bone density. Background art

[0005] In Patent Document 1, a technique for determining osteoporosis is described. In Patent Document 2, a technique for presuming the strength of bones is described.

[0006] Prior art documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2008-36068

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2002-523204 Summary of the invention

[0010] A presumption device, a presumption system, and a presumption program are disclosed. In one embodiment, the presumption device includes: an input unit that inputs input information having an image of a bone captured; an approximator that can presume a presumption result related to the bone density of the bone based on the input information input to the input unit; and an output unit that outputs the presumption result presumed by the approximator, and the approximator has parameters that have completed learning for obtaining a presumption result related to the bone density of the bone based on the input information.

[0011] In addition, in one embodiment, the presumption system includes: an input unit that inputs input information having an image of a bone captured; and an approximator that has parameters that have completed learning for obtaining a presumption result related to the bone density of the bone based on the input information and can presume a presumption result related to the bone density of the bone based on the input information input to the input unit, and when the input information is input to the input unit, the approximator performs arithmetic processing on the input information.

[0012] In addition, in one embodiment, the estimation program is an estimation program for causing the device to function as a neural network that performs operations based on parameters for which learning has been completed and outputs an estimated value of the bone density of the bone captured in the image, and the parameters for which learning has been completed are used to obtain an estimation result associated with the bone density of the bone based on input information having an image in which the bone is captured. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. is an example of a diagram showing the configuration of a computer device (estimation device).

[0014] Figure 2 FIG. is a diagram for explaining the operation of the estimation device.

[0015] Figure 3 FIG. is an example of a diagram showing the configuration of a neural network.

[0016] Figure 4 FIG. is an example of a diagram showing a state in which learning image data and reference bone density are mutually established.

[0017] Figure 5 FIG. is a diagram for explaining the operation of the estimation device.

[0018] Figure 6 FIG. is an example of a diagram showing an estimation system.

[0019] Figure 7 FIG. is an example of a diagram showing the configuration of the estimation device.

[0020] Figure 8 FIG. is an example of a diagram showing the configuration of the estimation device.

[0021] Figure 9 FIG. is an example of a diagram showing the configuration of the estimation device.

[0022] Figure 10 FIG. is an example of a diagram showing the configuration of the estimation device.

[0023] Figure 11 FIG. is an example of a diagram showing the configuration of the estimation device.

[0024] Figure 12 FIG. is an example of a diagram showing the configuration of the estimation device.

[0025] Figure 13 FIG. is a conceptual diagram schematically showing the configuration of the estimation system.

[0026] Figure 14 FIG. is a conceptual diagram schematically showing the configuration of a part of the estimation system.

[0027] Figure 15 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0028] Figure 16 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0029] Figure 17 It is a conceptual diagram schematically showing the structure of a part of a presumption system.

[0030] Figure 18 It is a conceptual diagram schematically showing the structure of a part of another embodiment of a presumption system.

[0031] Symbol Explanation

[0032] 1: Computer device (presumption device);

[0033] 20: Storage unit;

[0034] 100: Control program;

[0035] 110, 910, 960: Learned parameters;

[0036] 120: Data for presumption;

[0037] 130: Data for learning;

[0038] 140: Training data;

[0039] 200: Neural network;

[0040] 210: Input layer;

[0041] 230: Output layer;

[0042] 280, 832: Approximator;

[0043] 500: Processing device;

[0044] 600: Bone density presumption system;

[0045] 801: Presumption system;

[0046] 802: Terminal device;

[0047] 803: Presumption device;

[0048] 831: Input unit;

[0049] 833: Output unit;

[0050] 834: Control unit;

[0051] 835: Storage unit;

[0052] 900, 8322: The second neural network;

[0053] 930: The determination unit;

[0054] 950: The third neural network;

[0055] 980: The fracture prediction unit;

[0056] 8321: The first neural network;

[0057] O: The estimation result;

[0058] I: The input information;

[0059] E: The encoding unit;

[0060] D: The decoding unit;

[0061] C: The transformation unit. Detailed implementation manners

[0062] Embodiment 1.

[0063] Figure 1 FIG. is a block diagram showing an example of the structure of the computer device 1 according to Embodiment 1. The computer device 1 functions as an estimation device for estimating bone density. Hereinafter, the computer device 1 may sometimes be referred to as the "estimation device 1".

[0064] As Figure 1 shown, the estimation device 1 includes, for example, a control unit 10, a storage unit 20, a communication unit 30, a display unit 40, and an input unit 50. The control unit 10, the storage unit 20, the communication unit 30, the display unit 40, and the input unit 50 are electrically connected to each other through, for example, a bus 60.

[0065] The control unit 10 can uniformly manage the operation of the estimation device 1 by controlling other structural elements of the estimation device 1. The control unit 10 is also referred to as a control device or a control circuit. As will be described in further detail below, the control unit 10 includes at least one processor in order to provide control and processing capabilities for executing various functions.

[0066] According to various embodiments, at least one processor may also be implemented as a single integrated circuit (IC) or as multiple communicatively connected integrated circuits (ICs) and / or discrete circuits. At least one processor can be implemented according to various known techniques.

[0067] In one embodiment, a processor includes, for example, one or more circuits or units configured to execute one or more data calculation programs or processes by executing instructions stored in an associated memory. In another embodiment, the processor may also be firmware (e.g., discrete logic elements) configured to execute one or more data calculation programs or processes.

[0068] According to various embodiments, the processor may also include one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, field programmable logic devices, or any combination of these devices or structures, or a combination of other known devices and structures, and perform the functions described below. In this example, the control unit 10 includes, for example, a CPU (Central Processing Unit).

[0069] The storage unit 20 includes non-transitory storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory) that can be read by the CPU of the control unit 10. A control program 100 for controlling the estimation device 1 is stored in the storage unit 20. Various functions of the control unit 10 are realized by the CPU of the control unit 10 executing the control program 100 in the storage unit 20. It can also be said that the control program 100 is a bone density estimation program for causing the computer device 1 to function as an estimation device. In this example, by the control unit 10 executing the control program 100 in the storage unit 20, as Figure 2 shown, an approximator 280 capable of outputting an estimated value 300 of bone density is formed in the control unit 10. The approximator 280 includes, for example, a neural network 200. The control program 100 can also be said to be a program for causing the computer device 1 to function as the neural network 200. Hereinafter, the estimated value of bone density may sometimes be referred to as the "bone density estimated value". In addition, a structural example of the neural network 200 will be described in detail later.

[0070] In addition to the control program 100, learned parameters 110 related to the neural network 200, estimation data 120 (hereinafter, also referred to as "input information"), learning data 130, and training data 140 are also stored in the storage unit 20. The learning data 130 and the training data 140 are data used when the neural network 200 is being learned. The learned parameters 110 and the estimation data 120 are data used by the learned neural network 200 when estimating bone density.

[0071] The learning data 130 is the data input to the input layer 210 of the neural network 200 during the learning of the neural network 200. The learning data 130 is also referred to as learning data. The training data 140 is the data representing the correct value of bone density. The training data 140 is compared with the output data output from the output layer 230 of the neural network 200 during the learning of the neural network 200. The learning data 130 and the training data 140 are sometimes collectively referred to as the learning data with training.

[0072] The data for estimation 120 is the data input to the input layer 210 of the neural network 200 when the bone density is estimated by the learned neural network 200. The learned parameter 110 is the parameter learned within the neural network 200. The learned parameter 110 can also be referred to as the parameter adjusted through the learning of the neural network 200. Among the learned parameters 110, there are included the weighting coefficients representing the weights of the connections between artificial neurons. As shown in Figure 2 the learned neural network 200 performs an operation based on the learned parameter 110 for the data for estimation 120 input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230.

[0073] In addition, the data input to the input layer 210 can be input to the input layer 210 either via the input unit 50 or directly to the input layer 210. In the case of directly inputting to the input layer 210, the input layer 210 can also be set as part or all of the input unit 50. After that, the bone density estimation value 300 is sometimes referred to as the estimation result 300.

[0074] The communication unit 30 is connected to a communication network including the Internet or the like by wire or wirelessly. The communication unit 30 can communicate with other devices such as a cloud server and a network server through the communication network. The communication unit 30 can input the information received from the communication network to the control unit 10. In addition, the communication unit 30 can output the information received from the control unit 10 to the communication network.

[0075] The display unit 40 is, for example, a liquid crystal display or an organic EL display. The display unit 40 can display various information such as characters, symbols, and graphics under the control of the control unit 10.

[0076] The input unit 50 can accept the input from the user to the estimation device 1. The input unit 50 includes, for example, a keyboard and a mouse. The input unit 50 may also include a touch screen capable of detecting the operation of the user on the display surface of the display unit 40.

[0077] In addition, the structure of the estimation device 1 is not limited to the above example. For example, the control unit 10 may also include multiple CPUs. In addition, the control unit 10 may also include at least one DSP. In addition, all functions of the control unit 10 or some of the functions of the control unit 10 may also be implemented by a hardware circuit that does not require software for the implementation of its functions. In addition, the storage unit 20 may also include a non-transitory storage medium readable by a computer other than a ROM and a RAM. For example, the storage unit 20 may also include a small hard disk drive and an SSD (Solid State Drive), etc. In addition, the storage unit 20 may also include a memory such as a USB (Universal Serial Bus) memory that can be plugged into and unplugged from the estimation device 1. Hereinafter, a memory that can be plugged into and unplugged from the estimation device 1 is sometimes referred to as a "plug-and-play memory".

[0078] <Example of the Structure of the Neural Network>

[0079] Figure 3 FIG. is an example showing the structure of the neural network 200. In this example, the neural network 200 is, for example, a convolutional neural network (CNN (Convolutional Neural Network)). As Figure 3 shown, the neural network 200 includes, for example, an input layer 210, a hidden layer 220, and an output layer 230. The hidden layer 220 is also referred to as an intermediate layer. The hidden layer 220 includes, for example, multiple convolutional layers 240, multiple pooling layers 250, and a fully connected layer 260. In the neural network 200, a fully connected layer 260 exists before the output layer 230. Moreover, in the neural network 200, the convolutional layer 240 and the pooling layer 250 are alternately arranged between the input layer 210 and the fully connected layer 260.

[0080] In addition, the structure of the neural network 200 is not limited to Figure 3 the example. For example, the neural network 200 may also include one convolutional layer 240 and one pooling layer 250 between the input layer 210 and the fully connected layer 260. In addition, the neural network 200 may also be a neural network other than a convolutional neural network.

[0081] <An Example of Estimation Data, Learning Data, and Training Data>

[0082] The data 120 for estimation includes image data of a simple X-ray image of a bone that is an object for bone density estimation. The object for bone density estimation is, for example, a human. Thus, it can be said that the data 120 for estimation includes image data of a simple X-ray image of a human bone. In the data 130 for learning, it includes image data of a plurality of simple X-ray images of a human bone. A simple X-ray image is also referred to as a two-dimensional image, a general X-ray image, or a Roentgen image. In addition, the object for bone density estimation can also be other than a human. For example, the object for bone density estimation can be an animal such as a dog, a cat, or a horse. Furthermore, the bone set as the object is mainly cortical bone and cancellous bone from a living body, but among the bones set as the object, it can also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine or the like.

[0083] After that, sometimes the image data included in the data 120 for estimation is referred to as "estimation image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 120 for estimation is referred to as "estimation simple X-ray image". In addition, sometimes the image data included in the data 130 for learning is referred to as "learning image data". In addition, sometimes the simple X-ray image represented by the image data included in the data 130 for learning is referred to as "learning simple X-ray image". In the data 130 for learning, it includes a plurality of learning X-ray image data respectively representing a plurality of learning simple X-ray images.

[0084] As the imaging part of the simple X-ray image for estimation, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toe, shoulder joint, elbow joint, wrist joint, hand, finger or jaw joint can be used. In other words, as the estimation data 120, the following may be used: image data of a simple X-ray image obtained by irradiating the head with X-rays, image data of a simple X-ray image obtained by irradiating the neck with X-rays, image data of a simple X-ray image obtained by irradiating the chest with X-rays, image data of a simple X-ray image obtained by irradiating the body with X-rays, image data of a simple X-ray image obtained by irradiating the hip joint with X-rays, image data of a simple X-ray image obtained by irradiating the knee joint with X-rays, image data of a simple X-ray image obtained by irradiating the ankle joint with X-rays, image data of a simple X-ray image obtained by irradiating the foot with X-rays, image data of a simple X-ray image obtained by irradiating the toe with X-rays, image data of a simple X-ray image obtained by irradiating the shoulder joint with X-rays, image data of a simple X-ray image obtained by irradiating the elbow joint with X-rays, image data of a simple X-ray image obtained by irradiating the hand joint with X-rays, image data of a simple X-ray image obtained by irradiating the hand with X-rays, image data of a simple X-ray image obtained by irradiating the finger with X-rays, or image data of a simple X-ray image obtained by irradiating the jaw joint with X-rays. The simple X-ray image obtained by irradiating the chest with X-rays includes a simple X-ray image of the lungs and a simple X-ray image of the thoracic spine. In addition, the type of the imaging part of the simple X-ray image for estimation is not limited to this. In addition, the simple X-ray image for estimation can be a frontal image of the target part from the front or a side image of the target part from the side.

[0085] Among the imaging regions of the multiple learning X-ray images respectively represented by the multiple pieces of learning data 130, there is at least one of, for example, the head, neck, chest, waist, hip joint, knee joint, ankle joint, foot, toes, shoulder joint, elbow joint, wrist joint, hand, fingers, and jaw joint. In other words, in the learning data 130, there is included at least one of the 15 types of image data, namely, the image data of a simple X-ray image obtained by irradiating X-rays to the head, the image data of a simple X-ray image obtained by irradiating X-rays to the neck, the image data of a simple X-ray image obtained by irradiating X-rays to the chest, the image data of a simple X-ray image obtained by irradiating X-rays to the waist, the image data of a simple X-ray image obtained by irradiating X-rays to the hip joint, the image data of a simple X-ray image obtained by irradiating X-rays to the knee joint, the image data of a simple X-ray image obtained by irradiating X-rays to the ankle joint, the image data of a simple X-ray image obtained by irradiating X-rays to the foot, the image data of a simple X-ray image obtained by irradiating X-rays to the toes, the image data of a simple X-ray image obtained by irradiating X-rays to the shoulder joint, the image data of a simple X-ray image obtained by irradiating X-rays to the elbow joint, the image data of a simple X-ray image obtained by irradiating X-rays to the wrist joint, the image data of a simple X-ray image obtained by irradiating X-rays to the hand, the image data of a simple X-ray image obtained by irradiating X-rays to the fingers, and the image data of a simple X-ray image obtained by irradiating X-rays to the jaw joint. In the learning data 130, it may include some types of the 15 types of image data or all types of image data. In addition, the types of the imaging regions of the learning simple X-ray images are not limited to this. Furthermore, among the multiple learning simple X-ray images, there may be included a frontal image or a lateral image. In addition, among the multiple learning simple X-ray images, there may also be included both a frontal image and a lateral image of the same imaging region.

[0086] In the training data 140, for each of the plurality of learning image data included in the learning data 130, it includes the measured value of the bone density of a person having bones photographed in the learning simple X-ray image represented by the learning image data. Among the plurality of measured values of bone density included in the training data 140, for example, it includes at least one of the measured value of bone density measured by irradiating X-rays to the lumbar vertebra, the bone density measured by irradiating X-rays to the proximal part of the femur, the bone density measured by irradiating X-rays to the radius, the bone density measured by irradiating X-rays to the metacarpal bone, the bone density measured by applying ultrasonic waves to the wrist, and the bone density measured by applying ultrasonic waves to the heel. Thereafter, the measured value of bone density included in the training data 140 is sometimes referred to as "reference bone density".

[0087] Here, as a method for measuring bone density, the DEXA (dual-energy X-ray absorptiometry) method is known. In a DEXA device that uses the DEXA method to measure bone density, when measuring the bone density of the lumbar vertebra, X-rays (specifically, two types of X-rays) are irradiated to the lumbar vertebra from its front. In addition, in a DEXA device, when measuring the bone density of the proximal part of the femur, X-rays are irradiated to the proximal part of the femur from its front.

[0088] In the training data 140, it may include the bone density of the lumbar vertebra measured by a DEXA device, or may include the bone density of the proximal part of the femur measured by a DEXA device. In addition, in the training data 140, it may also include the bone density measured by irradiating X-rays to the target part from its side. For example, in the training data 140, it may also include the bone density measured by irradiating X-rays to the lumbar vertebra from its side.

[0089] In addition, as another method for measuring bone density, the ultrasonic wave method is known. In a device that uses the ultrasonic wave method to measure bone density, for example, ultrasonic waves are applied to the wrist to measure the bone density of the wrist, and ultrasonic waves are applied to the heel to measure the bone density of the heel. In the training data 140, it may also include the bone density measured by the ultrasonic wave method.

[0090] In the plurality of learning simple X-ray images represented by the plurality of learning image data included in the learning data 130, the bones of different people are photographed. And, as Figure 4As shown, in the storage unit 20, for each of the multiple learning image data included in the learning data 130, a correspondence is established with the reference bone density of a person having a bone photographed in the learning simple X-ray image represented by the learning image data. It can also be said that for each of the multiple learning simple X-ray images used in the learning of the neural network 200, a correspondence is established with the reference bone density of a person having a bone photographed in the learning simple X-ray image. The reference bone density corresponding to the learning image data is the bone density measured for the same person as the person having the bone photographed in the learning simple X-ray image represented by the learning image data at a time almost the same as the time when the learning simple X-ray image was photographed.

[0091] In the part photographed in the learning simple X-ray image represented by the learning image data (in other words, the photographing part of the learning simple X-ray image), the part (i.e., the bone) for which the reference bone density corresponding to the learning image data is measured may or may not be included. In other words, in the part photographed in the learning simple X-ray image, the part for which the reference bone density corresponding to the learning simple X-ray image is measured may or may not be included. As an example of the former, consider the case where a correspondence is established between the learning image data representing the learning simple X-ray image of the waist being photographed and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the hip joint being photographed and the reference bone density of the proximal part of the femur. On the other hand, as an example of the latter, consider the case where a correspondence is established between the learning image data of the chest being photographed and the reference bone density of the lumbar vertebra. As another example, consider the case where a correspondence is established between the learning image data of the knee joint being photographed and the reference bone density of the heel.

[0092] In addition, the orientation of the part captured in the learning-only X-ray image represented by the image data for learning, and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning image data may be the same or different. In other words, the orientation of the part captured in the learning-only X-ray image, and the orientation of the X-ray irradiation with respect to the object part in the measurement of the reference bone density corresponding to the learning-only X-ray image may be the same or different. As an example of the former, consider the case of establishing a correspondence between the learning image data representing a learning-only X-ray image of the chest captured from the front (hereinafter sometimes referred to as "front chest learning-only X-ray image"), and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing a correspondence between the learning image data representing a learning-only X-ray image of the waist captured from the front (hereinafter sometimes referred to as "front waist learning-only X-ray image"), and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front. On the other hand, as an example of the latter, consider the case of establishing a correspondence between the learning image data representing a learning-only X-ray image of the waist captured from the side (hereinafter sometimes referred to as "side waist learning-only X-ray image"), and the reference bone density measured by irradiating the lumbar spine with X-rays from its front. As another example, consider the case of establishing a correspondence between the learning image data representing a learning-only X-ray image of the knee joint captured from the side (hereinafter sometimes referred to as "side knee learning-only X-ray image"), and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its front.

[0093] In addition, among the multiple learning-only X-ray images respectively represented by the multiple learning image data included in the learning data 130, there may be included a learning-only X-ray image capturing the same type of part as the estimation-only X-ray image, or a learning-only X-ray image capturing a different type of part from the estimation-only X-ray image. As an example of the former, consider the case where the estimation-only X-ray image is a front chest learning-only X-ray image, and a front chest learning-only X-ray image is included among the multiple learning-only X-ray images. As another example, consider the case where the estimation-only X-ray image is a learning-only X-ray image of the knee joint captured from the front (hereinafter sometimes referred to as "front knee learning-only X-ray image"), and a side knee learning-only X-ray image is included among the multiple learning-only X-ray images. On the other hand, as an example of the latter, consider the case where the estimation-only X-ray image is a front waist learning-only X-ray image, and a front chest learning-only X-ray image is included among the multiple learning-only X-ray images. As another example, consider the case where the estimation-only X-ray image is a side waist learning-only X-ray image, and a front knee learning-only X-ray image is included among the multiple learning-only X-ray images.

[0094] In addition, among the multiple learning plain X-ray images, there may be included plain X-ray images of parts taken in the same orientation as the estimation plain X-ray image, or plain X-ray images of parts taken in an orientation different from that of the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar spine anterior plain X-ray image, and a lumbar spine anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the estimation plain X-ray image is a knee anterior plain X-ray image, and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a knee lateral plain X-ray image, and a knee anterior plain X-ray image is included among the multiple learning plain X-ray images. As another example, consider the case where the estimation plain X-ray image is a lumbar spine lateral plain X-ray image, and a chest anterior plain X-ray image is included among the multiple learning plain X-ray images.

[0095] In addition, in the training data 140, there may be included reference bone densities measured from parts (bones) included in the part imaged in the estimation plain X-ray image, or reference bone densities measured from parts (bones) not included in the part imaged in the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar spine anterior plain X-ray image, and the reference bone density of the lumbar spine is included in the training data 140. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a chest anterior plain X-ray image, and the reference bone density of the metacarpal head is included in the training data 140.

[0096] In addition, in the training data 140, there may be included reference bone densities measured by irradiating the target part with X-rays from the same orientation as the orientation of the part imaged in the estimation plain X-ray image, or reference bone densities measured by irradiating the target part with X-rays from an orientation different from that of the part imaged in the estimation plain X-ray image. As an example of the former, consider the case where the estimation plain X-ray image is a lumbar spine anterior plain X-ray image, and the reference bone density measured by irradiating the lumbar spine with X-rays from its anterior side is included in the training data 140. On the other hand, as an example of the latter, consider the case where the estimation plain X-ray image is a lumbar spine lateral plain X-ray image, and the reference bone density measured by irradiating the proximal part of the femur with X-rays from its anterior side is included in the training data 140.

[0097] In this example, the image data representing the gray scale of a simple X-ray image obtained by a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device) is compressed, and the image data obtained by reducing the number of gray scales is used as learning image data and estimation image data. For example, when the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is more than (1024×640) and the number of bits of the pixel data is 16 bits. In this case, the number of multiple pixel data constituting the image data obtained by the simple X-ray imaging device is compressed to, for example, (256×256), (1024×512), or (1024×640), and the number of bits of the pixel data is reduced to 8 bits, and then used as learning image data and estimation image data. In this case, each of the learning simple X-ray image and the estimation simple X-ray image includes (256×256), (1024×512), or (1024×640) pixels, and the value of the pixel is expressed in 8 bits.

[0098] For the learning image data and the estimation image data, they can be generated by the control unit 10 of the estimation device 1 based on the image data obtained by the simple X-ray imaging device, or can be generated by a device other than the estimation device 1 based on the image data obtained by the simple X-ray imaging device. In the former case, for the image data obtained by the simple X-ray imaging device, it can be received by the communication unit 30 through the communication network, or can be stored in the plug-in memory included in the storage unit 20. In the latter case, it can also be that the communication unit 30 receives the learning image data and the estimation image data from other devices through the communication network, and the control unit 10 stores the learning image data and the estimation image data received by the communication unit 30 in the storage unit 20. Or, the learning image data and the estimation image data generated by other devices can be stored in the plug-in memory included in the storage unit 20. In addition, for the training data 140, it can also be that the communication unit 30 receives it through the communication network, and the control unit 10 stores the training data 140 received by the communication unit 30 in the storage unit 20. Or, the training data 140 can be stored in the plug-in memory included in the storage unit 20. In addition, the number and number of bits of the pixel data of the learning image data and the estimation image data are not limited to the above.

[0099] <Example of learning of neural network>

[0100] Figure 5 It is a diagram for explaining an example of the learning of the neural network 200. When the control unit 10 performs the learning of the neural network 200, as Figure 5As shown, learning data 130 is input to the input layer 210 of the neural network 200. Then, the control unit 10 adjusts the variable parameters 110a in the neural network 200 so that the error of the output data 400 output from the output layer 230 of the neural network 200 with respect to the training data 140 is small. More specifically, the control unit 10 inputs each piece of learning image data in the storage unit 20 to the input layer 210. When inputting the learning image data to the input layer 210, the control unit 10 inputs the multiple pixel data constituting the learning image data to the multiple artificial neurons constituting the input layer 210 respectively. Then, when the learning image data is input to the input layer 210, the control unit 10 adjusts the parameters 110a so that the error of the output data 400 output from the output layer 230 with respect to the reference bone density corresponding to the learning image data is small. As a method for adjusting the parameters 110a, for example, the error backpropagation method can be adopted. The adjusted parameters 110a become the learned parameters 110 and are stored in the storage unit 20. In the parameters 110a, for example, the parameters used in the hidden layer 220 are included. Specifically, the filter coefficients used in the convolutional layer 240 and the weighting coefficients used in the fully connected layer 260 are included in the parameters 110a. In addition, the method for adjusting the parameters 110a, in other words, the learning method of the parameters 110a is not limited to this.

[0101] In this way, in the storage unit 20, the learned parameters 110 are stored, which have learned the relationship between the learning data 130 of the image data including a plurality of learning simple X-ray images and the measured values of the bone density as the training data 140 using the neural network 200.

[0102] In the above example, the estimation device 1 performs the learning of the neural network 200, but other devices can also perform the learning of the neural network 200. In this case, the learned parameters 110 generated by other devices are stored in the storage unit 20 of the estimation device 1. In addition, it becomes unnecessary to store the learning data 130 and the training data 140 in the storage unit 20. For the learned parameters 110 generated by other devices, they can also be received by the communication unit 30 through the communication network, and the control unit 10 stores the learned parameters 110 received by the communication unit 30 in the storage unit 20. Alternatively, the learned parameters 110 generated by other devices can be stored in the pluggable memory included in the storage unit 20.

[0103] In the neural network 200 learned as above, the image data of a plurality of learning simple X-ray images is input to the input layer 210 as the learning data 130, and the learned parameters 110a learned using the reference bone density as the training data 140 are included. As described above Figure 2As shown, the neural network 200 performs operations based on the learned parameters 110a on the estimation data 120 input to the input layer 210, and outputs the bone density estimation value 300 from the output layer 230. When the estimation image data as the estimation data 120 is input to the input layer 210, the multiple pixel data constituting the estimation image data are respectively input to the multiple artificial neurons constituting the input layer 210. Then, the convolutional layer 240 performs operations using the filter coefficients included in the learned parameters 110a, and the fully connected layer 260 performs operations using the weighting coefficients included in the learned parameters 110a.

[0104] For example, when the estimation image data representing the frontal chest plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the bones of the chest photographed in the frontal chest plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the frontal lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the lumbar vertebrae included in the lumbar region photographed in the frontal lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the lateral lumbar plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the lumbar vertebrae included in the lumbar region photographed in the lateral lumbar plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the frontal knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the bones of the knee joint photographed in the frontal knee plain X-ray image represented by the estimation image data is output from the output layer 230. In addition, when the estimation image data representing the lateral knee plain X-ray image is input to the input layer 210, the estimation value 300 of the bone density of the person with the bones of the knee joint photographed in the lateral knee plain X-ray image represented by the estimation image data is output from the output layer 230.

[0105] The estimation value 300 output from the output layer 230 can be represented by at least one of the bone mineral density per unit area (g / cm 2 ), the bone mineral density per unit volume (g / cm 3 ), YAM, T-score, and Z-score. YAM is an abbreviation for "Young Adult Mean" and is sometimes referred to as the young adult average percentage. For example, from the output layer 230, it is possible to output the representation as the bone mineral density per unit area (g / cm 2The estimated value 300 expressed as ) and the estimated value 300 expressed as YAM can also output the estimated value 300 expressed as YAM, the estimated value 300 expressed as T-score, and the estimated value 300 expressed as Z-score.

[0106] In addition, the storage unit 20 can also store a plurality of estimation data 120. In this case, among the plurality of estimation-only X-ray images respectively represented by the plurality of estimation data 120 in the storage unit 20, there may be included a plurality of X-ray images of the same type of part, or a plurality of X-ray images of parts of different types. In addition, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images of a part taken from the same direction, or a plurality of X-ray images of a part taken from different directions. In other words, among the plurality of estimation-only X-ray images, there may be included a plurality of X-ray images in which the orientation of the part taken is the same, or a plurality of X-ray images in which the orientation of the part taken is different. The control unit 10 inputs each of the plurality of estimation data 120 in the storage unit 20 to the input layer 210 of the neural network 200, and outputs a bone density estimated value 300 corresponding to each estimation data 120 from the output layer 230 of the neural network 200.

[0107] As described above, in this example, the image data of the plain X-ray image is used for the learning of the neural network 200 and the estimation of the bone density in the neural network 200. For the image data of the plain X-ray image, in other words, the image data of the Roentgen image, since it is used in various examinations and the like in many hospitals, it can be easily obtained. Therefore, it is possible to simply estimate the bone density without using an expensive device such as a DEXA device.

[0108] In addition, by using the image data of the plain X-ray image taken for examinations and the like as the estimation image data, it is possible to simply estimate the bone density by taking advantage of the opportunity of such examinations. Thus, by using the estimation device 1, the service for hospital users can be improved.

[0109] In addition, in a frontal simple X-ray image of the chest, etc., it may be difficult to capture bones due to the influence of internal organs. On the other hand, in many hospitals, the possibility of capturing a frontal simple X-ray image is high. In this example, even when a frontal simple X-ray image in which bones may be difficult to capture is used as an estimation simple X-ray image or a learning simple X-ray image, bone density can be estimated. Therefore, bone density can be simply estimated using image data of an easily available frontal simple X-ray image. In addition, a chest frontal simple X-ray image is often captured in health diagnosis, etc., and can be said to be a simple X-ray image that is particularly easy to obtain. By using a chest frontal simple X-ray image as an estimation simple X-ray image or a learning simple X-ray image, bone density can be further simply estimated.

[0110] Furthermore, in this example, even when the plurality of learning simple X-ray images include a simple X-ray image that captures a part of a different type from that of the estimation simple X-ray image, the bone density can be estimated based on the image data of the estimation simple X-ray image. This can improve the convenience of the estimation device 1 (in other words, the computer device 1).

[0111] Furthermore, in this example, even when the plurality of learning simple X-ray images include a simple X-ray image that captures a part in a different orientation from that of the estimation simple X-ray image, the bone density can be estimated based on the image data of the estimation simple X-ray image. This can improve the convenience of the estimation device 1.

[0112] Furthermore, in this example, even when the parts captured in the simple X-ray image for learning do not include the parts (bone) for which the reference bone density corresponding to the simple X-ray image for learning has been measured, the neural network 200 can estimate the bone density based on the learned parameters 110. This can improve the convenience of the estimation device 1.

[0113] Furthermore, in this example, even when the orientation of the part captured in the simple X-ray image for learning and the orientation of the X-ray irradiation of the target part in the measurement of the reference bone density corresponding to the simple X-ray image for learning are different from each other, the neural network 200 can estimate the bone density based on the learned parameter 110. Thus, the convenience of the estimation device 1 can be improved.

[0114] Furthermore, in this example, even when the training data 140 includes a reference bone density measured from a site not included in the site captured by the estimation simple X-ray image, the bone density can be estimated based on the image data of the estimation simple X-ray image. This can improve the convenience of the estimation device 1.

[0115] In addition, the bone density estimated value 300 obtained by the estimation device 1 can also be displayed by the display unit 40. Furthermore, the bone density estimated value 300 obtained by the estimation device 1 can also be used by other devices.

[0116] Figure 6 FIG. is an example of a bone density estimation system 600 including an estimation device 1 and a processing device 500 that processes the bone density estimated value 300 obtained by the estimation device 1. In Figure 6 this example, the estimation device 1 and the processing device 500 can communicate with each other via a communication network 700. The communication network 700 includes at least one of a wireless network and a wired network, for example. In the communication network 700, for example, a wireless LAN (Local Area Network) and the Internet are included.

[0117] In the estimation device 1, the communication network 700 is connected to the communication unit 30. The control unit 10 causes the communication unit 30 to send the bone density estimated value 300 to the processing device 500. The processing device 500 processes using the bone density estimated value 300 received from the estimation device 1 via the communication network 700. For example, the processing device 500 is a display device such as a liquid crystal display device, and displays the bone density estimated value 300. At this time, the processing device 500 can also display the bone density estimated value 300 in a table or as a curve graph. In addition, when a plurality of estimation devices 1 are connected to the communication network 700, the processing device 500 can also display the bone density estimated values 300 obtained by the plurality of estimation devices 1. The structure of the processing device 500 can be the same as that of the Figure 1 estimation device 1 shown, or different from the structure of the estimation device 1.

[0118] In addition, the processing using the bone density estimated value 300 performed by the processing device 500 is not limited to the above example. Furthermore, the processing device 500 can also communicate directly with the estimation device 1 wirelessly or by wire without going through the communication network 700.

[0119] <Other Examples of Estimation Data and Learning Data>

[0120] <First Other Example>

[0121] In this example, in the learning data 130, for each piece of learning image data, information related to the health status of a person with bones captured in the learning simple X-ray image represented by the learning image data is included. In other words, in the learning data 130, for each piece of learning image data, information related to the health status of the subject (the person being examined) of the learning simple X-ray image represented by the learning image data is included. After that, sometimes the information related to the health status of the subject of the learning simple X-ray image is referred to as "learning health-related information". In addition, sometimes the information related to the health status of the subject of the learning simple X-ray image represented by the learning image data is referred to as the learning health-related information corresponding to the learning image data.

[0122] The learning health-related information includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and information on the presence or absence of a fracture history. The learning health-related information is databaseized for each person and generated as a file in CSV (Comma-Separated Value) format or text format. Each of the age information, height information, and weight information is represented as numerical data of multiple bits, for example. In addition, regarding the gender information, for example, "male" or "female" is represented as 1-bit data, regarding the drinking habit information, "has a drinking habit" or "has no drinking habit" is represented as 1-bit data. In addition, regarding the smoking habit information, "has a smoking habit" or "has no smoking habit" is represented as 1-bit data, and regarding the information on the presence or absence of a fracture history, "has a fracture" or "has no fracture" is represented as 1-bit data. In addition, the learning health-related information may also include the body fat percentage or subcutaneous fat percentage of the subject.

[0123] When the learning data 130 includes the learning image data and the learning health-related information corresponding thereto, the reference bone density corresponding to the learning image data (refer to Figure 4It is associated with the learning health-related information corresponding to the learning image data. That is, the measured value of the bone density (reference bone density) of a certain person is associated with the learning image data representing the learning simple X-ray image of the bone of a certain person and the information related to the health status of the certain person (learning health-related information). Then, in the learning of the neural network 200, the learning image data and the corresponding learning health-related information are input to the input layer 210 at the same time. Specifically, the learning image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the learning health-related information is input to the other part of the multiple artificial neurons. Then, when the learning image data and the corresponding learning health-related information are input to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data and the learning health-related information.

[0124] In addition, in this example, the estimation data 120 includes the estimation image data and the information related to the health status of the person with the bone photographed in the estimation simple X-ray image represented by the estimation image data. In other words, the estimation data 120 includes the estimation image data and the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data. Hereinafter, the information related to the health status of the subject of the estimation simple X-ray image is sometimes referred to as "estimation health-related information (hereinafter, in another embodiment, sometimes also referred to as "individual data")." In addition, the information related to the health status of the subject of the estimation simple X-ray image represented by the estimation image data is sometimes referred to as the estimation health-related information corresponding to the estimation image data.

[0125] Similar to the learning health-related information, the estimation health-related information includes, for example, at least one of age information, gender information, height information, weight information, drinking habit information, smoking habit information, and the presence or absence of fracture history. The estimation health-related information includes the same types of information as the learning health-related information. In addition, the estimation health-related information may also include the body fat percentage or subcutaneous fat percentage of the subject, similar to the learning health-related information.

[0126] In this example, when estimating bone density, estimation target image data and corresponding health-related information for estimation are input to the input layer 210 simultaneously. Specifically, the estimation target image data is input to a part of multiple artificial neurons constituting the input layer 210, and the health-related information for estimation is input to other parts of the multiple artificial neurons. If the estimation target image data and the health-related information for estimation for a certain person are input to the input layer 210, an estimated value of the bone density of the certain person is output from the output layer 230.

[0127] In this way, by using not only the image data of a simple X-ray image but also the information related to the health state of the subject of the simple X-ray image, the estimation accuracy of bone density can be improved.

[0128] <Other Example 2>

[0129] In this example, the learning data 130 includes image data of N (N≥2) learning simple X-ray images that are taken of parts of the same person and have different orientations of the parts taken therein. Thereafter, the N learning simple X-ray images are sometimes collectively referred to as the "learning simple X-ray image group".

[0130] In the learning simple X-ray image group, for example, a front image and a side image of the same person are included. In the learning simple X-ray image group, for example, a chest front simple X-ray image and a waist side simple X-ray image of a certain person are included. The image sizes of the front image and the side image included in the learning simple X-ray image group may also be different from each other. For example, the horizontal width of the image size of the side image may be smaller than the horizontal width of the image size of the front image. Thereafter, the image data of each learning simple X-ray image in the learning simple X-ray image group is sometimes collectively referred to as the "learning image data group".

[0131] The learning data 130 includes learning image data groups for multiple different persons. Thus, the learning data 130 includes multiple learning image data groups. Moreover, one reference bone density is corresponded to one learning image data group. That is, the measured value (reference bone density) of the bone density of a certain person is corresponded to the learning image data group for the certain person.

[0132] In the learning of the neural network 200 in this example, with respect to the input layer 210, each learning image data set is input. When one learning image data set is input with respect to the input layer 210, the N learning image data constituting the one learning image data set are simultaneously input to the input layer 210. For example, assume that the learning image data set includes the first learning image data and the second learning image data. In this case, the first learning image data (for example, the image data of a frontal plain X-ray image of the chest) is input to a part of the multiple artificial neurons constituting the input layer 210, and the second learning image data (for example, the image data of a lateral plain X-ray image of the waist) is input to another part of the multiple artificial neurons. Then, when the learning image data set is input with respect to the input layer 210, the output data 400 output from the output layer 230 is compared with the reference bone density corresponding to the learning image data set.

[0133] In addition, in this example, in the estimation data 120, it includes the image data of N estimation plain X-ray images that capture parts of the same person and in which the orientations of the captured parts are different from each other. After that, the N estimation plain X-ray images are sometimes collectively referred to as the "estimation plain X-ray image group".

[0134] In the estimation plain X-ray image group, for example, it includes a frontal image and a lateral image of the same person. In the estimation plain X-ray image group, for example, it includes a frontal plain X-ray image of a person's waist and a lateral plain X-ray image of the knee. The image sizes of the frontal image and the lateral image included in the estimation plain X-ray image group may also be different from each other. For example, the horizontal width of the image size of the lateral image may be smaller than the horizontal width of the image size of the frontal image. After that, the image data of each estimation plain X-ray image in the estimation plain X-ray image group is sometimes collectively referred to as the "estimation image data set".

[0135] In this example, when using the estimation data 120 to estimate the bone density, with respect to the input layer 210, the N estimation image data constituting the estimation image data set are simultaneously input. For example, assume that the estimation image data set includes the first estimation image data and the second estimation image data. In this case, the first estimation image data is input to a part of the multiple artificial neurons constituting the input layer 210, and the second estimation image data is input to another part of the multiple artificial neurons. If the estimation image data set of a certain person is input to the input layer 210, the estimated value of the bone density of the certain person is estimated from the output layer 230.

[0136] In this way, by using the image data of a plurality of simple X-ray images that capture the parts of the same subject and have different orientations of the captured parts, the accuracy of bone density estimation can be improved.

[0137] In addition, the learning data 130 may also include a learning image data group and learning health-related information. In this case, in the learning of the neural network 200, the learning image data group and the learning health-related information for the same person are input to the input layer 210 simultaneously. Similarly, the estimation data 120 may also include an estimation image data group and estimation health-related information. In this case, the estimation image data group and the estimation health-related information are input to the input layer 210 simultaneously.

[0138] In each of the above examples, the same learned parameter 110 is used regardless of the type of bone captured in the X-ray image represented by the estimation image data. However, learned parameters 110 corresponding to the type of bone captured in the X-ray image represented by the estimation image data may also be used. In this case, the neural network 200 has a plurality of learned parameters 110 corresponding to multiple types of bones. The neural network 200 uses the learned parameter 110 corresponding to the type of bone captured in the X-ray image represented by the input estimation image data to estimate the bone density. For example, when the X-ray image represented by the input estimation image data captures the lumbar vertebrae, the neural network 200 uses the learned parameter 110 for estimating the bone density of the lumbar vertebrae to estimate the bone density. In addition, when the X-ray image represented by the input estimation image data captures the proximal part of the femur, the neural network 200 uses the learned parameter 110 for estimating the bone density of the proximal part of the femur to estimate the bone density. The neural network 200 uses, for example, the learned parameter 110 indicated by the user through the input unit 50 among the plurality of learned parameters 110. In this case, the user indicates the learned parameter 110 used by the neural network 200 according to the type of bone captured in the X-ray image represented by the estimation image data input to the neural network 200.

[0139] In the learning of the neural network 200, a plurality of learning image data respectively representing a plurality of X-ray images of the same type of bone are used to generate the learned parameter 110 corresponding to the type of bone.

[0140] As described above, the estimation device 1 and the bone density estimation system 600 have been described in detail. However, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0141] Embodiment 2.

[0142] Figure 7 FIG. is an example showing the structure of the estimation device 1A according to the present embodiment. In the estimation device 1A, the approximator 280 further includes a second neural network 900. The second neural network 900 can detect a fracture based on the learned parameters 910. In addition, the estimation device 1A according to the present embodiment has the same structure as the estimation device 1 according to the first embodiment, and the description of the same structure is omitted. In addition, for convenience of explanation, the neural network 200 described in the above example is referred to as the first neural network 200. The second neural network 900 has, for example, the same structure as the first neural network 200.

[0143] The second neural network 900 can detect a fracture based on the same estimation image data as the estimation image data included in the estimation data 120 input to the first neural network 200. That is, based on one piece of estimation image data, the bone density can be estimated in the first neural network 200, and a fracture can be detected in the second neural network 900. In addition, the detection result 920 of the second neural network 900 may be output from the output layer 230 of the second neural network 900 in the same manner as in the above example.

[0144] In the learning of the second neural network 900, the learning image data of a non-fractured bone and the learning image data of a fractured bone are used to learn the parameters. In addition, in the training data, information indicating whether there is a current fracture and information indicating the fracture site for the bone in which the learning image data is captured are associated with each learning image data. In addition, the training data may include information indicating a past fracture history and information indicating a past fracture site. As a result, the second neural network 900 can detect the presence and location of a fracture in the bone in which the estimation image data is captured based on the estimation image data, and output the detection result 920.

[0145] In addition, the estimation device 1A according to the present embodiment may also be as Figure 8As shown, there is a determination unit 930 for determining whether the subject has osteoporosis. The determination unit 930 can compare and study the estimation result 300 of the first neural network 200 and the perception result 920 of the second neural network 900 to determine whether the subject has osteoporosis.

[0146] The determination unit 900 can also determine osteoporosis based on, for example, specific criteria or well-known guidelines, etc. Specifically, the determination unit 900 can determine that it is osteoporosis when the perception result 920 indicates a fracture at the vertebral body or the proximal part of the thigh bone. In addition, when the bone density estimation value 300 output by the first neural network 200 represents YAM, the determination unit 900 can also determine that it is osteoporosis when YAM is a value less than 80% and the perception result 920 indicates a fracture other than the vertebral body and the proximal part of the thigh bone. In addition, the determination unit 900 can also determine that it is osteoporosis when YAM represented by the bone density estimation value 300 indicates a value of 70% or less.

[0147] In addition, in the estimation device 1A according to the present embodiment, the approximator 280 can also be as Figure 9 shown, and further has a third neural network 950. The third neural network 950 can divide the bones of the subject based on the learned parameters 960 and the estimation image data included in the estimation data 120.

[0148] The third neural network 950 outputs part information indicating the part of the bone represented by each pixel data of the input estimation image data. Thereby, the bone captured in the X-ray image represented by the estimation image data can be divided. Sometimes the part information is called segment data.

[0149] For example, when the lumbar vertebra is captured in the X-ray image represented by the input estimation image data, the third neural network 950 outputs part information indicating which part of L1 - L5 of the lumbar vertebra is indicated by each pixel data of the estimation image data. For example, when a certain pixel data of the estimation image data represents L1 of the lumbar vertebra, the third neural network 950 outputs part information representing L1 as the part information corresponding to the pixel data.

[0150] The third neural network 950 uses the learned parameters 960 corresponding to the types of bones captured in the X-ray image represented by the estimation image data. The third neural network 950 has a plurality of learned parameters 960 corresponding to multiple types of bones respectively. The third neural network 950 uses the learned parameters 960 corresponding to the type of bone captured in the X-ray image represented by the input estimation image data to divide the bones captured in the X-ray image represented by the estimation image data. For example, when a lumbar vertebra is captured in the X-ray image represented by the input estimation image data, the third neural network 950 uses the learned parameters 960 corresponding to the lumbar vertebra to divide the lumbar vertebra into L1 to L5. The third neural network 950 uses, for example, the learned parameters 960 indicated by the user through the input unit 50 among the multiple learned parameters 960. In this case, the user indicates the learned parameters 960 used by the third neural network 950 according to the type of bone captured in the X-ray image represented by the estimation image data input to the third neural network 950.

[0151] The third neural network 950 may also divide the bones captured in the X-ray image represented by the input estimation image data into a first part in which an implant is embedded, a second part with a tumor, and a third part with a fracture. In this case, for each pixel data of the estimation image data, the third neural network 950 outputs part information indicating which of the first part, the second part, and the third part the pixel data indicates. When the pixel data indicates a part other than the first part, the second part, and the third part, the third neural network 950 outputs part information indicating that the pixel data indicates a part other than the first part, the second part, and the third part. When the third neural network 950 divides the bones captured in the X-ray image represented by the estimation image data into the first part, the second part, and the third part, it can also be said that the third neural network 950 senses the implant embedded in the bones captured in the X-ray image represented by the estimation image data, the fracture of the bones, and the tumor of the bones.

[0152] In the learning of the third neural network 950, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 960 corresponding to the type of the bone. When the third neural network 950 divides the bone captured in the X-ray image represented by the input estimation image data into a first part, a second part, and a third part, among the multiple learning image data, there are included learning image data representing X-ray images of cases where an implant is embedded, learning image data representing X-ray images of cases where there is a tumor in the bone, and learning image data representing X-ray images of cases where there is a fracture. In addition, in the training data, for each learning image data, there is included annotation information for dividing the bone represented by the learning image data. In the annotation information, for each pixel data of the learning image data corresponding thereto, there is included part information indicating the part of the bone represented by the pixel data.

[0153] In addition, the first neural network 200 may also estimate the bone density for each part divided in the third neural network 950. In this case, as Figure 10 shown, the estimation image data 121 and the part information 965 corresponding to each pixel data of the estimation image data 121 output by the third neural network 950 based on the estimation image data 121 are input to the first neural network 200. The first neural network 200 outputs the bone density estimation value 300 for each part divided in the third neural network based on the learned parameters 110 corresponding to the type of the bone captured in the X-ray image represented by the estimation image data 121. For example, when the third neural network 950 divides the cervical vertebra captured in the X-ray image represented by the estimation image data 121 into L1 to L5, the first neural network 200 separately outputs the bone density estimation value 300 for L1, the bone density estimation value 300 for L2, the bone density estimation value 300 for L3, the bone density estimation value 300 for L4, and the bone density estimation value 300 for L5.

[0154] In the learning of the first neural network 200, multiple learning image data respectively representing multiple X-ray images of the same type of bone are used to generate the learned parameters 110 corresponding to the type of the bone. In addition, in the training data, for each learning image data, there is included the reference bone density of each part of the bone represented by the learning image data.

[0155] The first neural network 200 uses, for example, the learned parameters 110 input from the user via the input unit 50 among the plurality of learned parameters 110. In this case, the user indicates the learned parameters 110 to be used by the first neural network 200 according to the type of bone captured in the X-ray image represented by the estimation target image data input to the first neural network 200.

[0156] In the case where the third neural network 950 classifies the bones captured in the X-ray image represented by the estimation target image data into a first part where an implant is embedded, a second part with a tumor, and a third part with a fracture, it is also possible to adjust the brightness of the first partial image data representing the first part, the second partial image data representing the second part with a tumor, and the third partial image data representing the third part in the estimation target image data. Figure 11 It is a diagram showing a structural example in this case.

[0157] As Figure 11 shown, the estimation target image data 121 and the part information 965 corresponding to each pixel data of the estimation target image data 121 output by the third neural network 950 based on the estimation target image data 121 are input to the adjustment unit 968. The adjustment unit 968 determines the first partial image data, the second partial image data, and the third partial image data included in the estimation target image data 121 based on the part information 965. Then, the adjustment unit 968 adjusts the brightness of the determined first partial image data, second partial image data, and third partial image data.

[0158] The adjustment unit 968 stores, for example, the brightness of the first part captured in a general X-ray image as the first reference brightness. In addition, the adjustment unit 968 stores the brightness of the second part captured in a general X-ray image as the second reference brightness. Moreover, the adjustment unit 968 stores the brightness of the third part captured in a general X-ray image as the third reference brightness. The adjustment unit 968 subtracts the first reference brightness from the brightness of the first partial image data to adjust the brightness of the first partial image data. In addition, the adjustment unit 968 subtracts the second reference brightness from the brightness of the second partial image data to adjust the brightness of the second partial image data. Moreover, the adjustment unit 968 subtracts the third reference brightness from the brightness of the third partial image data to adjust the brightness of the third partial image data. The adjustment unit 968 inputs the estimation target image data with the brightness of the first partial image data, the second partial image data, and the third partial image data adjusted in the estimation target image data as the estimation target image data with adjusted brightness to the first neural network 200. The first neural network 200 estimates the bone density of the bone captured in the X-ray image represented by the estimation target image data with adjusted brightness.

[0159] Here, it is not easy to accurately estimate the bone density based on the first part where the implant is embedded, the second part where the tumor is located, and the third part where the fracture is present. By adjusting the brightness of the first partial image data representing the first part, the second partial image data representing the second part, and the third partial image data representing the third part to be smaller as described above, it is possible to accurately estimate the bone density of the bone captured in the X-ray image represented by the estimation target image data.

[0160] In addition, the adjustment unit 968 may also input, as the estimation target image data with adjusted brightness, the estimation target image data in which the brightness of the first partial image data, the second partial image data, and the third partial image data is forcibly set to zero, into the first neural network 200.

[0161] In addition, the third neural network 950 may also detect only one of the implant, the fracture, and the tumor. In addition, the third neural network 950 may also detect only two of the implant, the fracture, and the tumor. That is, the third neural network 950 may also detect at least one of the implant, the fracture, and the tumor.

[0162] In addition, the estimation device 1A may not include the second neural network 900, but may include the first neural network 200 and the third neural network 950. In addition, the estimation device 1A may not include the first neural network 200, but may include at least one of the second neural network 900 and the third neural network 950.

[0163] As described above in detail for the estimation device 1A, the above description is an illustration in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not contradict each other. Moreover, countless examples that are not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0164] Embodiment 3.

[0165] Figure 12 FIG. is an example showing the structure of the estimation device 1B according to the present embodiment. The estimation device 1B includes a fracture prediction unit 980. The fracture prediction unit 980 can, for example, predict the probability of a fracture based on the estimation result 300 of the neural network 200 of the estimation device 1 according to Embodiment 1. Specifically, for example, an arithmetic expression 990 representing the relationship between the estimation result related to the bone density (such as bone density, etc.) and the probability of a fracture is obtained based on past literature, etc. The fracture prediction unit 980 stores the arithmetic expression 990. The fracture prediction unit 980 can predict the probability of a fracture based on the input estimation result 300 and the stored arithmetic expression 990.

[0166] In addition, the arithmetic expression 990 may also be an arithmetic expression representing the relationship between the estimated result related to bone density and the probability of fracture after bone screw implantation. As a result, it is possible to study whether to implant a bone screw and a treatment plan including drug administration.

[0167] In addition, the estimation device 1B may also include a second neural network 900. Further, the estimation device 1B may also include a third neural network 950.

[0168] As described above, the estimation device 1B has been described in detail, but the above description is illustrative in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

[0169] Embodiment 4.

[0170] In Figure 13 it shows the concept of the structure of the estimation system 801 of this embodiment.

[0171] The estimation system 801 of the present disclosure can, for example, estimate the future bone mass of a subject based on an image of the subject's bone such as an X-ray image. The estimation system 801 of the present disclosure includes a terminal device 802 and an estimation device 803. In addition, the bone mass is an index related to bone density and includes the concept of bone density.

[0172] The terminal device 802 can acquire input information I for input to the estimation device 803. The input information I may be, for example, an X-ray image or the like. In this case, the terminal device 802 may be a device for a doctor or the like to take an X-ray image of the subject. For example, the terminal device 802 may be a simple X-ray imaging device (in other words, a general X-ray imaging device or a Roentgen imaging device).

[0173] In addition, the terminal device 802 is not limited to a simple X-ray imaging device. For example, the terminal device 802 can also be an X-ray fluoroscopic imaging device, CT (Computed Tomography), MRI (Magnetic Resonance Imaging), SPECT (Single Photon Emission Computed Tomography)-CT, or tomosynthesis, etc. In this case, the input information I only needs to be, for example, an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image.

[0174] The estimation system 801 can be used, for example, for the diagnosis of osteoporosis and the like of patients who go to the hospital. The estimation system 801 of the present disclosure uses, for example, the terminal device 802 provided in the X-ray room to take an X-ray photograph of the patient. Then, the image data is transmitted from the terminal device 802 to the estimation device 803, and through the estimation device 803, it is possible to not only estimate the bone mass or bone density of the patient at the current time point, but also estimate the bone mass or bone density of the patient in the future that is further in the future than the time of the photograph.

[0175] In addition, the terminal device 802 may not directly transmit the input information I to the estimation device 803. In this case, for example, the input information I obtained by the terminal device 802 can also be stored in a storage medium, and the input information I is input to the estimation device 803 via the storage medium.

[0176] In Figure 14 shows the concept of the structure of the estimation device 803 according to the present embodiment.

[0177] The estimation device 803 can estimate the future bone mass or bone density of the subject based on the input information I obtained by the terminal device 802. In the estimation device 803, the future bone mass or bone density of the subject can be estimated according to the image data obtained by the terminal device 802, and the estimation result O is output.

[0178] The estimation device 803 includes an input unit 831, an approximator 832, and an output unit 833. The input unit 831 inputs the input information I from the terminal device 802. The approximator 832 can estimate the future bone mass or bone density based on the input information I. The output unit 833 can output the estimation result O predicted by the approximator 832.

[0179] The estimation device 803 includes various electronic components and circuits. As a result, the estimation device 803 can form each structural element. For example, the estimation device 803 can integrate multiple semiconductor elements to form at least one integrated circuit (e.g., IC: Integrated Circuit or LSI: Large Scale Integration), or further integrate multiple integrated circuits to form at least one unit, etc., thereby constituting each functional part of the estimation device 803.

[0180] The multiple electronic components can be, for example, active components such as transistors or diodes, or passive components such as capacitors. In addition, the multiple electronic components and the integrated circuits formed by integrating them can be formed by conventionally known methods.

[0181] The input unit 831 inputs the information used in the estimation device 803. For example, the input information I having an X-ray image obtained by the terminal device 802 is input to the input unit 831. The input unit 831 has a communication unit, and the input information I obtained by the terminal device 802 is directly input from the terminal device 802. In addition, the input unit 831 may include an input device capable of inputting the input information I or other information. The input device can be, for example, a keyboard, a touch screen, or a mouse.

[0182] Based on the information input to the input unit 831, the approximator 832 estimates the future bone mass or bone density of the subject. The approximator 832 has AI (Artificial Intelligence). The approximator 832 has a program that functions as AI, and various electronic components and circuits for executing the program. The approximator 832 has a neural network.

[0183] The approximator 832 performs a learning process in advance for the input-output relationship. That is, machine learning can be applied to the approximator 832 using learning data and training data, so that the approximator 832 can calculate the estimation result O based on the input information I. In addition, the learning data or training data can be the input information I input to the estimation device 803 and the data corresponding to the estimation result O output from the estimation device 803.

[0184] Figure 15 Conceptually shows the structure of the approximator 832 of the present disclosure.

[0185] The approximator 832 includes a first neural network 8321 and a second neural network 8322. The first neural network 8321 may be any neural network suitable for processing temporal information. For example, the first neural network 8321 may be a ConvLSTM network that combines LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network), etc. The second neural network 8322 may be, for example, a convolutional neural network constituted by using a CNN, etc.

[0186] The first neural network 8321 has an encoding unit E and a decoding unit D. The encoding unit E can extract the temporal variation and the feature amount of the position information of the input information I. The decoding unit D can calculate a new feature amount based on the feature amount extracted by the encoding unit E, the temporal variation of the input information I, and the initial value.

[0187] Figure 16 Concept of the structure of the first neural network 8321 of the present disclosure is shown.

[0188] The encoding unit E has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers E1. The decoding unit D has a plurality of ConvLSTM (Convolutional Long Short-Term Memory) layers D1. Each of the encoding unit E and the decoding unit D may have three or more ConvLSTM layers E1, D1. In addition, the number of the plurality of ConvLSTM layers E1 and the number of the plurality of ConvLSTM layers D1 may be the same.

[0189] In addition, the content learned by each of the plurality of ConvLSTM layers E1 may be different. The content learned by each of the plurality of ConvLSTM layers D1 may be different. For example, for a certain ConvLTSM layer, it learns about fine content such as the change of each pixel, and for another ConvLSTM layer, it learns about rough content such as the change of the overall image.

[0190] Figure 17 Concept of the structure of the second neural network 8322 is shown.

[0191] The second neural network 8322 has a transformation unit C. The transformation unit C can transform the feature amount calculated by the decoding unit D of the first neural network 8321 into bone mass or bone density. The transformation unit C has a plurality of convolutional layers C1, a plurality of pooling layers C2, and a fully connected layer C3. The fully connected layer C3 is at the front stage of the output unit 33. Moreover, in the transformation unit C, the convolutional layer C1 and the pooling layer C2 are alternately arranged between the first neural network 8321 and the fully connected layer C3.

[0192] In addition, during the learning of the approximator 832, the learning data is input to the encoding unit E of the approximator 832. The training data is compared with the output data output from the transformation unit C of the approximator 832 during the learning of the approximator 832. The training data is data representing values measured using a conventional bone density measuring device.

[0193] The output unit 833 can display the estimation result O. The output unit 833 is, for example, a liquid crystal display or an organic EL display. The output unit 833 can display various information such as characters, symbols, and graphics. The output unit 833 can, for example, display numbers or images.

[0194] The estimation device 803 of the present disclosure further has a control unit 834 and a storage unit 835. The control unit 834 can uniformly manage the operation of the estimation device 803 by controlling other structural elements of the estimation device 803.

[0195] The control unit 834, for example, has a processor. The processor can include, for example, one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits (ASICs), digital signal processing devices, programmable logic devices, or combinations of these devices or any calibrated combinations, or other basic devices or calibrated combinations. The control unit 834, for example, includes a CPU.

[0196] The storage unit 835 includes, for example, a non-temporary storage medium such as a RAM (Random Access Memory) or a ROM (Read-Only Memory) that can be read by the CPU of the control unit 834. A control program for controlling the estimation device 803 such as firmware is stored in the storage unit 835. In addition, the input information I input, the learning data for learning, and the training data can also be stored in the storage unit 835.

[0197] The processor of the control unit 834 can execute one or more data calculation programs or processes according to the control program of the storage unit 835. Various functions of the control unit 834 are realized by the CPU of the control unit 834 executing the control program in the storage unit 835.

[0198] In addition, the control unit 834 may also perform other processes as preprocessing of computational processing as needed.

[0199] <An example of input information, learning data, and training data>

[0200] The input information (hereinafter, also referred to as the first input information I1) has image data of a bone of an object whose bone mass or bone density is to be estimated. The image data may be, for example, a simple X-ray image. The object whose bone mass or bone density is to be estimated is, for example, a human. In this case, it can be said that the first input information I1 is image data of a simple X-ray image of a human bone. A simple X-ray image is a two-dimensional image and is also referred to as a general X-ray image or a Roentgen image.

[0201] The first input information I1 is preferably a simple X-ray image that is relatively easy to obtain, but is not limited thereto. For example, there are cases where bone mass or bone density can be estimated more accurately by using an X-ray fluoroscopic image, a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, a bone scintigraphy image, or a tomosynthesis image as input information.

[0202] In addition, the object whose bone mass or bone density is to be estimated may be other than a human. For example, the object whose bone mass or bone density is to be estimated may be an animal such as a dog, a cat, or a horse. In addition, the bones that are the objects are mainly cortical bone and cancellous bone of biological origin, but the bones that are the objects may also include artificial bones mainly composed of calcium phosphate or regenerated bones artificially manufactured through regenerative medicine or the like.

[0203] As the shooting part of the X-ray image, for example, it may be any of the neck, chest, waist, proximal part of the femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, or jaw joint. In addition, parts other than bones may also be shot in the X-ray image. For example, in the case of a chest simple X-ray image, it may also include an image of the lungs and an image of the thoracic vertebrae. The X-ray image may be a front image of the object part shot from the front or a side image of the object part shot from the side.

[0204] The learning data or the training data may be data corresponding to the input information I input to the estimation device 803 and the estimation result O output from the estimation device 803.

[0205] The learning data has the same type of information as the first input information I1. For example, if the first input information I1 is a simple X-ray image, the learning data only needs to have a simple X-ray image. Further, when the first input information I1 is a chest simple X-ray image, the learning data only needs to have a chest simple X-ray image.

[0206] The learning image data including multiple simple X-ray images of bones is included in the learning data. Among the shooting parts of the multiple learning image data, for example, at least one of the neck, chest, waist, proximal femur, knee joint, ankle joint, shoulder joint, elbow joint, wrist joint, finger joint, and jaw joint is included. In the learning data, either a part of the types of the 11 types of image data can be included, or all types of image data can be included. In addition, among the multiple learning image data, a front image and a side image can be included.

[0207] In the learning data, bones of multiple different individuals are respectively photographed. For each of the multiple learning image data, as training data, the measured values of the bone mass or bone density of the subject of each learning image data are established in correspondence. The measured values of the bone mass or bone density are the measured values measured at a time period almost the same as the time period when the learning image data is photographed.

[0208] In addition, the learning image data of the learning data can also be a series of data with different time axes of photographing the same person. That is, the learning image data is the first learning data with an X-ray image of a bone and an image of the same person as the first learning data, and can also include the second learning data with an X-ray image photographed after the first learning data.

[0209] In addition, the learning image data of the learning data can also be a data set of photographing the same part of others with different ages, etc. In addition, the learning image data of the learning data can also be a series of data with different time axes of photographing the same person and the same part.

[0210] The learning data and the first input information I1 can also use input information obtained by compressing the image data representing the gray scale of the simple X-ray image photographed by a simple X-ray photographing device (in other words, a general X-ray photographing device or a Roentgen photographing device) and reducing the number of gray scales. For example, consider the case where the number of pixel data of the image data is more than (1024×640) and the number of bits of the pixel data is 16 bits. In this case, the input information obtained by compressing the number of pixel data to, for example, (256×256), (1024×512), or (1024×640) and reducing the number of bits of the pixel data to 8 bits is used as the first input information I1 and the learning data.

[0211] The training data includes, for each of the plurality of learning image data included in the learning data, a measured value of the bone mass or bone density of the bone captured in the simple learning X-ray image represented by the learning image data. The bone mass or bone density may be measured, for example, by a DEXA (dual-energy X-ray absorptiometry) method or an ultrasonic method.

[0212] <Neural network learning example>

[0213] The control unit 834 performs machine learning using the learning data and the training data on the approximator 832, so that the approximator 832 can calculate the estimated result O related to the bone mass or bone density based on the input information I. The approximator 832 is optimized by the well-known machine learning using the training data. The approximator 832 performs calculations based on the learning data input to the encoding unit E, and adjusts the variable parameters in the approximator 832 so that the difference between the pseudo estimated result output from the conversion unit C and the training data is small.

[0214] Specifically, the control unit 834 inputs the learning data in the storage unit 835 to the encoding unit E. When the learning data is input to the encoding unit E, the control unit 834 inputs the plurality of pixel data constituting the learning image data to the plurality of artificial neurons constituting the encoding unit E. Then, when the learning image data is input to the encoding unit E, the control unit 834 adjusts the parameters so that the error of the estimated result O output from the conversion unit C relative to the bone mass or bone density actual measurement value corresponding to the learning image data becomes smaller. The adjusted parameters become the parameters after learning and are stored in the storage unit 835.

[0215] As a parameter adjustment method, for example, an error back propagation method may be used. The parameters include, for example, parameters used by the encoding unit E, the decoding unit D, and the transformation unit C. Specifically, the parameters include weighting coefficients used by the ConvLSTM layers of the encoding unit E and the decoding unit D, and the convolutional layer and the fully connected layer of the transformation unit C.

[0216] As a result, the approximator 832 performs operations based on the learned parameters for the input information I input to the encoder E, and outputs the estimated result O from the transform unit C. When the X-ray image data as the input information I is input to the encoder E, the plurality of pixel data constituting the image data are respectively input to the plurality of artificial neurons constituting the input unit 831. Then, the ConvLSTM layer, the convolution layer, and the fully connected layer can perform operations using the weighting coefficients included in the learned parameters, thereby outputting the estimated result O.

[0217] As described above, in the estimation system 801, using the image data of the simple X-ray image, learning of the approximator 832 and estimation of bone mass or bone density in the approximator 832 are performed. Therefore, input information I can be input to the estimation system 801, and future bone mass or bone density can be output as the estimation result O.

[0218] The estimation result O of the estimation system 801 only needs to be the estimation result on a certain day in the future compared to the acquisition date of the input information I. For example, the estimation system 1 can estimate the bone mass or bone density from 3 months to 50 years after the shooting, more preferably from 6 months to 10 years after the shooting.

[0219] The estimation result O only needs to be output as a value. For example, it can be represented by at least one of YAM (Young Adult Mean), T-score, and Z-score. For example, from the output unit 833, the estimated value represented as YAM can be output, or the estimated value represented as YAM, the estimated value represented as T-score, and the estimated value represented as Z-score can be output.

[0220] In addition, the estimation result O can also be output as an image. When the estimation result O is an image, for example, an image imitating an X-ray image only needs to be displayable. In addition, an image imitating an X-ray image is an image that imitates an X-ray image. In addition, when using ConvLSTM to learn a series of data of the same person, the same part, and different time axes, the temporal change of the image can be predicted. Thus, a future image can be generated based on the X-ray image at one time point of another patient.

[0221] In the learning data and the input information I, in addition to bones, internal organs, muscles, fat, or blood vessels can also be photographed. Even in this case, highly accurate estimation can be performed.

[0222] The first input information I1 can also have the individual data (first individual data) of the subject. The so-called first individual data can be, for example, age information, gender information, height information, weight information, or fracture history, etc. As a result, highly accurate estimation can be performed.

[0223] The first input information I1 can also have the second individual data of the subject. The so-called second individual data can also include information such as blood pressure, lipids, cholesterol, neutral fat, and blood glucose level. As a result, highly accurate estimation can be performed.

[0224] The first input information I1 can also have the lifestyle information of the subject. The so-called lifestyle information can be information such as drinking habit, smoking habit, exercise habit, or eating habit, etc. As a result, highly accurate estimation can be performed.

[0225] The first input information I1 may also include the bone metabolism information of the subject. The so-called bone metabolism information can be, for example, bone resorption ability or bone formation ability. They can be measured, for example, by at least one of N-terminal telopeptide of type I collagen cross-linking (NTX), C-terminal telopeptide of type I collagen cross-linking (CTX), tartrate-resistant acid phosphatase (TRACP-5b), deoxypyridinoline (DPD) as bone resorption markers, bone-specific alkaline phosphatase (BAP), N-propeptide of type I collagen cross-linking (P1NP) as bone formation markers, and undercarboxylated osteocalcin (ucOC) as bone-related matrix markers. The bone resorption markers can be measured using serum or urine as the test body.

[0226] In the estimation system 801, as the input information I, the second input information I2 related to the future planned actions of the subject can also be further input. The so-called second input information I2 can be, for example, information related to individual data that is planned to be improved or has been improved, or planned or improved living habits and exercise and eating habits. Specifically, the second input information I2 can be information such as weight data, drinking habits, smoking habits, sun exposure time, daily steps or walking distance, dairy product intake, or intake of foods rich in vitamin D such as improved fish and mushrooms. As a result, the estimation system 801 can present an estimated result O in which the future bone mass or bone density has been improved.

[0227] In addition, the second input information I2 can also be, for example, information related to living habits that are expected to deteriorate. As a result, the estimation system can present an estimated result O in which the future bone mass or bone density has deteriorated.

[0228] In the estimation system 801, as the input information I, the third input information I3 related to the therapy for the subject can also be further input. The so-called third input information I3 is, for example, information related to physical therapy or drug therapy. Specifically, the third input information I3 can be at least one of calcium tablets, female hormone drugs, vitamin drugs, bisphosphonate drugs, SERM (Selective Estrogen Receptor Modulator) drugs, calcitonin drugs, thyroid hormone drugs, and denosumab drugs.

[0229] In the estimation system 801, the estimated result O can also output the first result O1 based only on the first input information I1, and the second result O2 based on at least one of the first input information I1 and the second and third input information I2 and I3. As a result, it is possible to compare the effects of future planned actions.

[0230] In the estimation system 801, the estimation result O outputs not only the future bone mass or bone density but also the current result. As a result, it is possible to compare the changes in bone mass or bone density over time.

[0231] Figure 18 Concept of the structure of the approximator 832 showing another embodiment of the estimation system 1.

[0232] The estimation device 803 of the estimation system 801 may also have a first approximator 832a and a second approximator 832b. That is, in addition to the above-mentioned approximator 832 (first approximator 832a), a second approximator 832b may also be provided. The second approximator 832b may be, for example, a CNN.

[0233] In this case, in the estimation system 801, the first approximator 832a outputs the first image and the first value as the first estimation result O1 to the first output unit 833a. Further, the second approximator 832b outputs the second value as the second estimation result O2 to the second output unit 833b based on the first image from the first output unit 833a. As a result, it is possible to compare the first value and the second value as the estimation result O of the future bone mass or bone density.

[0234] The estimation system 801 may also output a third value based on the first value and the second value as the estimation result O. As a result, for example, it is possible to set the result (third value) obtained by calibrating the first value based on the second value as the estimation result O.

[0235] As described above in detail for the estimation system 801, the above description is an example in all aspects, and the present disclosure is not limited thereto. In addition, the above various examples can be combined and applied as long as they do not conflict with each other. Moreover, countless examples not illustrated can be interpreted as examples obtained by assuming without departing from the scope of the present disclosure.

Claims

1. A presumption device comprising: A presumption unit that, based on input information of a first image having a simple X-ray image of a human skeleton and based on learned parameters, presumes the bone density of a bone, The learned parameters are obtained by learning the relationship between learning data and training data, and the learning data includes image data of a plurality of simple X-ray images of bones, In the training data, for each of the plurality of learning image data included in the learning data, it includes a measured value of the bone density of a person having a bone photographed in the learning simple X-ray image represented by the learning image data.

2. The presumption device according to claim 1, wherein The learned parameters are set based on learning data including a second image of a human skeleton and training data including the bone density of a person having a bone photographed in the second image.

3. The presumption device according to claim 2, wherein The training data includes bone density measured by irradiating X-rays from the front of a human skeleton.

4. The presumption device according to claim 2 or 3, wherein The training data includes the bone density of a part not included in the part of the skeleton photographed in the first image.

5. The presumption device according to claim 2 or 3, wherein The input information has a first image group, the first image group includes a plurality of first images of a skeleton of the same person, that is, a plurality of first images in which the orientations of the parts of the skeleton photographed in these first images are different from each other, The plurality of first images constituting the first image group are input to the presumption unit in parallel, The learning data includes a second image group, the second image group includes a plurality of second images of a skeleton of the same person, that is, a plurality of second images in which the orientations of the parts of the skeleton photographed in these second images are different from each other.

6. The presumption device according to claim 5, wherein The first image group includes at least one front image and a side image.

7. The presumption device according to claim 2 or 3, wherein The learned parameters are set based on learning data including a second image of a skeleton of a part different from the skeleton photographed in the first image.

8. The presumption device according to claim 2 or 3, wherein The learned parameters are set based on learning data including a second image of a skeleton of the same part as the skeleton photographed in the first image.

9. The presumption device according to claim 2 or 3, wherein The learned parameters are set based on learning data including a second image of a human skeleton and training data, The training data includes: The bone density of the bone photographed in the second image, the bone density of a plurality of parts of the skeleton of a person having the bone photographed in the second image, or the bone density of the bone of a person having the bone photographed in the second image.

10. The presumption device according to claim 2 or 3, wherein The learned parameters are set based on training data including the bone density of a human bone and learning data including a plurality of second images of the human skeleton of a person having the bone with the bone density included in the training data.

11. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data of a second image including a skeleton of the same part as the skeleton photographed in the first image, The orientation of the skeleton photographed in the first image is different from the orientation of the skeleton photographed in the second image.

12. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data of a second image including a skeleton of the same part as the skeleton photographed in the first image, The orientation of the skeleton photographed in the first image is the same as the orientation of the skeleton photographed in the second image.

13. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data including a second image of a human skeleton and information related to the health state of the person having the bone photographed in the second image.

14. The estimation device according to claim 13, wherein, The information related to the health state includes at least one of drinking habit information, smoking habit information, and information on the presence or absence of a fracture history.

15. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on a plurality of second images of a human skeleton, that is, a plurality of second images of a human skeleton photographed from different orientations.

16. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data including a plurality of second images with different time axes of the same person.

17. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data including a plurality of second images of the skeletons of different persons.

18. The estimation device according to claim 2 or 3, wherein, The training data has the bone density of at least one of the lumbar spine, proximal femur, radius, hand bone, tibia, and heel bone.

19. The estimation device according to claim 2 or 3, wherein, The training data has a bone density measured based on at least one of dual-energy X-ray absorptiometry and ultrasonic method.

20. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data of a simple X-ray image of a human skeleton, that is, a second image, and training data including the bone density of the skeleton measured by dual-energy X-ray absorptiometry, The irradiation orientation of the X-ray at the time of photographing the second image is different from the irradiation orientation of the X-ray at the time of measuring the bone density.

21. The estimation device according to claim 2 or 3, wherein, The learned parameters are set based on learning data including a simple X-ray image of a human skeleton, i.e., a second image, and training data including bone density measured by dual-energy X-ray absorptiometry of the skeleton. The irradiation direction of the X-ray at the time of taking the second image is the same as the irradiation direction of the X-ray at the time of measuring the bone density.

22. The estimation device according to claim 2 or 3, wherein The training data includes the bone density of at least one of the lumbar spine and the proximal part of the femur.

23. The estimation device according to any one of claims 1 to 3, wherein The first image includes a frontal image of the lumbar spine or chest, or an image of the knee joint.

24. The estimation device according to any one of claims 1 to 3, wherein The first image is at least one of a reduced image of an image obtained by a simple X-ray imaging device and an image with a reduced gray scale of an image obtained by a simple X-ray imaging device.

25. The estimation device according to claim 1, wherein The estimation device further comprises: A sensing unit that senses a fracture of a bone captured in the first image based on the first image.

26. The estimation device according to claim 25, wherein The sensing unit senses the location of a fracture of a bone captured in the first image based on the first image.

27. The estimation device according to claim 25 or 26, wherein The sensing unit senses the presence or absence of a fracture of a bone captured in the first image based on the first image.

28. The estimation device according to claim 27, wherein The estimation device further comprises: A determination unit that determines whether a person having the bone captured in the first image has osteoporosis based on the fracture sensing result in the sensing unit.

29. The estimation device according to claim 28, wherein The determination unit determines whether a person having the bone captured in the first image has osteoporosis based on the fracture sensing result in the sensing unit and the estimation result in the estimation unit.

30. The estimation device according to claim 25 or 26, wherein The estimation device comprises: An adjustment unit that adjusts the brightness of a partial image of a fracture site of a bone captured in the first image in the first image, The sensing unit senses a fracture of a bone captured in the first image based on the first image whose brightness of the partial image has been adjusted.

31. The estimation device according to claim 25 or 26, wherein The sensing unit senses a fracture of a bone captured in the first image based on the first image and based on second learned parameters, The second learned parameters are set based on second learning data including a plurality of second images and second training data including information indicating the presence or absence of a fracture of a bone captured in each second image, and the plurality of second images include images of bones at non-fractured sites and images of bones at fractured sites.

32. The estimation device according to claim 31, wherein The second image is a front image of a simple X-ray image of a human skeleton.

33. The estimation device according to claim 31, wherein the second learning data includes a second image of a skeleton of a part different from the skeleton photographed in the first image.

34. The estimation device according to claim 31, wherein the second learning data includes a second image of a skeleton of a part the same as the skeleton photographed in the first image.

35. The estimation device according to claim 25 or 26, wherein the first image is at least one of an image obtained by reducing an image obtained by a simple X-ray photographing device and an image with a reduced gray scale of an image obtained by a simple X-ray photographing device.

36. The estimation device according to claim 25 or 26, wherein the bone density estimated by the estimation unit is at least one of bone mineral density per unit area, bone mineral density per unit volume, YAM, T-score, and Z-score.

37. The estimation device according to claim 25 or 26, wherein the first image is a front image of a simple X-ray image of a human chest.

38. The estimation device according to claim 31, wherein the second image is an image of a human chest.

39. The estimation device according to claim 1, wherein the estimation device further includes: a processing unit that segments the simple X-ray image based on the input information and parameters completed in the second learning.

40. The estimation device according to claim 39, wherein the parameters completed in the second learning are set based on second learning data including a plurality of second simple X-ray images in which bones are photographed and second training data including annotation information indicating specific parts of the bones photographed in the plurality of second simple X-ray images.

41. The estimation device according to claim 39 or 40, wherein the segmented area specified by the segmentation includes at least one of a part of the simple X-ray image, L1 to L5 of the lumbar vertebra, a part where an implant is implanted, a part having a tumor, and a part having a fracture.

42. The estimation device according to claim 39 or 40, wherein the simple X-ray image is a front image or a side image.

43. The estimation device according to claim 39 or 40, wherein the estimation unit estimates the bone density of the segmented area specified by the segmentation based on the input information and parameters completed in the third learning.

44. The estimation device according to claim 43, wherein the segmented area includes a plurality of parts included in the first skeleton photographed in the simple X-ray image, and the estimation unit estimates the bone density for each of the plurality of parts.

45. The estimation device according to claim 44, wherein the parameters completed in the third learning are set based on third learning data including learning images of a third skeleton of a third person and third training data including the bone density of the third person.

46. The estimation device according to claim 45, wherein The third training data includes bone density of a part different from the first bone captured in the plain X-ray image.

47. The estimation device according to claim 45, wherein, The third bone is of the same type as the first bone captured in the plain X-ray image.

48. The estimation device according to claim 46, wherein, The orientation of the first bone captured in the plain X-ray image is the same as the orientation of the third bone captured in the learning image.

49. The estimation device according to claim 45, wherein, The learning image is a plain X-ray image.

50. The estimation device according to claim 45, wherein, The third learning data includes health-related information related to the health status of the third person.

51. The estimation device according to claim 50, wherein, The health-related information includes at least one of age information, gender information, height information, weight information, alcohol drinking habit information, smoking habit information, and information on the presence or absence of a fracture history.

52. The estimation device according to claim 45, wherein, The third training data includes bone density measured based on at least one of dual-energy X-ray absorptiometry and ultrasonic method.

53. The estimation device according to claim 52, wherein, The X-ray irradiation orientation at the time of capturing the learning image is different from the X-ray irradiation orientation at the time of measuring the bone density included in the third training data.

54. The estimation device according to claim 45, wherein, The third training data includes the bone density of the lumbar spine of the third person.

55. The estimation device according to claim 45, wherein, The third training data includes the bone density of the proximal part of the femur of the third person.

56. A method of estimation, which is a method of estimation in an estimation device. In the method of estimation, Based on the input information of the first image with a plain X-ray image of a human bone, the bone density of the bone is estimated based on the learned parameters. The learned parameters are obtained by learning the relationship between the learning data and the training data. In the learning data, it includes image data of a plurality of plain X-ray images capturing the bone. In the training data, for each of the plurality of learning image data included in the learning data, it includes a measured value of the bone density of a person with the bone captured in the learning plain X-ray image shown by the learning image data.

57. The method of estimation according to claim 56, wherein, Based on the first image, a fracture of the bone captured in the first image is detected.

58. An estimation system, comprising: An estimation unit that estimates the bone density of a bone based on the input information of the first image with a plain X-ray image of a human bone, based on the learned parameters. The learned parameters are obtained by learning the relationship between the learning data and the training data. In the learning data, it includes image data of a plurality of plain X-ray images capturing the bone. In the training data, for each of the multiple learning image data included in the learning data, it includes the measured value of the bone density of the person with the bone photographed in the learning simple X-ray image shown by the learning image data.

Images