Inference device and inference method

Abstract

To provide an inference device that improves inference accuracy of a bone state by using an X-ray image.SOLUTION: Provided is an inference device including at least one memory and at least one processor. The at least one processor performs inferring an internal organ state from an X-ray image by using a first model, and inferring a bone state from the X-ray image by using a second model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an inference device and an inference method. [Background technology]

[0002] A technique is known for screening for lung cancer from a patient's chest X-ray images using a trained lung cancer screening model. Also, a technique is known for screening for osteoporosis from a patient's chest X-ray images using a trained osteoporosis screening model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6744614 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide an inference device that improves the accuracy of inferring the state of bones using X-ray images. [Means for solving the problem]

[0005] An inference device according to one aspect of the present disclosure includes at least one memory and at least one processor, the at least one processor being configured to infer a state of an internal organ from an X-ray image using a first model, and to infer a state of a bone from the X-ray image using a second model. [Brief description of the drawings]

[0006] [Figure 1] 2 is a block diagram showing an example of a functional configuration of the inference device according to the present embodiment. FIG. [Diagram 2] 4 is a flowchart illustrating an example of an inference method according to the present embodiment. [Diagram 3]2 is a block diagram showing an example of a functional configuration of the inference device according to the present embodiment. FIG. [Figure 4] 4 is a flowchart illustrating an example of an inference method according to the present embodiment. [Diagram 5] 2 is a block diagram showing an example of a functional configuration of the learning device according to the present embodiment. FIG. [Figure 6] 2 is a block diagram showing an example of a functional configuration of the learning device according to the present embodiment. FIG. [Figure 7] FIG. 2 is a block diagram showing an example of the hardware configuration of the inference device and the learning device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0008] [Outline of inference device] One embodiment of the present disclosure is an inference device that infers a lung condition (an example of a state of an internal organ) and a bone condition using the same chest X-ray image (an example of an X-ray image). The lung condition is, for example, a lung-related disease such as lung cancer or pneumonia. The bone condition is, for example, osteoporosis, osteopenia, or abnormally high values ​​(when bone density is high and there is an asymptomatic fracture). In addition, the bone condition may also be bone density, YAM, bone mineral density, T-score, or Z-score.

[0009] The inference device in this embodiment has a function for improving the accuracy and convenience of an osteoporosis screening model (an example of a second model) that has been trained to infer the condition of bones using chest X-ray images, by utilizing a lung cancer screening model (an example of a first model) that has been trained to infer the condition of the lungs using chest X-ray images.

[0010] The inference device in this embodiment utilizes the evaluation result that the inference accuracy of the osteoporosis screening model is lower for chest X-ray images inferred as "abnormal findings" by the lung cancer screening model than for chest X-ray images inferred as "normal" by the lung cancer screening model. The inference device in this embodiment utilizes the trained lung cancer screening model as a pre-stage, and improves the inference accuracy of the osteoporosis screening model by excluding chest X-ray images inferred to have lung abnormalities from the inference targets of the osteoporosis screening model. In addition, the inference device in this embodiment is highly convenient for medical institutions and the like that perform screening, since it can examine the lung condition and the bone condition with high accuracy using the same chest X-ray image.

[0011] An embodiment of the present disclosure aims to improve the accuracy of inferring bone conditions using chest X-ray images. According to the embodiment of the present disclosure, the lung conditions and bone conditions can be examined with high accuracy using the same chest X-ray image, thereby improving the convenience of medical institutions and the like that perform the examinations.

[0012] [Functional configuration of the inference device] A functional configuration of an inference device 1 according to an embodiment of the present disclosure will be described. Fig. 1 is a block diagram showing an example of a functional configuration of the inference device 1 according to the present embodiment. The inference device 1 according to the present embodiment may include a chest X-ray image receiving unit 10, a chest X-ray image storage unit 12, a lung cancer inference unit 14, a lung cancer screening model 16, a chest X-ray image selecting unit 18, an osteoporosis inference unit 20, an osteoporosis screening model 22, a clinical information receiving unit 24, and an output unit 26.

[0013] The inference device 1 may function as a chest X-ray image receiving unit 10, a chest X-ray image storage unit 12, a lung cancer inference unit 14, a lung cancer screening model 16, a chest X-ray image selecting unit 18, an osteoporosis inference unit 20, an osteoporosis screening model 22, a clinical information receiving unit 24, and an output unit 26 by executing a pre-installed program. The inference device 1 is an information processing device such as a personal computer, a tablet terminal, a smartphone, a server, etc. For example, a doctor inputs a chest X-ray image of a patient to the information processing device and receives an inference result as an output.

[0014] The chest X-ray image receiving unit 10 may receive a chest X-ray image to be inferred. For example, the chest X-ray image receiving unit 10 receives a chest X-ray image of an examinee who is examining the condition of the lungs and the condition of the bones. The chest X-ray image receiving unit 10 may receive the chest X-ray image of the examinee from a chest X-ray image capturing device. The chest X-ray image receiving unit 10 may also receive the chest X-ray image of the examinee from an external device that stores the chest X-ray image captured by the imaging device. The chest X-ray image of the examinee may be one or multiple images including chest X-ray images captured in the past.

[0015] The chest X-ray image storage unit 12 may store the chest X-ray image of the examinee received by the chest X-ray image receiving unit 10. The lung cancer inference unit 14 may infer the state of the lungs from the chest X-ray image using the lung cancer screening model 16. The lung cancer screening model 16 has been trained to infer the state of the lungs from the chest X-ray image using the chest X-ray image obtained in the lung cancer screening and the doctor's findings (correct answer judgment label) as training data. The lung cancer screening model 16 can be trained using an existing supervised learning method that derives a model that infers (predicts) the correct answer from training data (teacher data) with a correct answer judgment label. The lung cancer inference unit 14 may infer whether the lungs are "abnormal" or "normal" from the chest X-ray image.

[0016] The chest X-ray image selection unit 18 performs a process of excluding chest X-ray images of subjects for which the inference result using the lung cancer screening model 16 indicates ``abnormal findings'' in the lungs from the chest X-ray images to be inferred for inferring the condition of the bones using the osteoporosis screening model 22.

[0017] Specifically, the chest X-ray image selection unit 18 may select chest X-ray images for which the inference result is “normal” from among the chest X-ray images for which the lung cancer inference unit 14 has performed inference using the lung cancer screening model 16, and provide them to the osteoporosis inference unit 20. In addition, the chest X-ray image selection unit 18 may select chest X-ray images for which the inference result is “abnormal findings” in the lungs from among the chest X-ray images for which the lung cancer inference unit 14 has performed inference using the lung cancer screening model 16, so as not to provide them to the osteoporosis inference unit 20.

[0018] The osteoporosis inference unit 20 may infer the bone condition from the chest X-ray image using the osteoporosis screening model 22. The osteoporosis inference unit 20 may infer the bone condition from the chest X-ray image and clinical information of the examinee using the osteoporosis screening model 22. The clinical information of the examinee may be, for example, biological information such as the age and sex of the examinee, or information on the medical history of the examinee.

[0019] The osteoporosis screening model 22 may be trained to infer the condition of bones from chest X-ray images using chest X-ray images obtained in osteoporosis screening and the doctor's findings (correct answer judgment label) or the results of measurement by DXA (Dual-energy X-ray Absorptiometry) as training data. The DXA method is a measurement method that measures bone components by applying X-rays of two types of energy to a measurement site to distinguish between bone and other soft tissues.

[0020] The osteoporosis screening model 22 may be trained to infer the bone condition from a chest X-ray image using a chest X-ray image obtained in an osteoporosis screening, clinical information of the examinee, and a doctor's findings or a result of measurement by a DXA method as training data. In addition, the osteoporosis screening model 22 may be trained using an existing supervised learning method that derives a model that infers a correct answer from training data with a judgment label that is the correct answer.

[0021] The osteoporosis inference unit 20 is provided with a chest X-ray image for which the result of inference by the lung cancer inference unit 14 is "normal." The osteoporosis inference unit 20 may infer the bone condition from the chest X-ray image of the examinee provided from the chest X-ray image selection unit 18, using the osteoporosis screening model 22. The osteoporosis inference unit 20 may infer the bone condition from the chest X-ray image of the examinee and the clinical information of the examinee provided from the chest X-ray image selection unit 18, using the osteoporosis screening model 22.

[0022] The osteoporosis inference unit 20 may infer "suspected osteoporosis", "suspected osteopenia", or "normal" from a chest X-ray image. The osteoporosis inference unit 20 may infer "suspected osteoporosis", "suspected osteopenia", or "normal" from a chest X-ray image and clinical information of the examinee. The osteoporosis inference unit 20 may infer "suspected osteoporosis", "suspected osteopenia", "abnormally high value", or "normal". The osteoporosis inference unit 20 may infer "suspected osteoporosis", "suspected osteopenia", "abnormally high value", or "normal" from a chest X-ray image and clinical information of the examinee.

[0023] The clinical information receiving unit 24 may receive clinical information of the examinee and provide the clinical information of the examinee to the osteoporosis inference unit 20. The output unit 26 may output the result of the inference of the lung condition by the lung cancer inference unit 14 and the result of the inference of the bone condition by the osteoporosis inference unit 20.

[0024] For example, when the result of the inference of the lung condition by the lung cancer inference unit 14 is "abnormal findings" in the lungs, the output unit 26 does not output the result of the inference of the bone condition by the osteoporosis screening model 22, since the accuracy of the inference of the bone condition by the osteoporosis screening model 22 is lower than when the result of the inference of the lung condition by the lung cancer inference unit 14 is "normal". Note that the output unit 26 may output the result of the inference of the bone condition by the osteoporosis screening model 22, after clearly indicating that the accuracy of the inference of the bone condition is low because the result of the inference of the lung condition is "abnormal findings".

[0025] When the result of the inference of the lung condition by the lung cancer inference unit 14 is "normal", the output unit 26 may output the result of the inference of the bone condition by the osteoporosis screening model 22, since the inference accuracy of the bone condition by the osteoporosis screening model 22 is higher than when the result of the inference of the lung condition by the lung cancer inference unit 14 is "abnormal findings present" in the lungs. Note that the output unit 26 may output the result of the inference of the bone condition by the osteoporosis screening model 22, after clearly indicating that the inference accuracy of the bone condition is high.

[0026] [Inference method processing procedure] An inference device 1 according to an embodiment of the present disclosure executes an inference method, for example, in the process steps shown in Fig. 2. Fig. 2 is a flowchart showing an example of the inference method according to this embodiment.

[0027] In step S10, lung cancer inference unit 14 of inference device 1 may infer the state of the lungs from the chest X-ray image using lung cancer screening model 16. In step S12, chest X-ray image selection unit 18 of inference device 1 may determine whether or not the result of inference using lung cancer screening model 16 is "abnormal findings" in the lungs.

[0028] If the result of the inference using the lung cancer screening model 16 is "abnormal findings present" in the lungs, the chest X-ray image selection unit 18 proceeds to the process of step S14. In step S14, the chest X-ray image selection unit 18 may exclude the chest X-ray image for which the lung cancer inference unit 14 inferred abnormality in the lungs in step S10 from the inference target of the bone condition. In other words, the chest X-ray image selection unit 18 may not provide the osteoporosis inference unit 20 with the chest X-ray image for which the lung cancer inference unit 14 inferred "abnormal findings present" in the lungs. Therefore, the osteoporosis inference unit 20 may not infer the bone condition of the chest X-ray image for which the result of the inference using the lung cancer screening model 16 was "abnormal findings present" in the lungs. In step S16, the output unit 26 may output "suspected lung cancer" as the result of the inference of the lung condition by the lung cancer screening model 16.

[0029] If the result of inference using the lung cancer screening model 16 is not "abnormal findings" in the lungs, the chest X-ray image selection unit 18 may use the chest X-ray image for which the lung cancer inference unit 14 inferred the lung condition in step S10 as the subject of inference of the bone condition. In other words, the chest X-ray image selection unit 18 may provide the chest X-ray image for which the lung cancer inference unit 14 inferred the "normal" to the osteoporosis inference unit 20. In step S18, the osteoporosis inference unit 20 may infer the bone condition of the chest X-ray image for which the result of inference using the lung cancer screening model 16 was "normal".

[0030] In step S20, the output unit 26 may output “normal” as a result of the inference of the lung condition by the lung cancer inference unit 14. In addition, in step S22, the output unit 26 may output “normal”, “suspected osteoporosis”, or “suspected osteopenia” as a result of the inference of the bone condition by the osteoporosis screening model 22.

[0031] [Modification of the functional configuration of the inference device] The functional configuration of the inference device 1 according to an embodiment of the present disclosure may be the configuration shown in FIG. 3. FIG. 3 is a block diagram showing an example of the functional configuration of the inference device 1 according to this embodiment. The block diagram of FIG. 3 is similar to the block diagram of FIG. 1 except for some parts. The inference device 1 according to this embodiment may include a chest X-ray image receiving unit 10, a chest X-ray image storage unit 12, a lung cancer / osteoporosis inference unit 30, a lung cancer screening model 16, an osteoporosis screening model 22, a clinical information receiving unit 24, and an output unit 26.

[0032] By executing pre-installed programs, the inference device 1 may function as a chest X-ray image receiving unit 10, a chest X-ray image memory unit 12, a lung cancer / osteoporosis inference unit 30, a lung cancer screening model 16, an osteoporosis screening model 22, a clinical information receiving unit 24, and an output unit 26.

[0033] The chest X-ray image receiving unit 10 may receive a chest X-ray image to be inferred. The chest X-ray image storage unit 12 may store the chest X-ray image of the examinee received by the chest X-ray image receiving unit 10. The lung cancer / osteoporosis inference unit 30 may infer the state of the lungs and the state of the bones from the chest X-ray image using the lung cancer screening model 16 and the osteoporosis screening model 22. As an example, the lung cancer screening model 16 may be used first for inference, or the osteoporosis screening model 22 may be used first for inference. The lung cancer screening model 16 has been trained to infer the state of the lungs from the chest X-ray image using the chest X-ray image obtained in the lung cancer screening and the results of the doctor's findings as learning data. The lung cancer / osteoporosis inference unit 30 may infer whether the lungs are "abnormal" or "normal" from the chest X-ray image.

[0034] Furthermore, the lung cancer / osteoporosis inference unit 30 may infer the bone condition from a chest X-ray image using the osteoporosis screening model 22. The osteoporosis screening model 22 may have been trained to infer the bone condition from a chest X-ray image using chest X-ray images obtained in an osteoporosis screening and the results of a doctor's findings or the results of measurement by a DXA method as training data. The lung cancer / osteoporosis inference unit 30 may infer "suspected osteoporosis", "suspected osteopenia", or "normal" from a chest X-ray image.

[0035] The clinical information receiving unit 24 may receive clinical information of the examinee and provide the clinical information of the examinee to the lung cancer / osteoporosis inference unit 30. The output unit 26 may output the result of the inference of the lung condition by the lung cancer inference unit 14 and the result of the inference of the bone condition by the osteoporosis inference unit 20. The output unit 26 may output the result of the inference of the lung condition by the lung cancer / osteoporosis inference unit 30, such as "normal" or "abnormal findings", and the result of the inference of the bone condition, such as "suspected osteoporosis", "suspected osteopenia", or "normal".

[0036] 3, an example in which the lung cancer / osteoporosis inference unit 30 uses a set of the lung cancer screening model 16 and the osteoporosis screening model 22 has been described, but the lung cancer screening model 16 and the osteoporosis screening model 22 may be a single model. By using the lung cancer screening model 16 and the osteoporosis screening model 22 as a single model, the inference device 1 can infer the state of the lungs and the state of the bones with a single model.

[0037] [Variations of the processing procedure of the inference method] The inference device 1 according to an embodiment of the present disclosure may execute the inference method, for example, in the process steps shown in Fig. 4. Fig. 4 is a flowchart showing an example of the inference method according to this embodiment.

[0038] In step S30, the lung cancer / osteoporosis inference unit 30 of the inference device 1 may infer the state of the lungs from the chest X-ray image using the lung cancer screening model 16. In step S32, the lung cancer / osteoporosis inference unit 30 may infer the state of the bones from the chest X-ray image using the osteoporosis screening model 22.

[0039] In step S34, the lung cancer / osteoporosis inference unit 30 may determine whether or not the result of the inference using the lung cancer screening model 16 is "abnormal findings present" in the lungs. If the result of the inference using the lung cancer screening model 16 is "abnormal findings present" in the lungs, the output unit 36 ​​may perform the process of step S36.

[0040] In step S36, the output unit 26 may output “suspected lung cancer” as a result of the inference of the lung condition by the lung cancer screening model 16. In step S38, the output unit 26 may output “normal”, “suspected osteoporosis”, or “suspected osteopenia” as a result of the inference of the bone condition by the osteoporosis screening model 22.

[0041] Furthermore, in step S34, if the result of the inference using the lung cancer screening model 16 is "normal", the output unit 26 may perform the process of step S40. In step S40, the output unit 26 may output "normal" as the result of the inference of the lung condition by the lung cancer screening model 16. In step S42, the output unit 26 may output "normal", "suspected osteoporosis", or "suspected osteopenia" as the result of the inference of the bone condition by the osteoporosis screening model 22.

[0042] [Functional configuration of the learning device] A functional configuration of a learning device 2 according to an embodiment of the present disclosure will be described. Fig. 5 is a block diagram showing an example of the functional configuration of the learning device 2 according to this embodiment. The learning device 2 in Fig. 5 performs unsupervised pre-learning using a large-scale chest X-ray image dataset to improve the inference accuracy of the osteoporosis screening model 22. For the large-scale chest X-ray image dataset, for example, MIMIC-CXR or CheXpert, which are publicly available free of charge, may be used.

[0043] The learning device 2 in FIG. 5 may include an osteoporosis screening model 22, a chest X-ray image receiving unit 50, a chest X-ray image storage unit 52, a self-supervised learning unit 54, a supervised learning unit 58, and an osteoporosis screening model transmitting unit 60.

[0044] By executing a pre-installed program, the learning device 2 may function as an osteoporosis screening model 22, a chest X-ray image receiving unit 50, a chest X-ray image storage unit 52, a self-supervised learning unit 54, a supervised learning unit 58, and an osteoporosis screening model transmitting unit 60. The learning device 2 is an information processing device such as a personal computer, a tablet terminal, a smartphone, or a server.

[0045] The chest X-ray image receiving unit 50 may receive a large-scale chest X-ray image data set without a correct decision label. The chest X-ray image storage unit 52 may store the large-scale chest X-ray image data set received by the chest X-ray image receiving unit 50.

[0046] The self-supervised learning unit 54 may perform pre-learning of the osteoporosis screening model 22 by self-supervised learning, utilizing a large-scale chest X-ray image data set stored in the chest X-ray image storage unit 52. Self-supervised learning is a learning method that performs pre-learning for solving an alternative task in which pseudo decision labels, called a pre-text task, are automatically generated, utilizing a large-scale data set without decision labels.

[0047] The supervised learning unit 58 may fine-tune the osteoporosis screening model 22 pre-trained by the self-supervised learning unit 54 in order to solve a target task (downstream task). The osteoporosis screening model transmission unit 60 may transmit the osteoporosis screening model 22 fine-tuned by the supervised learning unit 58 to the inference device 1.

[0048] According to the learning device 2 in FIG. 5, pseudo decision labels can be automatically generated for a large-scale chest X-ray image dataset without decision labels by self-supervised learning, thereby reducing the effort required for assigning decision labels.

[0049] [Modification of the functional configuration of the learning device] The functional configuration of the learning device 2 according to an embodiment of the present disclosure may be the configuration shown in FIG. 6. FIG. 6 is a block diagram showing an example of the functional configuration of the learning device 2 according to the present embodiment. The learning device 2 in FIG. 6 pre-learns the osteoporosis screening model 22 with prior knowledge used in interpreting a chest X-ray image, thereby improving the inference accuracy of the osteoporosis screening model 22. The prior knowledge used in interpreting a chest X-ray image is, for example, knowledge for first dividing the lung field and the chest region when a radiologist interprets a chest X-ray image. The learning device 2 in FIG. 6 detects the boundary (division of the region) between the lung field and the chest in the chest X-ray image in advance, and utilizes the division of the region as prior knowledge for learning and inference of the osteoporosis screening model 22, thereby improving the inference accuracy of the osteoporosis screening model 22.

[0050] The learning device 2 according to this embodiment may include an osteoporosis screening model 22, a chest X-ray image receiving unit 50, a chest X-ray image storage unit 52, an osteoporosis screening model transmitting unit 60, a pre-learning unit 70, a region segment receiving unit 72, and a supervised learning unit 74. The learning device 2 may function as the osteoporosis screening model 22, the chest X-ray image receiving unit 50, the chest X-ray image storage unit 52, the osteoporosis screening model transmitting unit 60, the pre-learning unit 70, the region segment receiving unit 72, and the supervised learning unit 74 by executing a program installed in advance.

[0051] The chest X-ray image receiving unit 50 may receive a large-scale chest X-ray image data set without a correct decision label. The chest X-ray image storage unit 52 may store the large-scale chest X-ray image data set received by the chest X-ray image receiving unit 50. The region division receiving unit 72 may receive information on the division of regions of the chest X-ray image (e.g., boundary coordinates of the lung field and the abdomen).

[0052] The pre-learning unit 70 may pre-learn the division of the region of the chest X-ray image. The pre-learning performed by the pre-learning unit 70 is, for example, a learning method in which a new layer is added to the neural network and the parameters of the added layer are adjusted according to the division of the region of the chest X-ray image.

[0053] The supervised learning unit 78 may perform supervised learning on the osteoporosis screening model 22 that has been pre-trained by the pre-training unit 70. The osteoporosis screening model sending unit 60 may send the osteoporosis screening model 22 that has been pre-trained by the supervised learning unit 74 to the inference device 1.

[0054] In addition, in order to assist interpretation, various image processing such as contrast adjustment or noise removal filtering is performed on chest X-ray image imaging devices. Due to differences in image processing performed by chest X-ray image imaging devices, the generalization performance of the osteoporosis screening model 22 may decrease.

[0055] Therefore, the learning device 2 in Figures 5 and 6 may improve the robustness and generalizability of the osteoporosis screening model 22 by performing augmentation of teacher data taking into account image processing performed by imaging equipment for chest X-ray images to assist in interpretation.

[0056] [summary] Osteoporosis screening is generally performed using DXA or ultrasound. DXA can measure bone density and is also used for definitive diagnosis, but the equipment is expensive and requires securing and managing a place to install it. Ultrasound measurements are relatively inexpensive, but there are differences between devices and technicians, and the consistency rate with DXA measurements is low.

[0057] The inference device 1 according to one embodiment of the present disclosure can examine the condition of the lungs and the condition of the bones with a high degree of accuracy using the same chest X-ray image, making it possible to perform osteoporosis screening through regular lung cancer screening or osteoporosis screening by a family doctor, which is expected to improve the rate of people receiving osteoporosis screening.

[0058] Furthermore, the inference device 1 according to an embodiment of the present disclosure can infer the state of the lungs and the state of the bones with high accuracy using one model or a set of models, and is therefore highly convenient for medical institutions and the like that introduce it.

[0059] The functions of the inference device 1 according to an embodiment of the present disclosure may be provided as a cloud service to medical examination facilities, clinics, and the like.

[0060] According to an embodiment of the present disclosure, it is possible to provide an inference device 1 that improves the accuracy of inferring a bone condition using a chest X-ray image.

[0061] [Hardware configuration of inference device and learning device] A part or all of each device (the inference device 1 and the learning device 2) in the above-mentioned embodiment may be configured with hardware, or may be configured with information processing of software (programs) executed by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like. In the case of being configured with information processing of software, software for realizing at least a part of the functions of each device in the above-mentioned embodiment may be stored in a non-transient storage medium (non-transient computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or a USB (Universal Serial Bus) memory, and the software information processing may be executed by reading the software into a computer. The software may also be downloaded via a communication network. Furthermore, the information processing by the software may be executed by hardware by implementing all or a part of the processing of the software in a circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0062] The storage medium that stores the software may be a removable medium such as an optical disk, or a fixed storage medium such as a hard disk, memory, etc. The storage medium may be provided inside the computer (main storage device, auxiliary storage device, etc.) or outside the computer.

[0063] 7 is a block diagram showing an example of a hardware configuration of each device (the inference device 1 and the learning device 2) in the above-mentioned embodiment. Each device may be realized as a computer 7 including, for example, a processor 101, a main storage device 102 (memory), an auxiliary storage device 103 (memory), a network interface 104, and a device interface 105, which are connected via a bus 106.

[0064] Although the computer 7 in FIG. 7 includes one of each component, it may include a plurality of the same components. Although one computer 7 is shown in FIG. 7, the software may be installed in a plurality of computers, and each of the plurality of computers may execute the same or different parts of the software. In this case, it may be in the form of distributed computing in which each computer executes the process by communicating via the network interface 104 or the like. That is, each device (the inference device 1 and the learning device 2) in the above-mentioned embodiment may be configured as a system in which one or more computers execute instructions stored in one or more storage devices to realize the function. Also, it may be configured such that information transmitted from a terminal is processed by one or more computers provided on the cloud, and the processing results are transmitted to the terminal.

[0065] Various calculations of each device (the inference device 1 and the learning device 2) in the above-mentioned embodiments may be executed in parallel using one or more processors, or using multiple computers via a network. Moreover, various calculations may be distributed to multiple computing cores in a processor and executed in parallel. Moreover, a part or all of the processes, means, etc. of the present disclosure may be realized by at least one of a processor and a storage device provided on a cloud capable of communicating with a computer 7 via a network. Thus, each device in the above-mentioned embodiments may be in the form of parallel computing using one or more computers.

[0066] The processor 101 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that at least controls a computer or performs calculations. The processor 101 may be a general-purpose processor, a dedicated processing circuit designed to perform a specific calculation, or a semiconductor device that includes both a general-purpose processor and a dedicated processing circuit. The processor 101 may also include an optical circuit, or may include a calculation function based on quantum computing.

[0067] The processor 101 may perform arithmetic processing based on data or software input from each device or the like configured inside the computer 7, and may output arithmetic results or control signals to each device or the like. The processor 101 may control each component constituting the computer 7 by executing the OS (Operating System) of the computer 7, applications, or the like.

[0068] Each device (the inference device 1 and the learning device 2) in the above-described embodiment may be realized by one or more processors 101. Here, the processor 101 may refer to one or more electronic circuits arranged on one chip, or to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other via wires or wirelessly.

[0069] The main memory device 102 may store instructions executed by the processor 101 and various data, and information stored in the main memory device 102 may be read by the processor 101. The auxiliary memory device 103 is a memory device other than the main memory device 102. Note that these memory devices refer to any electronic components capable of storing electronic information, and may be semiconductor memories. The semiconductor memories may be either volatile or non-volatile memories. The memory devices for saving various data in each device (the inference device 1 and the learning device 2) in the above-mentioned embodiments may be realized by the main memory device 102 or the auxiliary memory device 103, or may be realized by an internal memory built into the processor 101. For example, each memory unit in the above-mentioned embodiments may be realized by the main memory device 102 or the auxiliary memory device 103.

[0070] When each device (inference device 1 and learning device 2) in the above-mentioned embodiments is composed of at least one storage device (memory) and at least one processor connected (coupled) to this at least one storage device, at least one processor may be connected to one storage device. Also, at least one storage device may be connected to one processor. Also, a configuration may be included in which at least one processor of the multiple processors is connected to at least one storage device of the multiple storage devices. Also, this configuration may be realized by storage devices and processors included in multiple computers. Furthermore, a configuration may be included in which a storage device is integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache).

[0071] The network interface 104 is an interface for connecting to the communication network 8 wirelessly or by wire. The network interface 104 may be an appropriate interface, such as one conforming to an existing communication standard. The network interface 104 may exchange information with an external device 9A connected via the communication network 8. The communication network 8 may be any one of a wide area network (WAN), a local area network (LAN), a personal area network (PAN), etc., or a combination thereof, as long as information is exchanged between the computer 7 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or Near Field Communication (NFC).

[0072] The device interface 105 is an interface such as a USB that directly connects to the external device 9B.

[0073] The external device 9A is a device connected to the computer 7 via a network. The external device 9B is a device connected directly to the computer 7.

[0074] The external device 9A or the external device 9B may be, for example, an input device. The input device is, for example, a device such as a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, a touch panel, or the like, and provides acquired information to the computer 7. Alternatively, the input device may be a device equipped with an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0075] Moreover, the external device 9A or the external device B may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or a speaker that outputs sound or the like. Alternatively, the output device may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0076] Furthermore, the external device 9A or the external device 9B may be a storage device (memory). For example, the external device 9A may be a network storage or the like, and the external device 9B may be a storage device such as an HDD.

[0077] Furthermore, the external device 9A or the external device 9B may be a device having some of the functions of the components of each device (information processing device 100) in the above-described embodiment. That is, the computer 7 may transmit some or all of the processing results to the external device 9A or the external device 9B, or may receive some or all of the processing results from the external device 9A or the external device 9B.

[0078] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0079] In this specification (including claims), when expressions such as "using data as input / based on / according to / in response to data" (including similar expressions) are used, unless otherwise specified, this includes the use of data itself or data that has been processed in some way (e.g., data with noise added, normalized, feature values ​​extracted from data, intermediate representations of data, etc.). In addition, when it is stated that a result is obtained "using data as input / based on / according to / in response to data" (including similar expressions), unless otherwise specified, this includes the use of the result based only on the data, or the use of the result influenced by other data, factors, conditions, and / or states other than the data. In addition, when it is stated that "data is output" (including similar expressions), unless otherwise specified, this includes the use of data itself as output, or data that has been processed in some way (e.g., data with noise added, normalized, feature values ​​extracted from data, intermediate representations of various data, etc.) as output.

[0080] In this specification (including the claims), the terms "connected" and "coupled" are intended as open-ended terms including any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, physically connection / coupling, etc. The terms should be interpreted appropriately depending on the context in which the terms are used, but any connection / coupling form that is not intentionally or naturally excluded should be interpreted as being included in the terms without any restriction.

[0081] In this specification (including the claims), when the expression "A configured to B" is used, it may include that the physical structure of element A has a configuration capable of performing operation B, and that the permanent or temporary setting / configuration of element A is configured / set to actually perform operation B. For example, when element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, when element A is a dedicated processor, dedicated arithmetic circuit, etc., it is sufficient that the circuit structure, etc. of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0082] In this specification (including the claims), when terms implying containing or possessing (e.g., "comprising / including," "having," etc.) are used, they are intended to be open-ended terms that include cases in which the term contains or possesses things other than the object indicated by the object of the term. When the object of such terms implying containing or possessing does not specify a quantity or suggests a singular number (such as an expression using the article "a" or "an"), the expression should be construed as not being limited to a specific number.

[0083] In this specification (including the claims), even if expressions such as "one or more" and "at least one" are used in some places and expressions that do not specify a quantity or suggest a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or suggest a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0084] In this specification, when a particular advantage / result is described as being obtained from a particular configuration of an embodiment, it should be understood that the same effect can also be obtained from one or more other embodiments having the same configuration, unless there is a reason to the contrary. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or states, and that the effect is not necessarily obtained by the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or states are satisfied, and the effect is not necessarily obtained in the invention according to the claim that specifies the configuration or a similar configuration.

[0085] In this specification (including the claims), when a term such as "maximize" is used, it includes finding a global maximum, finding an approximation of a global maximum, finding a local maximum, and finding an approximation of a local maximum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these maxima probabilistically or heuristically. Similarly, when a term such as "minimize" is used, it includes finding a global minimum, finding an approximation of a global minimum, finding a local minimum, and finding an approximation of a local minimum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these minima probabilistically or heuristically. Similarly, when a term such as "optimize" is used, it includes finding a global optimum, finding an approximation of a global optimum, finding a local optimum, and finding an approximation of a local optimum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these optimum values ​​probabilistically or heuristically.

[0086] In this specification (including claims), when a plurality of pieces of hardware perform a predetermined process, each piece of hardware may cooperate to perform the predetermined process, or a portion of the hardware may perform all of the predetermined process. Also, a portion of the hardware may perform a portion of the predetermined process, and another piece of hardware may perform the remainder of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" (including similar expressions) is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. The hardware may include an electronic circuit, a device including an electronic circuit, and the like.

[0087] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices may store only a portion of the data, or may store the entire data. Also, a configuration in which only some of the multiple storage devices store data may be included.

[0088] In this specification (including the claims), terms such as "first", "second", etc. are used merely as a way of distinguishing between two or more elements, and are not necessarily intended to impose technical meanings such as time aspect, space aspect, sequence, quantity, etc. on the subject matter. Thus, for example, a reference to a first element and a second element does not necessarily mean that only two elements may be employed therein, that the first element must precede the second element, that the first element must be present for the second element to be present, etc.

[0089] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the above-mentioned individual embodiments. Various additions, modifications, replacements, partial deletions, etc. are possible within the scope of the conceptual idea and intent of the present invention derived from the contents defined in the claims and their equivalents. For example, in the above-mentioned embodiments, when numerical values ​​or formulas are used for explanation, these are shown for illustrative purposes and do not limit the scope of the present disclosure. In addition, the order of each operation shown in the embodiment is also illustrative and does not limit the scope of the present disclosure. [Explanation of symbols]

[0090] 1 Reasoning device 2 Learning device 10 Chest X-ray image receiving unit 12 Chest X-ray image storage unit 14 Lung Cancer Inference Department 16 Lung cancer screening model 18 Chest X-ray Image Selection Section 20 Osteoporosis Inference Department 22 Osteoporosis screening model 24 Clinical Information Receiving Department 26 Output section

Claims

1. At least one memory; At least one processor; The at least one processor inferring the state of the internal organs from the x-ray images using the first model; inferring bone condition from said X-ray images using a second model; and An inference device that executes the above.

2. The at least one processor The X-ray images for which the inference using the first model is abnormal are excluded from the X-ray images for which the bone condition is inferred using the second model. The inference device according to claim 1.

3. The first model is a lung cancer screening model that has been trained using X-ray images obtained in a lung cancer screening and findings by a doctor as training data; The second model is an osteoporosis screening model that has been trained using X-ray images obtained in an osteoporosis screening and the results of a doctor's findings as training data.

3. The inference device according to claim 1 or 2.

4. The at least one processor Using the second model, inferring whether osteoporosis is suspected, osteopenia is suspected, or normal from the X-ray image of the inference target.

4. The inference device according to claim 3.

5. The second model is an osteoporosis screening model that has been trained using X-ray images obtained in an osteoporosis screening, clinical information of a patient, and a doctor's findings as training data; The at least one processor inferring a bone condition using the second model from the X-ray image of the inference target and the medical information of the patient for which the inference result using the first model was normal; 4. The inference device according to claim 3.

6. The training data is generated by performing self-supervised learning on a dataset of X-ray images.

4. The inference device according to claim 3.

7. The learning data is created by performing image processing performed by an X-ray image capturing device to assist in interpretation of the X-ray image.

4. The inference device according to claim 3.

8. The second model is an osteoporosis screening model that has been pre-trained to divide the lung field and chest area of ​​an X-ray image.

4. The inference device according to claim 3.

9. An inference method in an inference device having at least one memory and at least one processor, comprising: The at least one processor inferring the state of the internal organs from the x-ray images using the first model; inferring bone condition from said X-ray images using a second model; and An inference method to perform.

Images