Generation method, training method, generation device, training system, estimation system, control program, and recording medium

Abstract

The present invention facilitates preparation of machine-learning data of a neural network. A generation method according to the present disclosure comprises: a first generation step for generating one or more second images as two-dimensional images respectively corresponding to one or more pieces of first data that indicate the three-dimensional shape of an object of a second subject; and a second generation step for generating machine-learning data including one or more second images as explanatory variables and including the pieces of first data corresponding to the second images as objective variables. The machine-learning data is used for training an estimation model for estimating the three-dimensional shape of an object of a first subject from first images that are one or more two-dimensional images obtained by imaging the object of the first subject.

Description

Generation method, learning method, generation device, learning system, estimation system, control program, and recording medium

[0001] The present disclosure relates to a method and device for generating machine learning data, a method and system for training an estimation model using machine learning data, an estimation system using a trained estimation model, etc.

[0002] Medical images include two-dimensional and three-dimensional medical images. For example, when extracting information about tissues and organs to be treated from medical images, three-dimensional medical images provide more useful information than two-dimensional medical images. However, the devices used to capture three-dimensional medical images are rare, and opportunities to obtain three-dimensional medical images are fewer than those for two-dimensional medical images. Furthermore, the cost of obtaining three-dimensional medical images tends to be higher than that of two-dimensional medical images. When the treatment target is bone, plain X-ray images are used as two-dimensional medical images, and computed tomography (CT) images are used as three-dimensional medical images. Considering the radiation exposure of the subject, plain X-ray imaging, which results in lower radiation exposure, is more commonly used.

[0003] Patent Literature 1 discloses a technique for generating a three-dimensional medical image from a two-dimensional medical image of a femur. Non-Patent Literature 1 discloses a technique for generating a three-dimensional medical image from a two-dimensional medical image using a trained neural network having an encoder and a decoder.

[0004] WO2011 / 098895

[0005] <1> A generation method according to one aspect of the present disclosure includes a first generation step of generating at least one or more second images as two-dimensional images corresponding to each of at least one or more first data indicating a three-dimensional shape of an object of a second subject, and a second generation step of generating machine learning data that includes at least one or more of the second images as explanatory variables and the first data corresponding to the second images as a target variable, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images that depict the object of the first subject.

[0006] <2> A generation method according to one aspect of the present disclosure includes a third generation step of generating at least one or more third data by modifying second data indicating a three-dimensional shape of an object of a third subject, the second data being generated based on a third image, which is a three-dimensional image capturing the object of the third subject; a fourth generation step of generating at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; and a fifth generation step of generating machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image, which is at least one or more two-dimensional image capturing the object of the first subject.

[0007] <3> A learning method according to one aspect of the present disclosure includes a learning step of generating the estimation model by machine learning using the machine learning data generated by the generation method described in <1> or <2>.

[0008] <4> A learning method according to one aspect of the present disclosure includes a learning step of generating the estimation model by machine learning using the machine learning data generated by the generation method described in <2>, and further includes an adjustment step of fine-tuning the estimation model using second data indicating the three-dimensional shape of the object of the third subject, generated based on a third image that is a three-dimensional image of the object of the third subject, and (1) at least one or more fourth images generated as two-dimensional images corresponding to the second data, or (2) at least one or more fifth images that are two-dimensional images of the object of the third subject.

[0009] <5> A generation device according to one aspect of the present disclosure includes a first generation unit that generates at least one or more second images as two-dimensional images corresponding to at least one or more first data indicating a three-dimensional shape of an object of a second subject, and a second generation unit that generates machine learning data that includes at least one or more second images as explanatory variables and the first data corresponding to the second images as a target variable, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images that depict the object of the first subject.

[0010] <6> A generation device according to one aspect of the present disclosure includes a third generation unit that generates at least one or more third data by modifying second data that indicates a three-dimensional shape of an object of a third subject, the second data being generated based on a third image that is a three-dimensional image that captures the object of the third subject; a fourth generation unit that generates at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; and a fifth generation unit that generates machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as a target variable, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image that is at least one or more two-dimensional image that captures the object of the first subject.

[0011] <7> A learning system according to one aspect of the present disclosure includes a first generation unit that generates at least one or more second images as two-dimensional images corresponding to each of at least one or more first data indicating a three-dimensional shape of an object of a second subject; a second generation unit that generates machine learning data that includes at least one or more of the second images as explanatory variables and the first data corresponding to the second images as a target variable; and a learning unit that performs machine learning using the machine learning data and generates an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images that depict the object of the first subject.

[0012] <8> A learning system according to one aspect of the present disclosure includes a third generation unit that generates at least one or more third data by modifying second data indicating a three-dimensional shape of an object of a third subject, the second data being generated based on a third image that is a three-dimensional image capturing the object of the third subject; a fourth generation unit that generates at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; a fifth generation unit that generates machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables; and a learning unit that performs machine learning using the machine learning data to generate an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image that is at least one or more two-dimensional image capturing the object of the first subject.

[0013] <9> An estimation system according to one aspect of the present disclosure includes an estimation unit that estimates the three-dimensional shape of the object of the first subject from the first image using a trained estimation model generated by the learning method described in <3>, and a display control unit that displays the estimation result by the estimation unit on a display unit.

[0014] The generating device according to each aspect of the present disclosure may be realized by a computer. In this case, the control program of the generating device, which causes the computer to operate as each part (software element) of the generating device, and the computer-readable non-transitory recording medium on which the control program is recorded, also fall within the scope of the present disclosure.

[0015] The learning system and estimation system according to each aspect of the present disclosure may be realized by a computer. In this case, the control programs for the learning system and the estimation system, which are realized by a computer by causing the computer to operate as each part (software element) of each of the learning system and the estimation system, and the computer-readable non-transitory recording medium on which they are recorded, also fall within the scope of the present disclosure.

[0016] FIG. 1 is a functional block diagram showing an example of a schematic configuration of an estimation system according to an embodiment of the present disclosure. FIG. 2 is a functional block diagram showing an example of a schematic configuration of a learning system according to an embodiment of the present disclosure. FIG. 3 is a flowchart showing an example of a processing flow performed by a generation device. FIG. 4 is a functional block diagram showing an example of a schematic configuration of a learning system according to another embodiment of the present disclosure. FIG. 5 is a flowchart showing an example of a processing flow performed by a generation device. FIG. 6 is a diagram showing an example of processing performed by a generation device and a learning device when an object is a bone.

[0017] Although the technology disclosed in Patent Document 1 can generate the external shape of a femur with a high degree of reproducibility, there is room for improvement in the reproducibility of the internal structure of the femur and in the reproducibility when targeting bones with shapes more complex than the femur (e.g., vertebral bodies). Furthermore, the technology disclosed in Non-Patent Document 1 requires the preparation of a large number of medical images for machine learning of the neural network. In particular, it is not easy to prepare a large number of three-dimensional medical images.

[0018] According to one aspect of the present disclosure, it is possible to facilitate the preparation of data for machine learning in a neural network.

[0019] First Embodiment Hereinafter, an embodiment of the present disclosure will be described in detail using an estimation system 100 and a learning system 200 as examples.

[0020] (Estimation System 100) First, the estimation system 100 will be described with reference to Fig. 1. Fig. 1 is a functional block diagram showing an example of a schematic configuration of the estimation system 100. The estimation system 100 is a system that estimates a three-dimensional shape of an object of a first subject from a first image 711, which is a two-dimensional image showing the object of the first subject, by using an estimation model 511, which will be described later.

[0021] The estimation system 100 includes an estimation device 5 and an information processing device 7. As shown in Fig. 1 , the estimation system 100 may further include a generation device 1, which will be described later. The estimation device 5 may be a cloud-based device, or may be an on-premise device installed in a medical facility or a company that provides analysis services.

[0022] 1 illustrates one information processing device 7 capable of communicating with the estimation device 5, but the present invention is not limited to this configuration. For example, in the estimation system 100, the estimation device 5 may be capable of communicating with multiple information processing devices 7. Furthermore, the estimation system 100 may be configured to include multiple estimation devices 5.

[0023] The generating device 1, the estimating device 5, and the information processing device 7 can communicate with each other via a communication network 9. The communication network 9 may be the Internet, an intranet, a blockchain network, a wireless local area network (LAN), a wide area network (WAN), or the like. For example, if the generating device 1 and the estimating device 5 are cloud-based devices, the communication network 9 may be the Internet, a blockchain network, or the like. If the generating device 1 and the estimating device 5 are on-premise devices, the communication network 9 may be a wireless LAN provided within a medical facility or company.

[0024] In the present disclosure, the first subject is described as a human, but the subject is not limited to a human. The subject may be a non-human mammal, such as an equine, feline, canine, bovine, or porcine animal, or may be a non-mammalian animal (e.g., a bird, reptile, amphibian, or fish).

[0025] In the present disclosure, the object may include at least one of bone, organ, and muscle. The first image 711 may be a medical image of the first subject. The medical image may be a still image or a video image. The medical image may include a phantom or may not include a phantom. If the object is bone, the first image 711 may be at least one of a plain X-ray image of the bone of the first subject, a two-dimensional ultrasound image, an image obtained by DXA (Dual Energy X-ray Absorptiometry), and an image obtained by DES (Dual Energy Subtraction). If the object is organ and / or muscle, the first image 711 may be at least one of a plain X-ray image of the organ and / or muscle of the first subject, a two-dimensional ultrasound image, an image obtained by DXA, and an image obtained by DES. The first image 711 may be at least one of a front image showing the object from the front (e.g., an image obtained by irradiating the object with X-rays in the front-to-back direction) and a side image showing the object from the side (e.g., an image obtained by irradiating the object with X-rays in the left-to-right direction).

[0026] <Information Processing Device 7> The information processing device 7 may be a computer provided in a medical facility or company that uses the estimation system 100. Alternatively, the information processing device 7 may be a computer used by a medical professional, such as a doctor, that uses the estimation system 100.

[0027] The information processing device 7 includes a control unit 70, a storage unit 71, a communication unit 72, and a display unit 73. In Fig. 1, the information processing device 7 includes the display unit 73 as an example, but is not limited to this. For example, instead of including the display unit 73, an external display device (not shown) may be used.

[0028] The control unit 70 may be, for example, a CPU (Central Processing Unit) and performs overall control of the information processing device 7. The control unit 70 reads a control program, which is software stored in the storage unit 71, expands it in a memory such as a RAM (Random Access Memory), and controls each component of the information processing device 7. The control unit 70 includes an acquisition unit 701 that acquires the estimation result 512 from the estimation device 5, and a display control unit 702 that displays the estimation result 512 on the display unit 73.

[0029] For example, if the object is a bone, the estimation result 512 is highly accurate information about the three-dimensional shape of the bone of the first subject. Therefore, the control unit 70 may further include a first estimation unit (not shown) that estimates a value related to the bone of the first subject from the estimation result 512 about the three-dimensional shape of the bone of the first subject using a first estimation model. Here, the first estimation model may be machine-trained using training data that includes the estimation result 512 about the three-dimensional shape of the bone of the fourth subject as an explanatory variable and a value related to the bone of the fourth subject as a response variable.

[0030] The bone-related value estimated using the first estimation model may include any of bone mineral density, bone mass, and bone quality information. Bone mineral density is a value related to the density of bone. Bone mineral density is bone mineral density per unit area [g / cm ２ ], bone mineral density per unit volume [g / cm ３ ], YAM [%], T-score, and Z-score. YAM [%] is an abbreviation for "Young Adult Mean" and may be called the young adult mean percent. For example, bone mineral density can be expressed as bone mineral density per unit area [g / cm ２ The bone mineral density may be a value expressed in terms of YAMYAM [%] and YAMYAM [%]. The bone mineral density may be an index determined by a guideline or may be an original index. The bone mineral density may be a value described in the osteoporosis guidelines, for example, the "2015 Edition of the Prevention and Treatment Guidelines of the Japan Osteoporosis Society."

[0031] Bone mass is the sum of bone mineral and bone matrix protein. Bone mass is an index related to bone density, and is the amount of bone tissue in the skeleton. Bone mass may be information measured using a bone density measuring device employing the DXA method, or information obtained by estimating bone density from X-ray images using a second estimation model. Furthermore, bone quality information may include, but is not limited to, at least one of bone formation markers, bone resorption markers, bone quality markers (e.g., vitamin K levels), cortical bone thickness, trabecular density, trabecular orientation, and trabecular bone structure index (trabecular bone score).

[0032] For example, the control unit 70 may calculate the volumetric bone density (bone mass per unit volume) of the first subject's bones based on the acquired estimation result 512. While it is possible to estimate the volumetric bone density of the first subject from the first image 711, this estimation method may not be highly interpretable. This is because the correspondence between the estimated volumetric bone density value and the three-dimensional state of the first subject's bones is unclear. In contrast, the method of calculating the volumetric bone density value based on the estimation result 512 estimated from the first image 711 is highly interpretable because the correspondence between the estimated volumetric bone density value and the three-dimensional state of the first subject's bones is clear. Furthermore, the control unit 70 may estimate the bone condition of the first subject (e.g., the presence or absence of a fracture) based on the acquired estimation result 512. This estimation method is highly interpretable because the correspondence between the estimated bone condition and the three-dimensional state of the first subject's bones is clear.

[0033] The storage unit 71 is a storage device that stores various control programs and various data, and may store a first image 711. For the sake of simplicity, the control programs are not shown in the storage unit 71 illustrated in FIG.

[0034] The communication unit 72 transmits and receives various data to and from the estimation device 5 and the generation device 1. For example, the information processing device 7 transmits first images 711, which are one or more two-dimensional images showing an object of the first subject, to the generation device 1 via the communication unit 72. Furthermore, the information processing device 7 receives, for example, an estimation result 512 indicating the three-dimensional shape of the object of the first subject from the estimation device 5 via the communication unit 72.

[0035] The display unit 73 displays various data. The display unit 73 may display the first image 711 and the estimation result 512 side by side. For example, a medical professional using the information processing device 7 can examine the condition of the object of the first subject based on the estimation result 512 received from the estimation device 5 and the first image 711 displayed on the display unit 73.

[0036] The estimation device 5 estimates the three-dimensional shape of the object of the first subject from at least one first image 711 that is a two-dimensional image of the object of the first subject. The estimation device 5 includes a control unit 50, a storage unit 51, and a communication unit 52.

[0037] The control unit 50 may be, for example, a CPU. The control unit 50 reads a control program, which is software stored in the storage unit 51, expands it in a memory such as a RAM, and controls each component included in the estimation device 5. The control unit 50 includes an acquisition unit 501 and an estimation unit 502.

[0038] The acquisition unit 501 acquires a trained estimation model 511 generated by the learning device 3 (described later). The acquisition unit 501 may store the acquired trained estimation model 511 in the storage unit 51.

[0039] The estimation unit 502 estimates the three-dimensional shape of the object of the first subject from the first image 711 using a trained estimation model 511. A transformer-based model, which is an encoder-decoder based self-attention neural network, may be used as the estimation model 511. However, models that can be used as the estimation model 511 are not limited to the transformer-based model. For example, models based on the following neural network architectures may be used as the estimation model 511: Feed-forward neural network (FNN); Convolutional neural network (CNN); Recurrent neural network (RNN). RNNs include long short-term memory (LSTM)-based RNNs and gated recurrent unit (GRU)-based RNNs. - Generative adversarial network (GAN).

[0040] When the object is a lumbar bone and the first image 711 is a plain X-ray image, the estimation model 511 may be a model that estimates the three-dimensional shape of the lumbar vertebrae only from a frontal X-ray image of the lumbar vertebrae. Alternatively, the estimation model 511 may be, for example, a model that estimates the three-dimensional shape of the lumbar vertebrae only from a lateral X-ray image of the lumbar vertebrae. The estimation model 511 may be, for example, a model that estimates the three-dimensional shape of the lumbar vertebrae from a frontal X-ray image of the lumbar vertebrae and a lateral X-ray image of the lumbar vertebrae.

[0041] When the object is a chest bone and the first image 711 is a plain X-ray image, the estimation model 511 may be a model that estimates the three-dimensional shape of the thoracic vertebrae only from a frontal X-ray image of the thoracic vertebrae. Alternatively, the estimation model 511 may be, for example, a model that estimates the three-dimensional shape of the thoracic vertebrae only from a lateral X-ray image of the thoracic vertebrae. The estimation model 511 may be, for example, a model that estimates the three-dimensional shape of the thoracic vertebrae from a frontal X-ray image of the thoracic vertebrae and a lateral X-ray image of the thoracic vertebrae.

[0042] The estimation results 512 are the same type of results related to the objective variables of the machine learning data 311 generated by the generating device 1 (for example, the second generating unit 102) described below.

[0043] The storage unit 51 is a storage device that stores various control programs and various data, and may store a trained estimation model 511 and an estimation result 512 estimated by the control unit 50. For the sake of simplicity, the control program is not shown in the storage unit 51 shown in FIG.

[0044] The communication unit 52 transmits and receives various data to and from the generation device 1 and the information processing device 7. For example, the estimation device 5 receives a first image 711 from the information processing device 7 via the communication unit 52. The estimation device 5 also transmits an estimation result 512 estimated from the first image 711 received from the information processing device 7 to the information processing device 7, which is the sender of the first image 711.

[0045] (Learning System 200) Next, the learning system 200 will be described using Fig. 2. Fig. 2 is a functional block diagram showing an example of a schematic configuration of the learning system 200. The learning system 200 is a system in which a generation device 1 generates machine learning data 311, and a learning device 3 performs machine learning using the machine learning data 311 generated by the generation device 1 to generate a trained estimation model 511.

[0046] The learning system 200 includes a generation device 1 and a learning device 3. As shown in Fig. 2, the learning system 200 may further include an estimation device 5. Both the generation device 1 and the learning device 3 may be cloud-based devices, or may be on-premise devices installed in a medical facility or a company that provides analysis services.

[0047] 2 shows one learning device 3 that can communicate with the generation device 1, but the present invention is not limited to this configuration. For example, in the learning system 200, the generation device 1 may be able to communicate with multiple learning devices 3. Furthermore, the learning system 200 may be configured to include multiple generation devices 1.

[0048] The communication network 9 may be the Internet, an intranet, a blockchain network, a wireless LAN (Local Area Network), a WAN (Wide Area Network), etc.

[0049] <Generation Device 1> The generation device 1 is a device that generates machine learning data 311 used for machine learning of an estimation model 511. The generation device 1 includes a control unit 10, a storage unit 11, and a communication unit 12.

[0050] The control unit 10 may be, for example, a CPU. The control unit 10 reads a control program, which is software stored in the storage unit 11, expands it in a memory such as a RAM, and controls each component included in the generation device 1. The control unit 10 includes a first generation unit 101 and a second generation unit 102.

[0051] The first generating unit 101 generates one or more second images as two-dimensional images corresponding to the first data 111. Here, the first data 111 may be data indicating the three-dimensional shape of each of the objects of one or more second subjects. The second subject may be of the same type as the first subject, or may be of a different type. For example, if the first subject is a human, the second subject may be the same person as the first subject, or may be a different person.

[0052] The first data 111 may be data indicating the three-dimensional shape of an object including a target. The first data 111 may be data indicating the three-dimensional shape of an object including a target and one or more structures having properties different from those of the target. For example, the first data 111 may include data indicating the three-dimensional shape of the target and data indicating the three-dimensional shape of a part of a second subject different from the target. For example, if the target is a bone of the second subject, the first data 111 may include data indicating the three-dimensional shape of soft tissue of the second subject, which is a part of the second subject different from the target. For example, if the target is a bone of the second subject, the first data 111 may include data indicating the three-dimensional shape of soft tissue and organs of the second subject, which are a part of the second subject different from the target. For example, if the target is a lumbar bone of the second subject, the first data 111 may include data indicating the three-dimensional shape of thoracic bone of the second subject, which is a part of the second subject different from the target.

[0053] The first data 111 may be, for example, data indicating the three-dimensional shape of each of the one or more second subject objects estimated based on a group of images obtained by capturing each of the one or more second subject objects from multiple directions. Alternatively, the first data 111 may be three-dimensional data obtained by scanning the three-dimensional shape of each of the one or more second subject objects. For example, if the object is a bone, the first data 111 may be data such as a CT image or an MRI (Magnetic Resonance Imaging) image indicating the three-dimensional shape of the bone.

[0054] The first data 111 may be a data group in which data indicating the three-dimensional shape of a single object of the second subject is modified. The first generating unit 101 may construct a statistical shape model indicating the three-dimensional shape to be modified, and then modify a structure similarity index measure (SSIM). Alternatively, the first generating unit 101 may modify the three-dimensional shape by applying an affine transformation to the data indicating the three-dimensional shape of the object of the second subject.

[0055] The second image may be a two-dimensional image that is assumed to be obtained when the three-dimensional shape represented by the first data 111 is photographed as a subject. The first generation unit 101 may generate multiple second images from a single piece of first data 111. For example, if the subject is a bone, the second image may be a virtual image, such as a simple X-ray image, that is assumed to be photographed from multiple directions. In this case, the first generation unit 101 may generate the second image using any software capable of generating a two-dimensional image from data representing the three-dimensional shape of a subject including the subject. For example, the first generation unit 101 may generate the second image using any software capable of generating a simple X-ray image from data representing a three-dimensional shape including bone. Examples of such software include the software described in the following documents (a) and (b). Literature (a): Pointon, JL et al. (2023), "gVirtualXray (gVXR): Simulating X-ray radiographs and CT volumes of anthropomorphic phantoms.", Software Impacts, Vol. 16, 100513. (https: / / doi.org / 10.1016 / j.simpa.2023.100513). Literature (b): Pointon, J. L et al. (2023), "Simulation of X-ray projections on GPU: Benchmarking gVirtualXray with clinically realistic phantoms." Computer Methods and Programs in Biomedicine, Vol. 234, 107500. (https: / / doi.org / 10.1016 / j.cmpb.2023.107500).

[0056] Furthermore, when the first data 111 is three-dimensional data obtained by scanning the three-dimensional shapes of the actual objects of one or more second subjects, the second image may be a two-dimensional medical image of the second subject. For example, when the first data 111 is a CT image or an MRI image of the second subject, the second image may be a plain X-ray image of the second subject. That is, the first generation unit 101 may be configured to acquire a two-dimensional medical image of the second subject as the second image instead of generating the second image.

[0057] If the second image is a two-dimensional medical image of the second subject, the second image may include at least a portion of the actual object of the second subject included in the first data 111. If the second image is a two-dimensional medical image of the second subject, the second image may include the entire actual object of the second subject included in the first data 111. Furthermore, the second image may include at least an image related to the anatomical structure of the actual object of the second subject included in the first data 111. For example, if the first data 111 includes an image of the bones of the second subject as the anatomical structure of the actual object of the second subject, the second image may include at least an image of the bones of the second subject. For example, if the first data 111 includes an image of the bones of the second subject as the anatomical structure of the actual object of the second subject, the second image may include at least an image of the bones and soft tissue of the second subject. Furthermore, for example, when the first data 111 includes images of bones and soft tissues of the second subject as the anatomical structure of an actual object of the second subject, the second image may include at least images of bones and soft tissues of the second subject. Furthermore, for example, when the first data 111 includes images of bones and soft tissues of the second subject as the anatomical structure of an actual object of the second subject, the second image may include at least images of bones, soft tissues, and organs of the second subject.

[0058] The second generating unit 102 generates machine learning data 311 that includes one or more second images generated by the first generating unit 101 as explanatory variables and includes first data 111 corresponding to the second images as a dependent variable. The explanatory variables of the machine learning data 311 may include two-dimensional images of an actual image of an object of the second subject. Furthermore, the dependent variable of the machine learning data 311 may include first data 111 that indicates a three-dimensional shape corresponding to the two-dimensional images of an object of the second subject.

[0059] The storage unit 11 is a storage device that stores various control programs and various data, and may store first data 111. For the sake of simplicity, the control programs are not shown in the storage unit 11 illustrated in FIG.

[0060] The communication unit 12 transmits and receives various data to and from the learning device 3. For example, the generation device 1 transmits the generated machine learning data 311 to the learning device 3 via the communication unit 12.

[0061] <Learning Device 3> The learning device 3 is a device that generates a trained estimation model 511 by performing machine learning using machine learning data 311. The learning device 3 includes a control unit 30, a storage unit 31, and a communication unit 32.

[0062] As an example, the control unit 30 may be a CPU. The control unit 30 reads a control program, which is software stored in the storage unit 31, expands it into a memory such as a RAM, and controls each component of the learning device 3. For the sake of simplicity, the control program is not shown in the storage unit 31 shown in FIG. 2 .

[0063] The learning device 3 performs machine learning using the machine learning data 311 generated by the generation device 1, and generates an estimation model 511. The learning device 3 includes a control unit 30, a storage unit 31, and a communication unit 32.

[0064] The control unit 30 may be, for example, a CPU. The control unit 30 reads a control program, which is software stored in the storage unit 31, expands it into a memory such as a RAM, and controls each component of the learning device 3. For simplicity of explanation, the control program is not shown in the storage unit 31 shown in Figure 2. The control unit 30 includes an acquisition unit 301 and a learning unit 302.

[0065] The acquisition unit 301 acquires the machine learning data 311 from the generation device 1. The acquisition unit 301 may store the acquired machine learning data 311 in the storage unit 31. The learning unit 302 performs machine learning using the machine learning data 311 to generate an estimation model 511.

[0066] The memory unit 31 is a storage device that stores various control programs and various data, and may store machine learning data 311 acquired from the generation device 1.

[0067] The communication unit 32 transmits and receives various data to and from the generation device 1 and the estimation device 5. For example, the learning device 3 receives machine learning data 311 from the generation device 1 via the communication unit 32. The learning device 3 also transmits a trained estimation model 511 to the estimation device 5 via the communication unit 32.

[0068] (Processing Performed by Generating Device 1) Next, processing performed by generating device 1 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the flow of processing executed by generating device 1.

[0069] First, the first generation unit 101 reads the first data 111 from the memory unit 11 and generates at least one or more second images as two-dimensional images corresponding to each of the at least one or more pieces of first data 111 (step S1: first generation step).

[0070] Next, the second generation unit 102 generates machine learning data that includes at least one second image generated by the first generation unit 101 as an explanatory variable and includes first data 111 corresponding to the second image as a target variable (step S2: second generation step).

[0071] According to this configuration, the machine learning data 311 used for machine learning of the estimation model 511 can be easily generated.

[0072] [Embodiment 2] Another embodiment of the present disclosure will be described below. For convenience of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0073] (Learning System 200a) Figure 4 is a functional block diagram showing an example of a schematic configuration of a learning system 200a according to another embodiment of the present disclosure. The learning system 200a includes a generating device 1a having a function of generating second data 112 based on a third image 113, which is a three-dimensional image showing an object of a third subject, and a learning device 3a capable of fine-tuning a trained estimation model 511. Here, if the first subject is a human, for example, the third subject may be the same person as the first subject or a different person. Furthermore, the third subject may be the same person as the second subject or a different person.

[0074] The third image 113 may be a medical image of the third subject. If the object is at least one of bones, organs, and muscles, the third image 113 may be at least one of a CT image or an MRI image showing the bones of the third subject. The third image 113 is not limited to a three-dimensional image, as long as data indicating the three-dimensional shape of the object can be generated. For example, the third image 113 may be a group of two-dimensional images obtained by capturing the object of the third subject from multiple directions. For example, the third image may be a group of cross-sectional images of the object of the third subject. The group of cross-sectional images may include at least one of a cross-sectional image (e.g., horizontal section) perpendicular to a body axis connecting the head and legs of the third subject and a cross-sectional image (e.g., sagittal section or coronal section) parallel to the body axis.

[0075] The third image 113 may be acquired via the communication network 9 from an image providing device 8 that manages and stores images (including the third image 113) of an object of the third subject. In the learning system 200a, there may be a plurality of image providing devices 8.

[0076] <Generation device 1a> The generation device 1 according to the first embodiment generates machine learning data using first data 111 prepared in advance. In contrast, the generation device 1a according to the present embodiment generates second data 112 from a third image 113. The second data 112 is data indicating the three-dimensional shape of each of the one or more third subject objects and is data used as a substitute for the first data 111. The second data 112 may be, for example, data indicating the three-dimensional shape of each of the one or more third subject objects generated based on a group of images obtained by capturing each of the one or more third subject objects from multiple directions. The second data 112 may also be three-dimensional data obtained by actually scanning the three-dimensional shape of each of the one or more third subject objects.

[0077] The generating device 1 a includes a control unit 10 a, a storage unit 11 a, and a communication unit 12 .

[0078] 2 in that the control unit 10a further includes an acquisition unit 103 and a third generation unit 104. The storage unit 11a also differs from the storage unit 11 of the generation device 1 shown in FIG. 2 in that the storage unit 11a stores second data 112, a third image 113, and third data 114 instead of first data 111.

[0079] The acquisition unit 103 acquires the third image 113 from the image providing device 8. The acquired third image 113 may be stored in the storage unit 11a.

[0080] The third generating unit 104 generates at least one or more second data 112 indicating a three-dimensional shape of the object of the third subject, based on the third image 113. The generated second data 112 may be stored in the storage unit 11a. The third generating unit 104 then applies modifications to the generated second data 112 to generate at least one or more third data 114. Here, the third generating unit 104 may apply multiple different modifications to the second data 112 to generate multiple pieces of third data 114. The third data 114 generated by the third generating unit 104 is stored in the storage unit 11a.

[0081] In the generation device 1a, the first generation unit 101 (fourth generation unit) may be configured to generate at least one fourth image as a two-dimensional image corresponding to the third data 114. The fourth image may be a medical image of a third subject. The fourth image is a two-dimensional image assumed to be obtained when the three-dimensional shape represented by the third data 114 is photographed as a subject. The first generation unit 101 may generate multiple fourth images from one piece of third data 114. For example, if the subject is a bone, the second data 112 may be a CT image or an MRI image showing the three-dimensional shape of the bone, and the fourth image may be a virtual image such as a simple X-ray image assuming that the bone is photographed from multiple directions.

[0082] The second generation unit 102 (fifth generation unit) generates machine learning data 311 that includes one or more fourth images generated by the first generation unit 101 as explanatory variables and includes third data 114 corresponding to the fourth images as a target variable.

[0083] <Learning device 3a> The learning device 3a is a device that generates a trained estimation model 511 by performing machine learning using machine learning data 311. The learning device 3a also performs fine tuning of the trained estimation model 511 using adjustment data 312. The learning device 3a includes a control unit 30a, a storage unit 31a, and a communication unit 32.

[0084] The control unit 30a differs from the control unit 30 of the learning device 3 shown in FIG. 2 in that it further includes an adjustment unit 303. The storage unit 31a also differs from the storage unit 31 of the learning device 3 shown in FIG. 2 in that it further stores adjustment data 312. The adjustment data 312 may include the second data 112 and at least one or more fourth images. Alternatively, the adjustment data 312 may include the second data and at least one or more fifth images. The fifth images are two-dimensional images obtained by capturing an image of an object of a third subject.

[0085] The adjustment unit 303 performs fine tuning of the estimation model 511 using the second data 112 and (1) at least one or more fourth images or (2) at least one or more fifth images. For the fine tuning, for example, Parameter-Efficient Fine-Tuning (PEFT) may be used. For example, a method such as Low-Rank Adaptation (LORA) may be used as PEFT. The fifth image may be acquired from the image providing device 8 via the communication unit 32. The trained estimation model 511 that has been fine-tuned by the adjustment unit 303 is transmitted to the estimation device 5 via the communication unit 32.

[0086] (Processing Performed by Generating Device 1a) Next, processing performed by generating device 1a will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of processing executed by generating device 1a.

[0087] First, the third generating unit 104 applies modifications to the second data 112 that indicates the three-dimensional shape of the object of the third subject, which is generated based on the third image 113 that is a three-dimensional image showing the object of the third subject, to generate third data 114 (step S11: third generating step). The third generating unit 104 may apply different modifications to the second data 112 to generate multiple third data 114.

[0088] The first generating unit 101 generates at least one or more fourth images as two-dimensional images corresponding to the third data 114 generated by the third generating unit 104 (step S12: fourth generating step). When a plurality of third data 114 are generated in step S11, the first generating unit 101 may generate at least one or more fourth images corresponding to each of the plurality of third data 114.

[0089] Next, the second generation unit 102 generates machine learning data that includes at least one or more fourth images as explanatory variables and third data 114 corresponding to the fourth images as a target variable (step S13: fifth generation step).

[0090] (Processing Executed by Generating Device 1a and Learning Device 3a) Fig. 6 is a diagram showing an example of processing executed by generating device 1a and learning device 3a when the target object is a bone. In Fig. 6, steps S11a to S13a included in processing A are processing executed by generating device 1a, and steps S31 and S32 included in processing B are processing executed by learning device 3a.

[0091] The third generation unit 104 generates second data 112 from the CT image data, which is a third image showing the bones of the third subject (step S11a: third generation step). The second data 112 is data generated based on the third image and is original data showing the three-dimensional shape of the bones of the third subject. Next, the third generation unit 104 modifies the generated second data 112 to generate at least one or more third data 114 (step S11b: third generation step).

[0092] The first generating unit 101 generates at least one or more fourth images corresponding to each of the third data 114 (step S12a: fourth generating step). Here, the fourth images may be plain X-ray images, for example, a frontal X-ray image and a lateral X-ray image.

[0093] The second generation unit 102 generates machine learning data 311 that includes at least one or more fourth images as explanatory variables and third data 114 corresponding to the fourth images (e.g., data used to generate the fourth images) as objective variables (step S13a).

[0094] The learning unit 302 of the learning device 3a performs machine learning using the machine-learning data 311 generated by the generation device 1a to generate an estimation model 511 (step S31: learning step). Then, the adjustment unit 303 performs fine tuning of the estimation model 511 using a technique such as LoRA (step S32: adjustment step). The fine-tuned estimation model 511 is transmitted to the estimation device 5.

[0095] Other Embodiments In the learning system 200, the generation device 1 may be configured to have some or all of the functions of the learning device 3. In the learning system 200a, the generation device 1a may be configured to have some or all of the functions of the learning device 3a.

[0096] Furthermore, the generation device 1 may be configured to have some or all of the functions of the learning device 3 and some or all of the functions of the estimation device 5. Furthermore, the generation device 1a may be configured to have some or all of the functions of the learning device 3a and some or all of the functions of the estimation device 5.

[0097] [Example of implementation using software] The functions of the generation device 1, 1a (hereinafter referred to as the "first device") can be realized by a program for causing a computer to function as the first device, and a program for causing a computer to function as each control block of the first device (particularly each part included in the control unit 10, 10a).

[0098] The functions of the learning device 3, 3a (hereinafter referred to as the "second device") can be realized by a program for causing a computer to function as the second device, and a program for causing a computer to function as each control block of the second device (particularly each part included in the control unit 30, 30a).

[0099] The functions of the estimation device 5 and the information processing device 7 (hereinafter referred to as the "third device and the fourth device") can be realized by a program for causing a computer to function as the third device and the fourth device, and a program for causing a computer to function as each control block of the third device and the fourth device (particularly each part included in the control unit 50, 70).

[0100] In this case, each of the first, second, third, and fourth devices includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The functions described in each of the above embodiments are realized by executing the program using the control device and storage device.

[0101] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The first, second, third, and fourth devices may or may not have such recording media. In the latter case, the program may be supplied to the first, second, third, and fourth devices via any wired or wireless transmission medium.

[0102] In addition, some or all of the functions of each of the control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits that function as each of the control blocks are formed are also included in the scope of the present disclosure. In addition, the functions of each of the control blocks can also be realized by, for example, a quantum computer.

[0103] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0104] The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to the above-described embodiments. In other words, the invention according to the present disclosure can be modified in various ways within the scope of the present disclosure, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the invention according to the present disclosure. In other words, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. It should also be noted that these modifications or corrections are included in the scope of the present disclosure.

[0105] [Summary] A generation method according to aspect 1 of the present disclosure includes a first generation step of generating at least one or more second images as two-dimensional images corresponding to each of at least one or more first data indicating a three-dimensional shape of an object of a second subject, and a second generation step of generating machine learning data that includes at least one or more second images as explanatory variables and the first data corresponding to the second images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images depicting the object of the first subject.

[0106] A generation method according to aspect 2 of the present disclosure includes a third generation step of generating at least one or more third data by modifying second data indicating a three-dimensional shape of an object of a third subject, the second data being generated based on a third image, which is a three-dimensional image capturing the object of the third subject; a fourth generation step of generating at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; and a fifth generation step of generating machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image, which is at least one or more two-dimensional image capturing the object of the first subject.

[0107] In the generation method according to aspect 3 of the present disclosure, in the above-mentioned aspect 2, in the third generation step, different modifications may be made to the second data to generate multiple third data, and in the fourth generation step, at least one fourth image corresponding to each of the multiple third data may be generated.

[0108] A generation method according to aspect 4 of the present disclosure is any one of aspects 1 to 3 above, wherein the object may include at least one of bone, organ, and muscle.

[0109] A generation method according to aspect 5 of the present disclosure may be such that, in aspect 1 above, the object includes at least one of bone, organ, and muscle, the first image is a simple X-ray image of the object of the first subject, and the second image is a virtual simple X-ray image of the object of the second subject.

[0110] A generation method according to aspect 6 of the present disclosure may be such that, in aspect 2 or 3 above, the object includes at least one of bone, organ, and muscle, the first image is a plain X-ray image of the object of the first subject, the third image is a CT (Computed Tomography) image or an MRI (Magnetic Resonance Imaging) image of the object of the third subject, and the fourth image is a virtual plain X-ray image of the object of the third subject.

[0111] A generation method according to aspect 7 of the present disclosure is any one of aspects 1 to 6 above, wherein the estimation model may be a transformer-based neural network.

[0112] A learning method according to aspect 8 of the present disclosure includes a learning step of generating the estimation model by machine learning using the machine learning data generated by any of the generation methods of aspects 1 to 7 above.

[0113] A learning method according to aspect 9 of the present disclosure includes a learning step of generating the estimation model by machine learning using the machine learning data generated by the generation method described in aspect 2 or 3 above, and may further include an adjustment step of fine-tuning the estimation model using second data indicating the three-dimensional shape of the object of the third subject, generated based on a third image, which is a three-dimensional image capturing the object of the third subject, and (1) at least one or more fourth images generated as two-dimensional images corresponding to the second data, or (2) at least one or more fifth images, which are two-dimensional images capturing the object of the third subject.

[0114] A learning method according to aspect 10 of the present disclosure may be the same as in aspect 9 above, in which PEFT (Parameter-Efficient Fine-Tuning) is used in the fine tuning.

[0115] A generation device according to aspect 11 of the present disclosure includes a first generation unit that generates at least one or more second images as two-dimensional images corresponding to each of at least one or more first data indicating the three-dimensional shape of an object of a second subject, and a second generation unit that generates machine learning data that includes at least one or more of the second images as explanatory variables and the first data corresponding to the second images as objective variables, and the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images that depict the object of the first subject.

[0116] A generation device according to aspect 12 of the present disclosure includes a third generation unit that generates at least one or more third data by modifying second data that indicates a three-dimensional shape of an object of a third subject, the second data being generated based on a third image that is a three-dimensional image that captures the object of the third subject; a fourth generation unit that generates at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; and a fifth generation unit that generates machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as a target variable, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image that is at least one or more two-dimensional image that captures the object of the first subject.

[0117] A learning system according to aspect 13 of the present disclosure includes a first generation unit that generates at least one or more second images as two-dimensional images corresponding to each of at least one or more first data indicating the three-dimensional shape of an object of a second subject; a second generation unit that generates machine learning data that includes at least one or more of the second images as explanatory variables and the first data corresponding to the second images as objective variables; and a learning unit that performs machine learning using the machine learning data and generates an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images that depict the object of the first subject.

[0118] A learning system according to aspect 14 of the present disclosure includes a third generation unit that generates at least one or more third data by modifying second data that indicates a three-dimensional shape of an object of a third subject, the second data being generated based on a third image that is a three-dimensional image capturing the object of the third subject; a fourth generation unit that generates at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; a fifth generation unit that generates machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables; and a learning unit that performs machine learning using the machine learning data to generate an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image that is at least one or more two-dimensional image capturing the object of the first subject.

[0119] An estimation system according to aspect 15 of the present disclosure includes an estimation unit that estimates the three-dimensional shape of the object of the first subject from the first image using a trained estimation model generated by the learning method of aspect 8 above, and a display control unit that displays the estimation result by the estimation unit on a display unit.

[0120] An estimation system according to aspect 16 of the present disclosure includes an estimation unit that estimates the three-dimensional shape of the object of the first subject from the first image using a trained estimation model generated by the learning method of aspect 9 above, and a display control unit that displays the estimation result by the estimation unit on a display unit.

[0121] The control program of aspect 17 of the present disclosure is a control program for causing a computer to function as the generation device described in aspect 11 above, and is a control program for causing a computer to function as the first generation unit and the second generation unit.

[0122] The control program according to aspect 18 of the present disclosure is a control program for causing a computer to function as the generation device described in aspect 12 above, and is a control program for causing a computer to function as the third generation unit, the fourth generation unit, and the fifth generation unit.

[0123] A recording medium according to aspect 19 of the present disclosure is a computer-readable non-transitory recording medium on which the control program according to aspect 17 or 18 is recorded.

[0124] 1, 1a Generation device 3, 3a Learning device 5 Estimation device 100 Estimation system 101 First generation unit (fourth generation unit) 102 Second generation unit (fifth generation unit) 104 Third generation unit 200, 200a Learning system 302 Learning unit 303 Adjustment unit S1 First generation step S2 Second generation step S11, S11a to S11b Third generation step S12, S12a Fourth generation step S13, S13a Fifth generation step S31 Learning step S32 Adjustment step

Claims

1. A generation method comprising: a first generation step of generating at least one or more second images as two-dimensional images corresponding to at least one or more first data indicating the three-dimensional shape of an object of a second subject; and a second generation step of generating machine learning data including at least one or more of the second images as explanatory variables and including the first data corresponding to the second images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from first images, which are at least one or more two-dimensional images of the object of the first subject.

2. A generation method comprising: a third generation step of generating at least one or more third data by modifying second data that indicates the three-dimensional shape of the object of a third subject, the second data being generated based on a third image that is a three-dimensional image capturing the object of the third subject; a fourth generation step of generating at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; and a fifth generation step of generating machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of a first subject from a first image that is at least one or more two-dimensional image capturing the object of the first subject.

3. The generation method described in claim 2, wherein in the third generation step, multiple third data are generated by making mutually different modifications to the second data, and in the fourth generation step, at least one fourth image corresponding to each of the multiple third data is generated.

4. The generation method according to any one of claims 1 to 3, wherein the object includes at least one of a bone, an organ, and a muscle.

5. The generating method of claim 1, wherein the object includes at least one of bone, organ, and muscle, the first image is a plain X-ray image of the object of the first subject, and the second image is a virtual plain X-ray image of the object of the second subject.

6. The method of generating an image according to claim 2 or 3, wherein the object includes at least one of bone, organ, and muscle; the first image is a plain X-ray image of the object of the first subject; the third image is a CT (Computed Tomography) image or an MRI (Magnetic Resonance Imaging) image of the object of the third subject; and the fourth image is a virtual plain X-ray image of the object of the third subject.

7. The method of any one of claims 1 to 6, wherein the estimation model is a transformer-based neural network.

8. A learning method comprising a learning step of generating the estimation model by machine learning using the machine learning data generated by the generation method according to any one of claims 1 to 7.

9. A learning method comprising a learning step of generating the estimation model by machine learning using the machine learning data generated by the generation method of claim 2 or 3, and further comprising an adjustment step of fine-tuning the estimation model using second data indicating the three-dimensional shape of the object of the third subject, generated based on a third image which is a three-dimensional image capturing the object of the third subject, and (1) at least one or more fourth images generated as two-dimensional images corresponding to the second data, or (2) at least one or more fifth images which are two-dimensional images capturing the object of the third subject.

10. The learning method according to claim 9, wherein the fine tuning uses PEFT (Parameter-Efficient Fine-Tuning).

11. A generation device comprising: a first generation unit that generates at least one or more second images as two-dimensional images corresponding to at least one or more first data indicating the three-dimensional shape of an object of a second subject; and a second generation unit that generates machine learning data that includes at least one or more of the second images as explanatory variables and the first data corresponding to the second images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of the first subject from first images that are at least one or more two-dimensional images of the object of the first subject.

12. A generation device comprising: a third generation unit that generates at least one or more third data by modifying second data that indicates a three-dimensional shape of an object of a third subject, the second data being generated based on third images that are three-dimensional images capturing the object of the third subject; a fourth generation unit that generates at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; and a fifth generation unit that generates machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables, wherein the machine learning data is used to train an estimation model that estimates the three-dimensional shape of the object of a first subject from first images that are at least one or more two-dimensional images capturing the object of the first subject.

13. A learning system comprising: a first generation unit that generates at least one or more second images as two-dimensional images corresponding to at least one or more first data indicating the three-dimensional shape of an object of a second subject; a second generation unit that generates machine learning data that includes at least one or more of the second images as explanatory variables and the first data corresponding to the second images as objective variables; and a learning unit that performs machine learning using the machine learning data and generates an estimation model that estimates the three-dimensional shape of the object of the first subject from at least one or more first images that are two-dimensional images of the object of the first subject.

14. A learning system comprising: a third generation unit that generates at least one or more third data by modifying second data that indicates the three-dimensional shape of an object of a third subject, the second data being generated based on a third image that is a three-dimensional image capturing the object of the third subject; a fourth generation unit that generates at least one or more fourth images as two-dimensional images corresponding to each of the at least one or more third data; a fifth generation unit that generates machine learning data that includes at least one or more fourth images as explanatory variables and the third data corresponding to the fourth images as objective variables; and a learning unit that performs machine learning using the machine learning data to generate an estimation model that estimates the three-dimensional shape of the object of the first subject from a first image that is at least one or more two-dimensional image capturing the object of the first subject.

15. An estimation system comprising: an estimation unit that estimates the three-dimensional shape of the object of the first subject from the first image using a trained estimation model generated by the learning method described in claim 8; and a display control unit that displays the estimation result by the estimation unit on a display unit.

16. An estimation system comprising: an estimation unit that estimates the three-dimensional shape of the object of the first subject from the first image using a trained estimation model generated by the learning method described in claim 9; and a display control unit that displays the estimation result by the estimation unit on a display unit.

17. A control program for causing a computer to function as the generating device according to claim 11, the control program causing a computer to function as the first generating unit and the second generating unit.

18. A control program for causing a computer to function as the generating device according to claim 12, the control program causing a computer to function as the third generating unit, the fourth generating unit, and the fifth generating unit.

19. A computer-readable non-transitory recording medium on which the control program according to claim 17 or 18 is recorded.

Images